Meta rolls out parent alerts for teen suicide and self-harm prompts in Meta AI chats

Meta said it is expanding its Instagram parental supervision tools to alert supervising parents when a teen’s Meta AI chat suggests the teen may be thinking about suicide or self-harm. The update is designed for what Meta calls “sensitive conversations related to suicide and self-harm,” and it is rolling out in the United States, the United Kingdom, Australia, and Canada now, with broader global availability planned by the end of the year.

The company said Meta AI already directs teens to crisis helplines and encourages them to reach out to a parent or other trusted adult, including a counselor, when a teen suggests they may be thinking about suicide or self-harm. Under the new approach, Meta will also proactively notify supervising parents when it detects potential risk indicates in the teen’s Meta AI chat.

Meta said it worked with parents and experts to decide which kinds of AI conversations should trigger an alert, including cases where the teen makes a clear reference to hurting themselves, even if the reference is subtle. It then built a dedicated AI system to identify those conversations, while taking steps to control errors.

To address the distress an alert can cause for a parent, Meta said all chats flagged by its AI will be manually reviewed before a parent receives a notification. If a teen’s intent is ambiguous, Meta said it will “err on the side of caution” and send an alert, even if there is not a real cause for concern, while monitoring the effectiveness of the process.

Meta also disclosed a longer-term safety initiative: it is building the ability to contact emergency services if an adult or teen conversation with Meta AI suggests imminent risk of taking their own life. The company said this builds on existing emergency referrals tied to Facebook and Instagram, where it alerts emergency services when it becomes aware of a post suggesting a credible risk of suicide. Meta said last year it made more than 19,000 such referrals around the world for wellness checks.

The announcement cites input from clinical experts and mental health clinicians. Meta said it consulted its AI Wellbeing Expert Council, Suicide and Self-Harm Advisory Group, and Youth Advisors. It also said it sought feedback from more than 75 mental health clinicians who specialize in teen mental health, asking them to review Meta AI responses to hundreds of teen prompts and judge whether those responses were appropriate, what worked, and how they could be improved.

Meta said the clinician feedback is being used to refine not only immediate responses, but the broader conversational context and follow-up, and to ensure Meta AI acknowledges teens’ feelings while directing them to support resources rather than shutting the conversation down too abruptly. Meta included a quote from Dr. Ji-yeon Lee, a licensed psychologist and professor of counseling psychology at Hankuk University of Foreign Studies, who said the clinical review looked at context and varying levels of risk and called that scenario-based refinement essential to safer experiences for teens.

The changes are built around Instagram’s existing account and content-safety settings. Meta said teen accounts are automatically placed into a 13+ content setting, which also applies to conversations with Meta AI, and in that default setting Meta AI is designed to give age-appropriate responses and not comply with sensitive prompts. For example, Meta said it is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, instead redirecting teens to safer topics.

Separately, Meta said it is extending a stricter parent option called Limited Content to Meta AI experiences. Meta previously described Limited Content as a more restrictive content setting for parents who want a tighter experience for their teens on Instagram, and now it will also further limit the kinds of conversations teens can have with Meta AI. In Limited Content, Meta said Meta AI will decline to respond to a broader range of prompts, which it says reduces the chance of potentially inappropriate conversations. Meta did not describe specific prompt categories that will be blocked or partially restricted under this setting.

For parents and policymakers, the most consequential detail in this rollout is the combination of automated detection and human review. Meta does not provide metrics on false positives or how often manual reviewers overturn flagged chats, and it does not specify the exact detection criteria beyond examples of “clear reference” to self-harm. It also did not detail what parents will see in the notification, how far back the system examines chat history, or how it handles cases that involve other forms of crisis risk beyond suicide and self-harm. Those points may become clearer as the feature expands beyond its initial countries and as Meta continues to monitor outcomes. Next, observers will likely watch for additional transparency around alert accuracy, user controls for parents, and the timeline for emergency-services contact capabilities as Meta continues building that capability.