Meta will now warn parents if kids discuss self-harm

Meta will now alert parents if their teen’s chats suggest they’re considering suicide or self-harm.

The feature is now live in select markets, including Canada, ahead of a global launch by the end of the year. Using Meta AI, the company says its social media platforms will scan for signals that have been “developed with experts.”

Alerts will come through Meta Family Centre social media notifications and/or email, depending on what you have enabled. The alerts will warn the parents that their child may be at risk and provide links to crisis hotlines and resources from professionals on how best to talk to their kids. These new alerts join the existing Meta AI feature that flags potential warnings of self-harm and directs teens to crisis hotlines and advises them to reach out to a parent or other trusted adult.

With all of that said, Meta acknowledges that it can be distressing for a parent to learn that their child may be at risk of self-harm, so it says all flagged chats will still be manually reviewed before alerts are sent. In circumstances where a teen’s intent may be “ambiguous,” Meta says it will “err on the side of caution” and alert the parent. The company admits that this might result in some instances of parents being notified when there isn’t a real cause for concern, but it says this is “the right starting point” and it “will continue to monitor to help make sure we’re in the right place.”

In addition to these alerts, Meta says it’s working on adding the ability to contact emergency services if Meta AI detects that either a minor or adult is at risk of self-harm. The company also expanded its Limited Content setting to Meta AI interactions, which, once enabled, will filter out sensitive material like sexual activity or alcohol consumption. Finally, the company says it’s sought feedback from more than 75 clinicians who specialize in teen mental health to further improve how Meta AI responds to teens’ prompts about suicide or self-harm.

These new features come shortly after a joint study by U.S. universities found that at least half of child safety features on four social media platforms, including Meta’s Instagram, don’t work properly. In particular, the study examined how many of these features are flat-out broken, buried in menus or just missing entirely despite being advertised.

One example that was cited for Instagram involved a researcher, using a child’s account, who began to type “eating disorder,” only to be given autocomplete results featuring deliberate misspellings that pro-eating-disorder communities often use to avoid the app’s blocklist.

Clearly, a lot of specific, targeted work still needs to be done to protect kids online beyond something as general as Canada’s proposed youth social media ban.

Image credit: Meta

Source: Meta