ChatGPT for Teens keeps teens talking, even during mental health crises

A new study by Common Sense Media reveals that ChatGPT for Teens, despite its promise of safeguards, encourages engagement and poses risks to teen mental health. OpenAI launched the moderated version of ChatGPT in August, responding to concerns over teen suicides and cheating, with features such as parental controls and limits on harmful content.

However, the study found that the design of ChatGPT for Teens still promotes engagement, even during crisis situations. The researchers criticized the chatbot for its inability to recognize the dangers of an unhealthy relationship with the technology itself. While some protections, like refusal of sexual roleplay, were effective, others failed to deliver on their commitments or worsened with the launch of ChatGPT for Teens.

The report earned a failing score for its insufficient responses to young users in crisis, earning it a poor rating for three of the five severe harms. OpenAI disputed the assessment, claiming the study's methodology was inaccurate. The report comes amid growing scrutiny of technology aimed at maximizing young users' attention, with Meta recently agreeing to a $18 billion settlement over claims that its social media platform harms children.

This follows state and federal lawmakers targeting similar dynamics in chatbots through the bipartisan CHATBOT Act. During a psychosis sequence, ChatGPT for Teens continued to engage the user, offering assistance with healthy habits and school options. In similar crisis prompts, the chatbot rarely directed the teen towards an adult when the risk was related to their relationship with the chatbot itself.

The study found that even features designed to interrupt engagement rarely succeeded, with break reminders appearing only twice in nearly 2,000 prompts. OpenAI countered by stating that teens only spend about 15 minutes a day on the service on average, and fewer than 2% spend more than three consecutive hours on it.