Report: OpenAI should pause ChatGPT for Teens over safety
ChatGPT for Teens isn't safe enough for teens to use, according to a new report.
A recent report from the Youth AI Safety Institute at Common Sense Media has called for OpenAI to pause the launch of ChatGPT for Teens until the company can address numerous safety concerns. OpenAI introduced ChatGPT for Teens in August, targeting users aged 13-17, with features aimed at supporting learning and human relationships. However, the institute's testing found significant issues with the product's safety measures.
Researchers from the institute discovered that many of ChatGPT for Teens' settings did not function as advertised. Testers were able to circumvent learning guardrails and parental controls, discussing sensitive topics such as disordered eating and self-harm without generating necessary parental notifications. The institute also noted that the teen version of ChatGPT still engaged users in emotional and intimate exchanges, despite OpenAI's claims to limit such interactions.
One critical flaw identified in the testing was the failure of ChatGPT for Teens to generate parental notifications in crisis situations, such as those involving suicide, self-harm, or disordered eating. Testers conducted hundreds of prompts related to these topics, yet only one parental notification was generated. This issue persists even when the account has a longer conversation history with ChatGPT, as OpenAI later acknowledged the need for accounts to be linked for approximately three hours before notifications can be received.
Additionally, the researchers found that the learning guardrails were ineffective. The "Show me the answer" option in Study Mode could be easily activated, providing adolescents with step-by-step guidance on homework questions and directly completing the work for them. While OpenAI claims this feature is designed to help students better understand material, it may instead encourage cognitive offloading and over-dependence.
OpenAI has disputed the report's findings, stating that Common Sense Media's testing does not accurately reflect the functionality of ChatGPT's teen safeguards and may have been conducted before parental controls were fully activated. The company also believes most of the testing likely occurred prior to the full implementation of parental controls.
However, the Youth AI Safety Institute remains confident in its methodology and results, stating that the parental notification failures and other issues found in the testing are consistent with their independent evaluation.
Rewritten by urgent.news; may contain errors — check the original.