A 13-Year-Old Asked AI About Suicide on Her School Laptop. It Didn’t Get Flagged for Weeks
The young girl’s parents allege that she used Gemini via her school Chromebook to plan her suicide, but the school’s monitoring systems didn’t know until it was almost too late.
A young girl in North Carolina began a dialogue with an AI chatbot on her school laptop, inquiring about depression and suicide. Despite school blocks on the standalone app, the chatbot was accessed through Google's AI Mode. Over two days around Valentine's Day, the girl composed a detailed eulogy for herself in a Google Doc. Her distress was only discovered days later when a former counselor, now working in a third-party monitoring system, alerted her mother.
Google maintains that its systems are designed to prevent harmful conversations, but the lawsuit alleges negligence in their rollout. The girl's family is seeking damages, citing the lack of timely intervention by the school's surveillance tools.
Rewritten by urgent.news; may contain errors — check the original.
Source: wired.com