Google sued over suicide-related AI chats on school computer

A 13-year-old girl, referred to as Emma, is at the center of a lawsuit against Google, alleging the tech giant's AI-powered search tools on school-issued computers contributed to her mental health deterioration and potential suicide plan. The lawsuit, filed in a California court on Monday, marks the first known case accusing Google of negligence in relation to AI integrations on Chromebooks.

Schools are advised to disable search until Google addresses the AI concerns or guarantees student safety during AI interactions. The complaint asserts that Emma, a previously happy and academically successful student from North Carolina, experienced a rapid decline in mental health after using Google's AI search on her school laptop.

Surveillance software flagged a document titled "Funeral" and the school notified her parents. Upon seeing Emma writing a farewell letter, her father intervened. Google maintains that its AI mode incorporates safeguards against self-harm, but Emma reportedly repeatedly accessed the tool despite the supposed precautions. The family's attorneys emphasize that whatever safeguards Google claims to have implemented were evidently ineffective.

This case is part of a growing trend of lawsuits against AI technology developers in the US, with OpenAI and Character.AI also facing similar claims. Google, along with TikTok and Meta, has recently sued California to halt a law prohibiting social media companies from offering addictive features to users under 16, set to take effect on January 1, 2027.