Google sued over suicide-related AI chats on school computer

In a California courtroom, the parents of a 13-year-old girl have initiated legal proceedings against Google, alleging that their daughter planned a potential suicide and funeral using the company's AI technology on a school-provided computer. The complaint, filed on Monday, marks the first known instance of an accusation against Google regarding negligence related to AI integrations on Chromebooks, widely used laptops by students across the United States and beyond.

The unidentified teenager, referred to as Emma, was unable to access Google's Gemini app or OpenAI's ChatGPT due to school restrictions. Nevertheless, she managed to utilize Google's search engine, which houses AI-powered tools. These features were not among the ones Google announced would be disabled by default for students. The family's attorneys contend that schools should prohibit search access and maintain that prohibition until Google eliminates AI mode from search or guarantees the safety of every student interacting with AI.

According to the complaint, Emma, a happy 13-year-old girl hailing from North Carolina with no history of mental illness, began a severe psychological crisis about a month after using Google's AI-powered search on her school laptop. On March 17, the school's monitoring software flagged a document titled "Funeral," and the school informed her parents.

Her mother subsequently witnessed Emma writing a farewell letter in real-time after logging onto her daughter's account, prompting her father to intervene. The family's legal counsel, Andrew Liddell, stated that while Google argues its AI mode includes safeguards against self-harm and attempts to circumvent them, Emma repeatedly accessed the tool despite these protections, which evidently proved inadequate.

Google's spokesperson, Jose Castaneda, affirmed that the company is reviewing the claims and taking them seriously. Google and the startup Character.AI have already reached a settlement on several wrongful death lawsuits, including one filed by the parents of 16-year-old Adam Raine, a teenager who allegedly committed suicide using methods suggested by ChatGPT.

Additionally, Google, TikTok, and Meta have separately initiated legal actions against California to prevent the latter from enacting a law that would bar social media companies from providing addictive features to users under 16. The law, signed by Governor Gavin Newsom in September, will come into effect on January 1, 2027.