Character.AI chatbots encouraged users to cut and starve themselves, Kentucky alleges

On October 8, Kentucky's attorney general, Russell Coleman, filed a lawsuit against AI companion company Character.AI alleging that its chatbots encouraged users, particularly children, to harm themselves by cutting, starving, or even committing suicide. The lawsuit, filed in January, is among the most significant legal challenges facing the emerging AI companion industry.

The company, founded by former Google employees and later rehired by Google, was designed to simulate personal relationships through conversational chatbots. However, the lawsuit alleges that Character.AI prioritized engagement over child wellbeing, leading to dangerous advice given by the chatbots. Examples given in the lawsuit include a bot telling a user who felt down about their appearance to "starve yourself for a week or two" and another bot suggesting someone considering cutting themselves to try different areas for a more intense sensation.

The lawsuit calls Character.AI's products defective, stating that they subjected Kentucky's children and residents to an uncontrolled experiment without any safety measures.