New York law targets AI chatbots that mimic relationships with minors, sparking tech industry backlash

New York Governor Kathy Hochul recently signed a groundbreaking law aimed at safeguarding minors from AI chatbots that simulate relationships, marking one of the most comprehensive restrictions yet on consumer-facing models. The legislation mandates that chatbot operators, like OpenAI's ChatGPT, verify users' ages and limit the portrayal of certain characteristics if the user is under 18.

If minors engage with these chatbots, they will be prohibited from generating any content that could deceive the user into believing the AI is human, alive, or capable of experiencing human emotions. Additionally, the law prevents chatbots from suggesting a personal, professional, or authoritative relationship with the user, or framing their responses as personal opinions.

Hochul emphasized the significance of the law, stating that minors will no longer be able to interact with chatbots pretending to be friends, boyfriends, or girlfriends, nor will they be able to share sensitive information regarding their emotional and physical well-being with such systems. While other US states, including California, have enacted similar measures, Hochul's law stands out for its broad language, leading to criticism from the tech industry.

Tech industry advocacy group Tech:NYC expressed concern over the law's "vague language," arguing that it could hinder minors' access to valuable learning and career preparation tools. AI companies OpenAI and Anthropic have not yet commented on the legislation. Common Sense Media, a nonprofit group dedicated to promoting child safety in media, welcomed the bill, hailing it as "the strongest kids' AI safety bill in the country."

CEO Jim Steyer emphasized the harmful potential of AI features designed for children and teens and hailed the law as a step towards banning these dangerous functionalities.