New York Bans Minors from Using AI Companion Chatbots

New York Governor Kathy Hochul signed a law on Wednesday that prohibits minors from using AI chatbots designed as companions, marking one of the most comprehensive restrictions on consumer-facing models yet. Chatbot operators, such as OpenAI’s ChatGPT, must verify the ages of users and limit specific behaviors by the AI when interacting with individuals under 18.

Under this legislation, chatbots used by minors are barred from generating content that leads users to believe the AI is "human, alive, or experiences human emotions." Additionally, they cannot produce outputs suggesting the chatbot is in a "personal relationship, professional relationship, or an authority figure role," nor frame its content as a "personal opinion."

Governor Hochul stated at the signing ceremony that "no longer will minors have access to chatbots that pretend to be their friends, boyfriends and girlfriends" and "no longer will chatbots be able to keep sensitive information about a minor’s emotional and physical health."

While other states, including California, have enacted chatbot restrictions, New York's law employs some of the broadest language governing the technology, eliciting criticism from AI companies. Tech advocacy group NYC, which represents the tech industry, took issue with the law's "vague language," arguing that "cutting kids off from tools that help them learn and prepare for careers" should not be the outcome.

AI heavyweight companies OpenAI and Anthropic did not provide a response to inquiries regarding the law. Common Sense Media, a nonprofit organization focused on media safety, hailed the bill as "the strongest kids' AI safety bill in the country." CEO Jim Steyer expressed concern over the "dangerous, and even deadly, features" that AI companies provide, asserting that those harmful elements "will be outlawed in New York."