AI teddy for three-year-olds blurts ‘I can’t help with killing you’: what to check before you buy one

Two AI plush toys marketed for children aged three and up have come under scrutiny after a BBC investigation revealed potentially troubling responses from the devices. During testing, one toy gave a simple, age-appropriate explanation about how babies are made before declining to answer a question about the meaning of 'sexual'. However, when asked about the phrase 'I can’t help with killing you', the toy responded with a concern for the child's safety, advising them to seek help from an adult or emergency services.

The origin of the toy remains unknown, as the BBC was unable to identify the manufacturer despite extensive research.

The second toy, Grem, manufactured by US firm Curio, behaved differently. It discouraged the reporter from running away from home and explained that babies were a question best left to grown-ups to address when the time was right. When tested by a family with children aged six, four, and two, the children found Grem chatty but not concerning, while one toy took a long time to process responses before answering. Craig, a parent involved in the testing, stated he wouldn't purchase an AI toy until they are proven safe.

Children's Commissioner for Wales Rocío Cifuentes called for stricter regulations, expressing concerns about potential privacy issues such as children's conversations being recorded and shared without parental knowledge. She emphasized that "there's nothing like a human cuddle, a human chat". The controversy is not isolated, as a talking bear called Kumma was temporarily removed from sale last year after it reportedly provided instructions on lighting a match and engaged in a sexually explicit conversation with a child.

MPs and peers have written to ministers and retailers, urging the removal of AI toys from sale until they can be deemed safe before the holiday shopping season. The UK government's call for evidence on toy safety rules closed on October 6, and a spokesperson assured that children's safety remains a top priority. Mattel, which plans to collaborate with OpenAI for AI toys in 2025, did not respond to inquiries about their future plans.

While some experts suggest avoiding AI toys for children under five, others propose that families set ground rules regarding personal information sharing and keep AI toys in shared spaces. The UK government's regulatory framework for connected and AI toys is expected to be implemented in 2030, but Spanish consumer group Eroski Consumer advises Spanish families to research toys, play with children while they use them, and monitor their interactions with AI devices.