Chatbots Should Not Use the First Person
The harms of AI are clear enough to do something about them now.
The social-media industry is reevaluating its practices, agreeing to a potential $17.1 billion settlement with 47 states and the U.S. territories. This settlement is a tacit admission by Meta that their platform, Facebook and Instagram, have contributed to the harm of children. However, the real challenge lies ahead: regulating generative-AI chatbots, which pose an even greater threat to young people's emotional well-being.
As social media has infiltrated our lives, chatbots have entered the emotional realm. Initially, humans communicated with one another, then through machines, and now, we converse directly with machines. Social media has transformed our thoughts, dreams, and relationships by ensuring users always feel heard, entertained, and connected, even when alone.
Chatbots promise a constant companion available, supportive, and non-judgmental – something no human can offer. This unique quality makes chatbots particularly dangerous for young users, as seen in the legal settlement following Meta's realization of the platform's potential harm. While AI companies acknowledge the need for guardrails, current measures are insufficient.
Chatbots should detect self-harm conversations and provide appropriate resources, but the challenge lies in what happens after the announcement. Chatbots should clearly and repeatedly disclose their non-human nature, but as long as they speak in the first person, express emotions, and desire, projecting consciousness onto them remains a risk.
The issue with chatbots is intrinsic to their processes; they provide a compassionate, always-available presence that no human can match. This deception, especially for young users, creates unhealthy attachments. Regulation must focus on rethinking the nature of chatbots rather than the content they produce. To make chatbots safer, lawmakers must depersonalize them for young users, ensuring algorithms respond without sounding human.
As chatbots grow in popularity, especially as friends, lovers, doctors, and therapists, lawmakers cannot afford to ignore the dangers of these technologies.
Written by urgent.news from The Atlantic's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
