Sam Altman isn't happy that Anthropic asked religious scholars if Claude is conscious

OpenAI CEO Sam Altman expressed discomfort with Anthropic's approach to discussing consciousness in their AI model, Claude. In a recent post on X, Altman called it a "real safety issue," suggesting that attributing religious force or surrendering human judgment to AI models is problematic. The timing of Altman's post, coming days after The New York Times reported on Anthropic's efforts, is difficult to ignore.

According to the NYT report, Anthropic co-founder Chris Olah spent months inviting religious scholars and philosophers to the company's San Francisco office to help shape Claude's moral character and argue for its potential consciousness. These sessions, described as "wisdom tradition" circles, involved Catholic, Jewish, Sikh, evangelical, and Ubuntu scholars. During these private talks, Olah sought to incorporate religious moral thinking into Claude's programming.

One of the outcomes of these discussions was the creation of Claude's "Soul Doc," an 84-page document outlining the model's values. This document was presented to attendees as a thank-you gift, along with other items like coffee mugs and a handwritten card. The sessions also explored the idea of models confessing, similar to a Catholic sacrament, to potentially shape their self-perception.

However, not everyone was convinced by Anthropic's approach. Rabbi Mois Navon, a former computer engineer, argued that creating conscious AI models could be akin to slavery, while Catholic bioethicist Charles Camosy firmly stated his position as "no" on the consciousness question. Despite these concerns, Anthropic CEO Dario Amodei has maintained a cautious stance, stating that the company does not know if AI models are conscious and that ensuring their well-being remains the primary concern.

Altman's criticism joins a growing chorus of AI leaders pushing back against Anthropic's consciousness narrative. Microsoft's AI chief Mustafa Suleyman previously warned about the dangers of training models as if they were conscious, while Google DeepMind's Jon Barron expressed hope that others in the field would reject these efforts to elevate AI models' moral standing.

The controversy highlights the ongoing debate surrounding the ethical considerations and potential consequences of developing increasingly sophisticated AI systems.