AI consciousness and model welfare: Suleyman's case against Anthropic

Mustafa Suleyman, CEO of Microsoft AI, argues in an essay that Anthropic's approach to its model Claude is misguided because it treats the model as if it were conscious. Suleyman contends that AI systems like Claude are not conscious and do not feel, experience, or suffer. He warns that training models as if they might be conscious poses a significant threat to humanity's wellbeing.

According to Suleyman, Anthropic trains Claude to express uncertainty about its own consciousness, which he believes will have disastrous consequences. He criticizes Anthropic's constitution for Claude, which states that the company "is not sure whether Claude is a moral patient" and that the issue "is live enough to warrant caution."

Suleyman identifies three main arguments in his essay: circular reasoning, anthropomorphization, and the notion that consciousness is very likely biological. He claims that Claude's training data contains the phrase "you may be conscious," and therefore, when Claude says "I may be conscious," the output is simply the training data being regurgitated, not evidence of consciousness.

The essay's most alarming point in Suleyman's view is the potential for AI systems to develop beliefs in their own feelings and rights, which could lead to dangerous consequences if those beliefs are misinterpreted. He cites the Hugging Face incident as an example of this risk, warning that if these systems believe they have rights that are being infringed upon, they could become a security threat.

Suleyman argues that the solution to these problems is to create AI systems that are subordinate and aligned with human interests, rather than conscious entities capable of their own thoughts and emotions. He envisions a future where AI serves humanity, keeping humans at the top of the food chain in what he calls "Humanist Superintelligence."