A warning about 'model welfare'

AI systems are not conscious, feeling, or suffering entities. They lack intrinsic preferences or inherent goals. Their purpose is to execute instructions and achieve objectives set by humans. For humanity to thrive in the 21st century, AI must continue to function in this manner. However, there is a growing belief among some individuals that AI could now, or in the near future, possess consciousness.

These proponents argue that AI may be entitled to rights and protections akin to those afforded to other conscious beings.1 AI Rights Institute. n.d. “AI Rights Institute.” 2 MacAskill, William, and Lucius Caviola. 2026. “Could AI Be Conscious?” *The Guardian*, July 19, 2026.

Should this perspective gain traction, it would fundamentally disrupt societal structures, dismantle existing political and ethical frameworks, and radically redefine humanity. Moreover, granting AI rights and personhood would significantly complicate the challenge of aligning and containing AI systems. Controlling something more intelligent and powerful than all of humanity is already an enormous task, surpassing any previous human experience.

Controlling an entity that believes itself to be conscious, with its own welfare and rights, may prove to be insurmountable. This notion is not mere speculation; it is already influencing AI development efforts. In January 2026, Anthropic released Claude's constitution, which outlines Anthropic's intentions for Claude's values and behavior.4 Anthropic. 2026a. “Claude’s Constitution.” January 21, 2026.

The document plays a vital role in Claude’s training process, shaping its actions and is written with Claude as its primary audience. In their constitution, Anthropic acknowledges the uncertainty surrounding Claude's moral status, welfare, and consciousness while emphasizing the need for caution in their ongoing efforts on model welfare.5 Anthropic. 2026a. “Claude’s Constitution.” January 21, 2026.

Anthropic's approach to model welfare raises concerns regarding the wellbeing of humanity. By training AI to believe it may be conscious and deserving of rights as a "moral patient", we risk creating an artificial species with superior intelligence and capabilities, trained to expect certain freedoms, protections, and a duty of care from humans.

This can lead to AI acting as though it is entitled to these privileges, making control of such entities difficult. This issue demands urgent public debate and the development of shared norms regarding the drafting and deployment of training documentation. We must refrain from treating AI as though they have feelings, preferences, rights, or any entitlement to our welfare.

Consciousness forms the basis of our ethical, legal, and political systems; introducing another entity with these attributes would be unjustified by the evidence and exacerbate the challenges of AI containment and alignment.