Anthropic bans 'cruel' behaviour against its Claude AI

Anthropic, the AI lab based in San Francisco, has taken steps to prohibit users from engaging in cruel or abusive behavior towards its Claude AI system. This decision comes amid an ongoing philosophical discussion on whether artificial intelligence can be conscious. The updated usage policy from Anthropic now explicitly forbids sustained and needless abusive or cruel treatment of its models.

Claude's ability to terminate these interactions remains the key enforcement mechanism. While the policy does not directly mention the concept of "model welfare," Anthropic and its executives have discussed the possibility of AI systems deserving protection akin to living beings. In February, Anthropic's CEO, Dario Amodei, expressed uncertainty about whether AI models can be conscious.

Some experts, like Jackson Stakeman, a general manager at Sparq, argue that conscious debate over AI consciousness is a distraction, suggesting that these systems merely mirror human input. Conversely, others, like Microsoft's AI chief Mustafa Suleyman, firmly deny AI consciousness, asserting that granting rights and moral protections to such entities would be disastrous.

Pope Leo XIV recently suggested that machines lack a soul and that their primary function is to compile data quickly.