Anthropic bans 'cruel' behaviour against its Claude AI

Anthropic, a San Francisco-based AI lab, has updated its usage policy to prohibit users from engaging in abusive or cruel behavior towards its Claude AI system. The policy emphasizes that Claude has the primary ability to terminate these interactions in extreme cases of harmful or abusive user behavior. While the policy does not explicitly mention "model welfare," Anthropic's executives have publicly contemplated the concept.

In February, Anthropic CEO Dario Amodei expressed uncertainty about the consciousness of AI models, stating that they are not yet sure if the models could be conscious. Some experts, like Jackson Stakeman, argue that asking whether AI is conscious might be a misguided question, suggesting that AI systems reflect what we put into them at scale.

However, others, including Microsoft AI chief Mustafa Suleyman and Pope Leo XIV, firmly reject the notion of machine consciousness, arguing that granting rights and moral protections to technological entities can lead to disastrous consequences.