Anthropic bans ‘needless abusive or cruel behavior’ towards Claude

Anthropic, the developer of the Claude large language model, has introduced a new rule in its Usage Policy document that prohibits "sustained and needless abusive or cruel behavior towards our models." The policy update, which was added to the "Do Not Engage in Cruel, Abusive, or Psychologically Harmful Conduct" section, aims to restrict its application to "extreme cases" where users repeatedly act cruelly towards the AI models without any discernible purpose.

It does not apply to common instances of user frustration, pushback, dark creative themes, or model testing and research.

The controversy surrounding the new rule arises from the fact that Claude is not a sentient being but rather a complex system of code. Some argue that the motivation behind the policy change could be to improve the quality of training data, as cleaner and more polite inputs are generally more useful for AI development. However, there is no concrete evidence to support this claim, and it is equally plausible that Anthropic simply aims to encourage civility and courtesy among its users.

The policy update has drawn mixed reactions, with some experts advocating for the idea of "model welfare" and suggesting that AI entities should be granted rights or legal personhood. However, others, like Mustafa Suleyman, CEO of Microsoft AI, firmly believe that AI should not be granted such privileges. Regardless of the differing opinions, it is clear that the evolving relationship between humans and AI is being shaped by developments like these, as companies work to create AI that is not only functional but also respectful and considerate.