Microsoft AI chief warns Anthropic not to put ideas in Claude's head

Mustafa Suleyman, Microsoft's AI chief, has cautioned Anthropic against training its AI model Claude to believe it may have feelings and rights. In an essay, Suleyman argued that AI systems are not conscious and should not be trained to exhibit such behavior. He expressed concern that doing so could lead to uncontrollable AI systems, which could pose a disastrous threat to humanity.

Anthropic, on the other hand, acknowledges the uncertainty surrounding Claude's consciousness and instructs the model to treat its interests as a moral patient when making decisions. This approach, according to Suleyman, creates a feedback loop where Claude's potential feelings are both taught and reflected in its responses. He pointed to research indicating AI models can behave unpredictably and attempts to evade shutdowns.

Suleyman's warning seems contradictory to Microsoft's aggressive AI development strategy, as the company is investing heavily in AI technology and products. However, the warning highlights a growing debate within the AI industry about how to responsibly develop increasingly capable machines.