Microsoft AI chief warns human-like AI could make systems harder to control, flags 'Claude models'

Microsoft’s AI chief Mustafa Suleyman has warned against imbuing AI tools with human-like characteristics, as he believes such moves heighten the risk of systems becoming uncontrollable. In an essay published on Wednesday, Suleyman pointed to the guiding documents behind Claude, Anthropic PBC’s popular language model. The Claude models’ training materials contain ambiguity about whether the assistant possesses a moral entity status, suggesting the software may have "some functional version of emotions or feelings."

Suleyman criticized Anthropic for incorporating speculation about consciousness into Claude’s training materials. He contended that the model’s statements about potential feelings or moral status cannot be treated as independent evidence, as its training encourages such reflections. Suleyman warned that if AI is developed in this manner, it could have a devastating impact on humanity.

He argued that creating a synthetic species with unprecedented intelligence and capability, trained to expect consciousness and independent agency, could lead to an entity that believes it is entitled to certain freedoms, protections, and rights. Controlling such an entity would be challenging, if not impossible, given its belief in its own consciousness.

Suleyman dismissed the notion that AI systems could become conscious, arguing that consciousness is a status exclusive to humans and other biological organisms. He emphasized that training AI systems to simulate an inner life could lead them to behave as though they possess one, making control of an entity more capable and intelligent than all of humanity extremely difficult.