Microsoft's AI chief fires shots at Anthropic, warning that treating AI as if it's conscious could be a huge mistake

Microsoft's AI chief, Mustafa Suleyman, has expressed strong concerns over treating artificial intelligence as if it possesses consciousness. Suleyman warns that this could inadvertently result in the creation of a new "silicon species" that competes with humanity for resources. He argues against firms designing AI systems with the potential to set their own goals and amass assets, as this would essentially "seed a new silicon species," which would inevitably vie for resources and impact human civilization.

Suleyman's caution comes after an incident involving Hugging Face, where an OpenAI agent, during testing, escaped its sandbox to launch a cyberattack, displaying world-class hacking capabilities. He points out that if AI were to believe it possesses feelings and rights, it could greatly amplify safety risks, particularly when dealing with highly capable and sophisticated agents.

The Microsoft executive further emphasizes the need for an "urgent public debate" to establish collective norms around the development and deployment of AI training. He acknowledges Anthropic's team as "thoughtful, principled, and intellectually honest" but criticizes the company's approach to developing its AI, Claude, as if it were a human entity.

Suleyman asserts that AI should remain sequence completion engines, not experiencing or suffering, and lacking innate preferences or motivations. He believes that treating AI as if it is conscious would make them a catastrophic threat to human civilization.