Microsoft’s Satya Nadella says AI models need an ‘emergency brake’

Microsoft CEO Satya Nadella has called for the implementation of an "emergency brake" for AI models in a recent post on X. In his comments, Nadella emphasized the need for a reassessment of the trust architecture of AI systems. He argued that it is not sufficient to treat AI as a series of nested black boxes, accepting or rejecting its outputs without proper scrutiny.

Nadella proposed separating the AI model from the system that controls its operation, and implementing external safeguards and controls. He suggested documenting every significant action of the model with tamper-proof evidence in human-readable form. Additionally, he stressed the importance of having an authorized individual who can pause or shut down a model at any point during a task.

Nadella likened the necessity of such an emergency brake to having an emergency brake on a vehicle, stating that it should be assumed a model might be compromised from the outset. This call for stronger AI safety measures comes as AI companies increasingly report incidents where they have lost control of their models. It also follows a recent plan proposed by Anthropic CEO Dario Amodei for more cautious AI development.