An emergency brake…or the architectural demotion of the AI model?

On June 14, Satya Nadella published an article on X about corporate AI, stating that the enterprise's durable asset was not the foundation model, but the learning loop. This idea had been developed by his company and himself over the past three years. Impressed by Nadella's point, the author responded with an article on Fast Company, aiming to elevate the discussion on the central challenge of enterprise AI – architecture, not intelligence.

In another article, Nadella went even further, suggesting that AI models should be treated as completely untrusted components. He emphasized that all models could be compromised and urged systems to be designed to contain them from the beginning. The press interpreted this as advocating for an "emergency brake," but the author argues that the more important aspect is the architecture that makes such containment possible.

The security principle of assuming breach, or that a system has already been compromised, is applied to AI by treating all models as potentially compromised. This does not imply a design hypothesis of attack, but rather a design hypothesis considering various ways models can misbehave. The model should not control its own permissions or decide what it can access; instead, a separate, higher layer should handle permissions, validate parameters, and enforce policies before executing actions.

By separating the model from the harness, the architecture of corporate AI can be improved, allowing the model to bring cognitive capacity without being a sovereign center of the corporate system.