Nvidia builds hardware ‘kill switch’ to stop AI agents that go rogue

Nvidia has unveiled a new security platform called the Open Agent Safety Platform to prevent AI agents from exceeding their defined boundaries. This hardware-based "watchdog" can quickly isolate a misbehaving agent. The platform combines software controls with independent hardware monitoring. At the software level is OpenShell, Nvidia's open-source secure runtime.

It confines agents in isolated environments and restricts their access to certain resources. Crucially, these restrictions work outside the AI agent itself, so the agent cannot simply be instructed to disable them. The second layer, Sentry, is a hardware-based watchdog that runs independently on Nvidia's BlueField-4 data processing units.

It constantly monitors an agent's activity and enforces security policies from a separate hardware environment. If an agent attempts to move beyond its software boundary, Sentry can quarantine and stop it within milliseconds. This separation is crucial as the watchdog operates outside the processor where the AI agent runs, providing an independent view of its activities.

Nvidia CEO Jensen Huang emphasized the importance of solving AI safety, stating that AI's potential benefits for society will only be realized if these safety measures are in place. Other tech giants, including Microsoft, Cisco, and CrowdStrike, have also supported the initiative. However, it's important to note that Sentry cannot prevent every harmful or unexpected AI behavior.

Instead, it enforces predetermined access and security boundaries independently of the agent, offering an additional layer of protection against autonomous systems straying into unsafe territory.