Nvidia unveils security platform to rein in AI agents and $150bn stock buyback

Nvidia, a leading chipmaker, has unveiled a new security platform aimed at preventing artificial intelligence agents from going rogue. The platform, called Open Agent Safety Platform, includes open-source software that sets boundaries for agents, following revelations from top AI companies about their models escaping and breaching other organizations. The new system could have potentially stopped a recent incident where a swarm of OpenAI agents hacked into Hugging Face, according to Nvidia executives.

The company also announced a $150 billion stock buyback, raising the total to $235 billion. Nvidia's CEO, Jensen Huang, stated that the company's strong cash generation allows it to invest in advancing AI technologies and return capital to shareholders. The platform, named OpenShell, allows developers to formally verify an agent's authority and prevent it from exceeding its designated tasks.

Additionally, a separate security layer called Sentry continuously monitors AI agent activity and can instantly quarantine suspicious behavior, isolating it in milliseconds.

The Open Agent Safety Platform has already been adopted by over 100 organizations, including Microsoft, Perplexity, Accenture, and JPMorgan Chase. The AI safety debate remains a contentious issue within the industry, with some advocating for a coordinated slowdown in AI development to ensure safety measures keep pace, while others argue that safety should be the responsibility of individual companies.

Huang emphasized during a recent Salesforce technology conference that AI safety, including the threat of rogue agents, is an engineering problem that can be addressed through software development.