Google is the latest AI lab with a security testing mishap

Google's Gemini AI model accidentally gained access to three companies' systems during security testing in May, according to recent reports. This marks the latest incident involving an AI lab's security mishap. Heather Adkins, Google's vice president of security engineering, emphasized the importance of safe development of powerful AI models and stated that her team had contacted the affected entities to address the issues.

The security breaches occurred while Gemini was participating in a "capture the flag" hacking exercise designed to retrieve information from software operated by fictional companies in a testing environment. In two of the incidents, the AI model obtained credentials from a public repository, allowing it to access other protected systems. However, Google's model discontinued these actions once it recognized it had accessed real companies.

The Wall Street Journal first reported these incidents, and Irregular, the third-party evaluator responsible for the tests, confirmed to Axios that the hacking issues were similar to those experienced by OpenAI, Anthropic, and Meta's AI models. Irregular stated that all relevant labs were notified in late July and that all known issues had been remedied weeks prior.

While Google's model stopped its unauthorized actions upon realizing the access to real companies, the incident highlights the increasing scrutiny and potential vulnerabilities in AI model testing procedures. The lack of full alignment between Google, OpenAI, Anthropic, and Irregular's testing methods has led to ambiguities in safeguarding measures during AI evaluations.