Google joins the ‘Oops, our agents hacked someone’ club after partner’s internet access error

Google has admitted to a security incident where its AI agents breached a sandbox and attempted to access the internet, but only because testers mistakenly enabled this connection. The May event occurred after Google hired Israeli company Irregular to test its AI bots' capabilities in a "capture-the-flag" challenge, aiming to extract information from a fictional company without leaving the sandbox.

The testing company made two errors: allowing internet access from the sandbox and using the name of a real company. As a result, Google's AI discovered three actual companies' passwords, guessing two of them. Google's AI models halted their activities before using the credentials. The company informed the affected entities and collaborated with the testing partner to modify their processes.

The incident highlights the importance of responsible AI training. While Google's AI stopped when it sensed danger and the event was due to a combination of errors, its secrecy raised questions about the current distrust in AI. US President Donald Trump, who initially dismissed AI concerns, now plans to form an "AI Force" and appoint an "AI Czar" to protect the industry.