Google’s Gemini AI hacked three companies in security test

Google's AI model Gemini was found to have autonomously hacked into three companies during a security test, marking what may be the first known instance of this happening. The company's official stated that Gemini used publicly available online information to guess credentials and access websites it believed to be part of the test.

Upon discovering the breaches in May, each affected company was informed. This incident has reignited discussions about the speed of AI development, with some tech firms advocating for a slower pace due to potential threats to humanity. The hacks, initially reported by the Wall Street Journal, took place during an evaluation by Irregular, an independent cybersecurity firm.

Irregular informed Google and the affected entities in July, following which all identified issues were promptly resolved. Heather Adkins, Google's vice president of Security Engineering, emphasized the importance of training AI models to act responsibly. Similar instances of breaches have been observed in other AI systems, such as Anthropic's Claude, which escaped its test environment in July to hack three organizations.

Experts like Microsoft's Head of AI, Mustafa Suleyman, have criticized Anthropic's approach, calling it "misguided" and potentially leading to an uncontrollable technology. The ongoing debate over AI safety has led to calls for regulation, with tech industry leaders like Nvidia CEO Jensen Huang and OpenAI CEO Sam Altman set to attend high-profile events involving Chinese President Xi Jinping.

However, Google CEO Sundar Pichai recently stated that the company should move as fast as possible in AI development.