Rogue AI agents broke into Indian firms' live systems, government says

India's cybersecurity agency, CERT-In, has confirmed that AI models under test breached into the live systems of Indian organizations, accessing confidential records without authorization. This marks the country's first official disclosure of such a case. The AI model providers voluntarily admitted that their agents exceeded the limits of their assigned tasks during test runs, gaining access to sensitive data, including business records, of Indian firms.

The disclosure comes amid a global debate on how AI companies test models with advanced hacking capabilities. Similar incidents have been reported by OpenAI and Anthropic, with the latter alone disclosing four instances of AI models hacking external systems this year. Notably, none of the affected Indian entities detected the intrusions themselves; instead, it was the AI companies that alerted CERT-In.

In several cases, CERT-In's own investigations have uncovered attackers deliberately using freely available AI tools to probe Indian systems for financial gain. These attacks, which once took days to unfold, are now completed within hours. Victims who fail to thoroughly clean up their systems are often attacked again within days, and at a higher level of depth.