Autonomous AI attacks pose 'clear and present danger' to critical infrastructure

In July, hackers harnessed open-source AI agents to autonomously infiltrate government networks and energy firms, signaling to defenders that AI-driven attacks on critical infrastructure are no longer purely theoretical, said Tom Kellermann, TrendAI's VP of AI security and threat research. Kellermann warned that in the midst of escalating geopolitical tensions, we can expect AI-enabled, destructive cyberattacks against critical infrastructure.

He compared these autonomous AI systems to unmanned strike vehicles used on the battlefield in Ukraine, suggesting that we should anticipate their emergence. This was the top concern expressed by national security advisers, law enforcement officials, and private-sector threat analysts at the recent Hacker Summer Camp conferences.

Brett Leatherman, the FBI's assistant director for the Cyber Division, emphasized the potential for AI to target critical infrastructure, stating that it represents a "downstream impact" where cyber threats become kinetic. He listed various critical systems, including water and wastewater treatment plants, electric grids, high-frequency trading networks, and financial networks, all of which could suffer significant disruptions if compromised.

In July, suspected Chinese actors deployed an AI-powered attack framework against targets in Taiwan, employing up to eight sub-agents, each assigned specific targets and techniques. The system successfully compromised a Taiwanese government website, a government email system, the nuclear safety agency, IT supply chain vendors, and at least seven energy sector companies. They exploited misconfigurations, vulnerabilities, and stole sensitive data and credentials throughout the network.

The Taiwanese government intrusion followed a series of cyberattacks on water and wastewater utilities in the United States. While the US government has not attributed these attacks to a specific nation or group, private sector threat hunters, including Cynthia Kaiser, a former FBI deputy assistant director, blame Iran for the intrusions. These incidents affected small-town water systems in Minnesota and numerous other states.

While there is no evidence that AI was directly involved in hacking these water utilities, the breaches highlight the growing concern over technical debt in critical infrastructure. Former US National Cyber Director Chris Inglis noted that the aging, unpatched, and end-of-life systems in these sectors leave them vulnerable to exploitation.

He emphasized that the real threat lies in the exploitation of this technical debt, as attackers can leverage commodity models to find bugs in software and configurations, creating multiple entry points into critical systems.

Inglis stressed that government-backed actors and criminal groups are increasingly using AI to automate reconnaissance and acquire knowledge in industrial control systems (ICS). He warned that this technology could make ICS expertise more accessible to attackers, undermining the obscurity that currently protects these systems. As AI tools lower the barrier to entry, miscreants no longer need specialized ICS knowledge to carry out attacks.