CISA case renews scrutiny of AI accountability

A recent incident involving ChatGPT has reignited debate around the accountability of artificial intelligence (AI). The episode centered on Madhu Gottumukkala, acting director of the US Cybersecurity and Infrastructure Security Agency (CISA), who inadvertently uploaded sensitive government contracting documents to a public version of the AI platform during July and early August 2025.

This action prompted automated security alerts and a subsequent review by the Department of Homeland Security (DHS). Despite assurances from CISA that no classified information was involved and that suitable safeguards were in place, the case has underscored the challenges of governing AI usage. AI systems have evolved beyond simple chatbots to become more autonomous, capable of retrieving records, invoking software tools, sending messages, and executing actions on behalf of users.

This shift has introduced new risks, such as privilege escalation, misconfiguration, unpredictable behavior, structural vulnerabilities, and accountability issues. In response, CISA and AI security agencies from multiple nations, along with the US National Security Agency, issued joint guidance in May, cautioning organizations against granting excessive autonomy to AI systems and advising them to avoid exposing sensitive data or critical systems to these tools.

The guidance emphasizes the importance of implementing robust identity controls, continuous monitoring, threat modeling, and security assessments. However, despite these recommendations, a significant gap remains in the implementation of these controls. According to IBM's 2026 Cost of a Data Breach research, approximately one in five organizations surveyed reported experiencing an AI-related breach, with 92% lacking proper AI access controls.

Moreover, fewer than half of organizations actively manage non-human identities, which now encompass AI agents operating through various platforms. The difficulty in tracing accountability arises from the fact that autonomous systems can execute multiple steps after a single instruction, often appearing in audit records as if they were the human user whose credentials they inherited.

Traditional enterprise controls typically assign access to named employees, service accounts, or system owners. However, agentic AI, which acts under delegated authority and chains tasks across services, can blur these boundaries, making it challenging to distinguish between human decisions and autonomous actions. To address this issue, CISA recommends that organizations define clear roles and responsibilities for agentic systems, apply least-privilege access, and ensure appropriate human oversight proportionate to the potential impact.

These measures aim to maintain a traceable link between automated actions and the human or organizational authority behind them. The accountability challenge extends beyond the government sector, affecting businesses across various industries, including procurement, customer data management, financial workflows, software development, and internal communications, when agents inadvertently send information to incorrect destinations, retrieve unauthorized material, or execute actions in unsafe contexts.

Security experts argue that AI governance must operate at the transaction level, recording details such as the individual who authorized the agent, the permissions held, accessed data, external systems contacted, and whether the action exceeded predefined limits. This approach ensures traceability and accountability in AI-driven processes.