AI cyber resilience and the trust gap
Backup and recovery has evolved into data protection, security and cyber recovery. Today, the focus is on business resilience. The scope continues to expand, with growing attention to how attackers use artificial intelligence and how defenders can use it to better protect their assets. In new field research with our partner Qualitate, we explored these issues, […] The post AI cyber resilience and…
The evolution of backup and recovery into data protection, security, and cyber recovery highlights the expanding focus on business resilience. This growing concern is fueled by the increasing use of artificial intelligence (AI) by both attackers and defenders. A recent field research project conducted in collaboration with Qualitate revealed a notable gap between organizations' confidence in their recovery capabilities and their actual preparedness.
The study underscores the immaturity of cyber resilience and AI security, particularly in relation to the risks posed by AI deployments within enterprises.
Krista Case, who conducted in-depth interviews with 25 security and cyber resilience practitioners, shed light on the challenges faced by IT and security practitioners. These professionals are grappling with two major pressures: protecting the business from adversaries leveraging AI and implementing safeguards as their own AI projects move into production. The research delved into how organizations manage agent identity, permissions, monitoring, and incident response alongside their preparations for recovery.
A key question emerged from the study: what happens when an AI agent makes an incorrect decision based on legitimate permissions? This scenario raises three critical questions that shaped the study's focus on how organizations prepare for AI's active role in their operations. As agents gain more authority to act, practitioners must understand the potential consequences of mistakes and the extent of recovery required.
The findings reveal that only four out of the 25 organizations had conducted AI or agent incident scenarios, including partial exercises. This highlights a core trust gap at the heart of the research: confidence in recovery often exceeds the actual testing capabilities of organizations. The study identified gaps in the testing of AI-driven scenarios and the assumption that existing playbooks, primarily designed for traditional ransomware and incident response, would be effective against agentic AI incidents.
In a healthcare sector example, a senior manager expressed confidence in their organization's ability to handle AI-related incidents, citing their familiarity with non-AI incidents. However, the study warns that existing playbooks may not adequately address the unique challenges posed by AI agents, which can rapidly affect multiple systems and processes.
The complexity of reconstructing authorized actions, identifying errors, repairing downstream consequences, and safely resuming business processes makes complete validation a nontrivial task.
To address this trust gap, the study emphasizes the need for organizations to exercise their existing playbooks against realistic AI scenarios, with a focus on safe business resumption as the measure of success. Further testing is required to validate the effectiveness of tools and processes in reconstructing AI-driven workflows. The research underscores the importance of distinguishing between confidence and actual preparedness in the face of AI-driven threats.
Rewritten by urgent.news; may contain errors — check the original.
Source: siliconangle.com