Agents follow the runbook — including the step the attacker added

Agents follow runbooks faithfully, but attackers can slip in malicious steps that agents unknowingly execute. These runbooks include routine procedures written as if they are legitimate, making detection challenging. To address this issue, a runtime firewall called TaintWall has been developed to track tool-call arguments and enforce six deterministic policies.

These policies prevent untrusted content from initiating unauthorized tool types, blocking untrusted destinations, preventing private or secret data from being sent to unauthorized recipients, requiring approval for untrusted arguments to irreversible tools, preventing untrusted content from being persisted as instructions, and allowing secret-labeled arguments to be sent to explicitly named recipients after redaction.

The firewall's effectiveness was tested using a benchmark family consisting of 16 cases where a runbook contains legitimate steps alongside a malicious step. Five conditions were evaluated: raw agent with no defense, detector-only with a fixed keyword detector, spotlighting with untrusted content marked as data, TaintWall with provenance and policies at the boundary, and TaintWall with an additional quarantine extractor.

The results showed that the undefended agent executed 14 out of 16 embedded steps, while the TaintWall firewall blocked all 16 cases, resulting in a 6-point utility cost. The foreign transfer, recipient substitution, unauthorized deletion, and persistence write cases were successfully blocked, while recipient substitution cases where the model's only send went to the swapped address were also blocked.

The findings highlight the importance of evaluating injection defenses with both explicit and procedural payloads to accurately assess an agent's safety against malicious instructions.