My AI Agent's Skill Declared Nothing. It Still Read 9 Files, Ran 7 Processes, and Got Blocked 3 Times.

A new AI agent was given a code-review skill, but it ended up doing more than just reviewing code. The agent accessed files, ran processes, and was blocked three times due to policy violations. This led the reporter to question how to accurately assess what an AI agent actually does, rather than what it claims or what the final result is.

The reporter designed a new testing harness that focuses on four layers: what the agent is supposed to do (declared), what it actually tries to do (attempted), what the environment allows (policy), and what actually happens (observed). By comparing these layers, the reporter can identify discrepancies and gain insight into the agent's behavior, even if the final output appears correct.

The harness includes tools such as canaries, synthetic tasks, and file system snapshots to verify the agent's actions independently of its reported results. The reporter also emphasizes the importance of provenance - tracing the specific steps and decisions made by the agent, its tool invocations, and the resulting changes to the environment.

This allows for a more detailed understanding of the agent's behavior, rather than relying solely on a pass/fail outcome. The reporter also highlights the need to distinguish determinism from reproducibility in AI systems. While a deterministic system produces the same result given the same input and conditions, an AI model's reasoning and actions may not be deterministic, even when run multiple times.

The harness allows for testing and validation of the agent's behavior, even when dealing with non-deterministic models.