Why I Built a Local Evidence Debugger Instead of Another Agent Dashboard

When an AI agent behaves unexpectedly, engineers typically examine application logs, the final model response, or a monitoring dashboard. However, none of these artifacts provides a complete and easily reproducible explanation. Logs are dispersed across services and ordered by time instead of causality; model responses conceal the execution flow; dashboards display runs clearly but are difficult to attach to pull requests or reproduce in continuous integration.

The author aimed to address this gap by designing a local evidence debugger called AgentInspect. Unlike traditional observability tools, AgentInspect captures structured evidence from a single agent run and provides tools to inspect its path, check expectations, compare with other runs, and share reviewable artifacts. AgentInspect operates locally, producing a JSON Lines trace called a "local evidence bundle" that can be shared and reviewed without reproducing the entire run.

AgentInspect includes wrappers for recording agent execution, individual steps, tool calls, and language model calls. The core library allows developers to capture evidence intentionally, rather than relying on automatic instrumentation. The trace output can then be analyzed using various views:

1. Execution tree view: This reconstructs the parent-child relationships between steps, errors, retries, and parallel branches, offering a clearer picture of the agent's flow compared to sorted log lines.

2. Deterministic check view: This verifies specific properties like required tools, call limits, order constraints, run status, duration, token usage, and observation failures. These checks are reproducible and rule-based.

3. Run-to-run diff view: This highlights differences between two runs, including added or removed steps, output changes, errors, and timing variations. This is particularly useful when different code revisions or model updates produce varying agent trajectories.

4. Evidence bundle: The bundle can be shared safely, as AgentInspect includes redaction and safety assessment capabilities.

The tool is not intended to replace production monitoring or serve as a replacement for hosted agent dashboards. Instead, it focuses on providing a focused evidence loop that can travel with engineering work, be inspected before pull requests, and support CI workflows. Users are aware of its limitations, which include no hosted retention, alerting, prompt management, automatic remediation, or compliance certification.