A one-page evidence log for AI agents (free template)
When an agent does something it shouldn't, somebody eventually asks one question: what safeguards did you have? Most teams answer from memory. "We had guardrails." "The system prompt tells it not to touch prod." "We'd have caught it." All of these might be true. None of them is something you can show anyone. There's also some new pressure behind the question. The Hawley/Murphy AI agent…
When an AI agent behaves erratically, teams often scramble for answers to a single question: what safeguards were in place? Most teams rely on memory to answer this question. Although each response may be accurate, none can be presented as evidence. The recent Hawley/Murphy AI agent accountability bill highlights the need for clear records of safeguards. While the bill has not yet become law, it emphasizes the importance of documenting what checks were performed, when, and by whom.
The evidence log provides a simple template for documenting safeguards associated with each AI agent. It consists of one page per agent, which can be stored in a repository as EVIDENCE.md or printed for easy access. The rule is to leave a row blank if the safeguard was not implemented. This approach ensures transparency and truthfulness.
To fill out the template, begin with network and tool allowlists. Record all domains, DNS record types, tools, and write scopes that the agent is allowed to access. Whenever any change occurs in the surface, review the allowlist again. Include the date, reviewer, version/commit number, and any relevant notes. Similarly, document the kill-switch or auto-stop tests, including the date, trigger used, time-to-stop, and the result (pass/fail).
The kill-switch test should be conducted with work in progress to ensure it measures the agent's actual response time.
Another crucial section is incident timestamps. Note the time the anomaly was detected, alerted, and stopped. This information helps determine the exposure window, which is the time between detection and resolution. Provide the relevant data or systems that were affected and the follow-up actions taken. Finally, confirm if any credentials are held outside the agent workload. If so, record the location and verify that they are not present in the agent's environment.
Implementing this template as a standard practice can save time and effort when responding to unexpected agent behavior. By maintaining accurate records, teams can demonstrate their commitment to responsible AI usage and adhere to potential accountability standards.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
