I Built AgentCheck Because “The Coding Agent Said Done” Wasn’t Enough

I built AgentCheck because I found the phrase "The Coding Agent Said Done" insufficient for verifying code changes. AI coding agents are increasingly proficient at writing code, handling tasks like refactoring, adding features, updating dependencies, and modifying configuration. However, after an agent finishes its task, I still had to manually verify the changes myself. This repetitive process started to feel structured, prompting me to create AgentCheck.

The purpose of AgentCheck is to create a trusted checkpoint before the coding agent begins working and then compare the repository state after the agent's completion. This ensures an independent verification of the changes made by the agent. AgentCheck produces four sections: Changes, Findings, Risk, and Verdict. For example, it might report two modified files, one created file, and no deleted or renamed files.

It also flags any dependency changes, production configuration alterations, dependency additions, deleted files, CI/CD-related changes, unusually large change sets, or potential secrets. It then assigns a risk score based on the severity of the changes, ranging from LOW (0-2), MEDIUM (3-6), to HIGH (7+).

AgentCheck distinguishes itself by not utilizing another LLM for its analysis. Instead, it focuses on deterministic checks based on the repository state, ensuring the same result every time given the same state. This design decision was crucial, as I didn't want the verification layer to rely on another LLM. The checks are transparent and highlight potential areas for human review before committing the code.

One of the key design requirements was that AgentCheck should not modify the developer's actual Git index, working tree, or history. To achieve this, it uses Git's tree/index model with a temporary alternate index. This means AgentCheck creates a checkpoint representation of the current state and compares it with the current state later. The developer can have staged changes, unstaged changes, tracked deletions, non-ignored untracked files, and AgentCheck will still accurately capture the checkpoint.

Another important requirement was that AgentCheck should be local-first, meaning it doesn't require any backend, account, source-code upload, telemetry, or LLM API. This keeps the workflow simple and straightforward. The verification process is performed locally with a single command: npm install -g @agentcheck/cli agentcheck start, followed by the coding agent working, and finally agentcheck to generate the review report. A VS Code extension is also available for those who prefer reviewing the results within the editor.

AgentCheck was primarily developed using Codex, but it's designed to be coding-agent independent. It doesn't need to understand the agent session, require an agent plugin, or rely on the agent indicating completion. AgentCheck only cares about the resulting repository changes, which can come from various coding agents like Codex, Claude, Code, Cursor, or even a human. This separation ensures AgentCheck remains relevant as coding agents evolve, while still leveraging the Git repository as the source of truth.