How to review pull requests written by coding agents

Reviewing pull requests from coding agents presents unique challenges compared to those from human contributors. The agent-written PRs appear more polished and complete, but this polish can also hide potential issues. Instead of relying on obvious red flags like messy diffs, vague descriptions, or missing tests, reviewers must adopt a different approach.

First, focus on the diff itself. Read it carefully to understand what has changed, then compare it to the agent's description. Discrepancies between the two are where problems may lie. For instance, a refactored retry logic that also changes the timeout default or a new field that makes an existing one optional may not be immediately apparent from the diff alone.

Next, identify all callers of the changed functions, types, or exported values. An agent might only search for a subset of these, leading to missed dependencies. Use tools like your editor's "find references" feature or grep to locate every occurrence. Be cautious if a tool reports no callers, as this usually means the result is unknown, not that the code is safe.

Consider the team's established practices and policies. Agents may not be aware of internal rules, such as database access through a repository layer or prompts stored in separate files. Check whether the PR adheres to these team-specific guidelines or if it deviates towards what is commonly found online.

Watch out for scope creep. Agents often aim to tidy up by making minor changes to unrelated files. Each of these small adjustments should be addressed separately. Request that the agent splits out these changes or reverts them if unnecessary.

Review the tests thoroughly. Agents are skilled at making tests pass, sometimes at the expense of test quality. Examine test diffs closely for any loosened assertions, new skips, or mock behaviors that no longer reflect the intended functionality. Remember that a passing PR with edited tests does not guarantee the same level of assurance as a PR where tests were written correctly from the outset.

Finally, examine the error handling. Agents tend to focus on the expected, happy path, often overlooking potential failure branches. Investigate how the code handles errors, swallowed errors, catch statements that log and continue, or fallback mechanisms that return default values. These paths are often untested and can lead to unexpected behavior.

To streamline this process, consider writing a clear checklist or decision documentation (e.g., CLAUDE.md, .cursor/rules, or AGENTS.md) that agents can reference before opening a PR. Automating tasks like finding all callers of changed code and documenting team decisions can also significantly reduce the workload on reviewers. Tools like Tently AI code reviewer for GitHub can help by checking PRs against dependencies, team decisions, and learning from past PRs, review threads, and documentation.

Implementing these practices can greatly improve the quality of pull requests from coding agents and ensure a smoother review process.