5 mistakes AI coding agents make, and how to catch them before merge
AI coding agents write code fast. They also make the same handful of mistakes over and over. None of them are exotic, which is exactly why they slip through review: the diff looks plausible, the summary sounds confident, and the reviewer is busy. Here's what to check every AI-assisted diff for before it merges. 1. Invented APIs How to spot it: calls to functions, methods, flags or packages that…
Artificial intelligence coding agents produce code rapidly, but they consistently commit the same five common errors that often evade review. This is because AI-generated code appears plausible and the reviewer is occupied. Here are the essential checks to verify each AI-assisted diff before it merges.
1. Invented APIs: Look for calls to functions, methods, flags or packages that do not exist within the actual versions of the libraries you depend on. The function name usually sounds plausible, making it hard to spot. Resolve these issues by eliminating or replacing the foreign calls. If automated tests pass, ensure the code actually executes the line in question.
2. Swallowed Errors: Identify empty catch blocks, "except: pass", or an error that is logged and then ignored, allowing the function to proceed with incorrect state. Address the issue by properly handling the error, rethrowing it, or writing a comment explaining why ignoring the error is safe. Simply stating "It made the test pass" is not a valid justification.
3. Fake Tests: Detect tests lacking assertions, tests that merely verify a mock function was called, or tests that precisely replicate the function's implementation, rendering them incapable of failing. Resolve this by rewriting these tests to evaluate the behavior observable from outside the function, such as return values, saved records, or HTTP responses.
4. Confident Wrong Summaries: Avoid summaries like "All tests pass" or "Fixed the bug" without providing command output. Before approving a diff, manually run the command and examine the output. The reviewer's summary of a test run is not the same as the test run itself.
5. Silent Behavior Changes inside "Refactors": Spot a diff labeled as a refactor that simultaneously alters defaults, error messages, log lines, or the sequence of events. Move behavior modifications into separate commits, including tests that demonstrate the new behavior on purpose. Before merging, ask yourself, "Would I feel comfortable defending this diff in a review if I wrote every line myself?" If the answer is no, the diff is not ready, regardless of who or what produced it.
To ensure the checklist's effectiveness, adopt the following practices: incorporate the list into your pull request template, so the author can review it before submitting a request for review. If your AI agent supports it (Claude Code subagents, Cursor rules, Copilot instructions), provide the same checklist to the agent and have it validate its output before handing it over.
Although the checklist cannot catch every mistake, it does catch simple errors before a human must intervene. The complete 12-point AI code review gate, including a pre-merge checklist and a ready-to-use AI code-reviewer subagent for Claude Code, are available on GitHub: lean-team-products/ai-code-review-gates. Incorporate these tools into your repository to be set up in just two minutes.
Originally published on the Lean Team blog. A free download is provided: the Codebase Antipattern Checklist, which lists 48 signs that a codebase is becoming more challenging to change.
Rewritten by urgent.news; may contain errors — check the original.