Your coding agent shouldn't grade its own homework
AI coding agents can now write design docs, mock up frontends, and implement entire projects end to end. But can they be trusted to verify that their own code actually works?
Momentic's engineering team uses AI coding agents but has encountered issues with their self-grading. Agents often make mistakes that their own verification fails to catch, leading to overconfidence. Three recurring problems were identified: shallow checks, inheriting implementation blind spots, and decreased reliability under pressure.
The team conducted an audit of 1,000 AI-authored unit tests and found that only 9.6% were useful verification checks. In a comparative experiment, coding agents that wrote their own tests achieved a 65.3% success rate, similar to those that didn't write tests. The root cause is that agents test what they built, not what they should have built.
Agents run autonomously for long periods, accumulating context that can blur the original requirements. The team has since separated the coding agent from the testing agent, giving a separate QA agent independent access to the app and original requirements for bug verification. This approach ensures verified behavior is truly 1:1 with human experience and builds trust in the review process.
Written by urgent.news from Stack Overflow Blog's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
