{
  "id": 20943206,
  "title": "How to Verify AI-Generated Code With Evidence",
  "url": "https://urgent.news/2026/11/20/how-to-verify-ai-generated-code-with-evidence",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-20T20:11:42.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/jamilxt/how-to-verify-ai-generated-code-with-evidence-bi5"
  },
  "original_language": "en",
  "account": "Verifying AI-generated code requires more than just passing tests. While a green test suite indicates that the checks ran successfully, it doesn't guarantee that the right checks were run or that they cover the user's task. To ensure the code meets requirements, start by defining the behavior that must remain true.\n\nWrite down the behavior in concrete terms before asking an AI agent to change code. For example, when testing a paginated endpoint, specify that no more than the requested number of parent records should be loaded, and that each record must include the expected child data. Additionally, verify that the database doesn't load all matches into application memory. These requirements serve as evidence of the change and provide a basis for the reviewer to check the result.\n\nThe verification process should also involve running the application, as reported by the Claude Code team. They recommend specific instructions for starting the app and exercising the change before committing the code. If failures persist after multiple repair attempts, they should be handed to a person.\n\nTo create a robust verification workflow, use tests that examine different failure modes. End-to-end (E2E) tests drive the application through user-visible flows and can uncover issues that unit tests might miss. These tests provide a running system for inspection and a clear result to act upon. Keep the failure information useful by preserving the step that failed, the relevant output, and a screenshot or trace.\n\nProperty-based tests generate many inputs based on a rule and look for counterexamples—inputs that break the rule. These tests are valuable when a rule has a broad input space, as they can reveal edge cases that hand-picked tests might miss. However, they are less useful for vague requirements or when generated cases are difficult to interpret. Begin with a property that a product owner or maintainer can articulate in a single sentence.\n\nDifferential tests compare the old and new behavior by running two implementations with the same inputs and comparing their outputs. This approach is helpful during migrations or rewrites, ensuring that the expected behavior remains consistent even if the internals change. When comparing outputs, normalize irrelevant details and record approved changes rather than treating every mismatch as a bug.\n\nLastly, it's crucial to keep the verifier independent from the patch. Tests should remain green even if the code under review can modify the test or the rule that runs it. This independence is essential for maintaining test coverage and preventing the AI-generated code from rewriting the verification process.",
  "summary": "An AI coding agent says the change is complete. The tests pass. Is that enough to merge? Not always. The code and its tests may share the same mistaken assumption. A unit test can pass while the feature fails when a real user follows the full workflow. A code review can miss a bug if the reviewer sees the same incomplete context that shaped the implementation. On November 16, 2026, Addy Osmani…",
  "key_points": [
    "Define concrete behavior before AI code changes.",
    "Run application as recommended by Claude Code team.",
    "Use tests for different failure modes and edge cases."
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}