{
  "id": 4911960,
  "title": "I Built an Agent to Fix Bad Tests. I Found Eight Bugs in My Own Ruler.",
  "url": "https://urgent.news/2026/09/01/i-built-an-agent-to-fix-bad-tests-i-found-eight-bugs-in-my-own-ruler",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-01T18:01:34.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/marvinoka4/i-built-an-agent-to-fix-bad-tests-i-found-eight-bugs-in-my-own-ruler-1eap"
  },
  "original_language": "en",
  "account": "The author built an agent to automatically generate tests for fixing bad code, specifically focusing on mutation testing. They discovered that many existing tests in popular Python libraries were either too weak or not executing the relevant code at all. This led the author to reframe their project as a measuring instrument rather than a test generator. The author identified eight bugs in their own measuring instrument, which would have resulted in inaccurate results if not caught. These bugs included incorrect handling of editable installs, silent failures on certain package layouts, concurrency issues, and incorrect test file selection. The author suggests that similar measurement bugs may be common in published results, as better numbers are more likely to be published while flaws go unnoticed.",
  "summary": "Here is a Python function and a test for it. def withdraw ( balance , amount ): if amount <= 0 : raise ValueError ( \" amount must be positive \" ) if amount > balance : raise ValueError ( \" insufficient funds \" ) return balance - amount def test_withdraw (): assert withdraw ( 100 , 30 ) == 70 That test gives you 47% line coverage. It gives you a 9.5% mutation kill score. The harness generates 21…",
  "key_points": [
    "Author built an agent for mutation testing to fix bad tests",
    "Identified eight bugs in own measuring instrument",
    "Bugs include handling editable installs and concurrency issues"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}