{
  "id": 7730683,
  "title": "Zero Failures and Zero Tests Look the Same",
  "url": "https://urgent.news/2026/09/16/zero-failures-and-zero-tests-look-the-same",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-16T07:20:49.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/sergueyasaelshinder/zero-failures-and-zero-tests-look-the-same-241b"
  },
  "original_language": "en",
  "account": "The build system appears green, unchanged for the past week, which is concerning. A directory was renamed in a commit from nine days ago, but the matching files no longer align with the pattern used. As a result, two hundred tests are no longer being collected. Despite this, no failures were reported, as no tests ran. Test suites only report failures, not the absence of them. Consequently, a zero of two hundred passing tests and a zero of zero passing tests result in the same color, tick, and message in the channel. This practice of shipping on faith and assuming everything is fine is problematic. There are numerous ways this can occur, such as a tag filter accidentally excluding a group, a skip annotation added for release purposes without a ticket, a guard around an import to allow the suite to start when a dependency is missing, a pipeline step marked to continue on error, and a test class lacking assertions due to a moved assertion that was never reinstated. Each of these scenarios may have seemed reasonable at the time they were implemented, but they leave no indication of their presence. Coverage reports cannot rescue you, as they only measure the code that ran against the code that ran, and a suite of nothing covers nothing perfectly. It is crucial to find the actual number of tests executed, not just the color-coded result. Compare this count to yesterday's figure and make the pipeline fail if it drops. A single ten-line guard can catch every version of this issue. Additionally, perform a cruder check by intentionally breaking something or inverting a condition. Run the pipeline and observe if the suite still passes. If it does, you have learned a valuable lesson that could have been prevented by proper testing. Remember that green indicates someone asked a question and received an answer they wanted, but it does not guarantee that anyone actually asked. This issue is common and has multiple causes.",
  "summary": "The build is green. It has been green all week. That is the part you should have found suspicious. Somebody renamed a directory in a tidy-up commit nine days ago. The runner matches files by pattern. The pattern no longer matches. Two hundred tests stopped being collected. Nothing failed, because nothing ran. A test suite reports failures. It does not report absence. Zero of two hundred passing…",
  "key_points": [
    "Build system unchanged for a week despite 200 tests missing",
    "Zero passing tests reported despite no failures in pipeline",
    "Single guard can catch all versions of this issue"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}