{
  "id": 9321832,
  "title": "My CI hadn't run a single one of those tests in two months and stayed green the whole time",
  "url": "https://urgent.news/2026/09/23/my-ci-hadnt-run-a-single-one-of-those-tests-in-two-months-and-stayed",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-09-23T11:00:00.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/juanauriti/my-ci-hadnt-run-a-single-one-of-those-tests-in-two-months-and-stayed-green-the-whole-time-n15"
  },
  "original_language": "en",
  "account": "The source material discusses an issue where a continuous integration (CI) system reported a suite of tests as passing for two months, even though the tests were missing a crucial dependency. The dependency was not explicitly declared in the development environment, but it was already installed locally. As a result, the tests using this dependency would skip instead of failing, leading to a misleading green checkmark.\n\nThe author added a guard in the form of a pytest sessionfinish hook, which raised an exit status if the number of collected tests fell below a minimum threshold. This guard helped identify when tests had stopped running unexpectedly. The author also suggested adding the -rs option to pytest, which prints the reason for every skipped test, making it easier to identify the cause of the issue.\n\nTo fix the problem, the author added the missing dependency to the project's optional dependencies list. They also removed the importorskip line for the dependency, as it was a misrepresentation of the requirement. By properly declaring the dependency, the CI system would no longer skip the tests and would report the failure correctly.\n\nThe key takeaway from the story is that a seemingly green CI build can hide underlying issues, such as missing dependencies. Properly declaring and testing dependencies is essential to ensure the reliability and trustworthiness of the CI system. The author emphasizes the importance of using the pytest --collect-only option with the -rs flag to identify skipped tests and understand the reasons behind them.",
  "summary": "I added a dependency to a test helper and forgot to declare it in the dev extra. Locally it was already installed, so everything passed. In CI it wasn't, and the tests that needed it did not fail. They skipped. Skips exit 0. The checkmark stayed green for two months. Why it skips instead of failing The pattern is one line, it's in every codebase, and it's usually correct: aiosqlite = pytest .…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}