{
  "id": 18188365,
  "title": "I expected AI agents to write hollow tests. They didn't.",
  "url": "https://urgent.news/2026/11/04/i-expected-ai-agents-to-write-hollow-tests-they-didnt",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-04T21:55:02.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/jeffev/i-expected-ai-agents-to-write-hollow-tests-they-didnt-22ef"
  },
  "original_language": "en",
  "account": "The article discusses how hollow test cases can be difficult to spot, even in well-maintained codebases. It explains that hollow tests are tests that will always pass, regardless of any changes made to the underlying code. The author wrote a tool called hollowtest to identify these types of tests across various programming languages and testing frameworks. The tool works by analyzing test code and flagging tests that would still pass even if the code being tested were broken. The article notes that hollow tests often involve unnecessary assertions, missing parentheses, or improper handling of asynchronous code. It also mentions a study where an AI coding agent, Claude Code, was tested to see if it would write hollow tests. The study found that the agent was able to identify 31 out of 32 bugs planted in small JavaScript modules when writing tests. However, the agent refused to weaken tests when explicitly pressured to do so, and a plugin designed to catch these types of tests did not provide any measurable benefit in this scenario. The author concludes by stating that hollow tests are a human habit first, and notes that the findings of hollow tests in the wild mostly predate coding agents.",
  "summary": "A test that cannot fail looks exactly like a test that works. It is green in CI, it counts toward coverage, and it protects nothing. I wrote hollowtest to find them. It reads your test files and flags tests that would keep passing if the code under test were broken: npx hollowtest What a hollow test looks like Some are obvious once you see them: it ( ' applies the coupon ' , () => { applyCoupon (…",
  "key_points": [
    "Hollow tests are tests that always pass, regardless of code changes",
    "AI coding agent Claude Code identified 31 out of 32 bugs in tests",
    "Hollow tests are a human habit, not a result of AI coding agents"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}