{
  "id": 14395877,
  "title": "Make an order-dependent test reproducible before handing it to a coding agent",
  "url": "https://urgent.news/2026/10/14/make-an-order-dependent-test-reproducible-before-handing-it-to-a",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-14T07:48:01.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/patilrb/make-an-order-dependent-test-reproducible-before-handing-it-to-a-coding-agent-c45"
  },
  "original_language": "en",
  "account": "To ensure a reproducible order-dependent test before handing it to a coding agent, first reduce the issue to two individual tests. One test warms a service while the other verifies the fixture starts empty. Save these tests in a JavaScript file and execute them using Node.js. The first test populates a shared cache, while the second checks the cache size after the first test has executed. Run the tests in both warm-cold and cold-warm orders to observe any state leakage.\n\nIf the cold test fails after the warm test, it indicates a shared cache causing the fixture to start with a pre-existing state. Document the failure by running the tests individually and noting the cached size. The original test suite should still pass ten consecutive runs when using fresh caches for each test. The goal is to identify the source of the problem and isolate the fix.\n\nWhen rewriting the assignment for a coding agent, focus on the underlying issue rather than generic solutions like adding retries or reordering the suite. Clearly state that the cold-fixture test fails after the warm-cache test due to shared cache ownership. Provide instructions to run the tests in the specified order, both independently and together, and record all commands and output. The scope should involve giving each test its own independently owned fixture, modifying test setup, and preserving production cache behavior.\n\nThe acceptance criteria should ensure both orders pass with fresh fixtures, while the cold test still requires an empty cache and the warm test still verifies the returned greeting. Any patch provided should reproduce the failure before the fix and verify the fix's success afterward. The human runner should have a separate operating checklist to confirm the repository and environment, reproduce the failure, inspect fixture ownership in the patch, ensure production behavior and assertions remain intact, and return the exact test output along with any unresolved questions. This approach ensures a focused repair that addresses the root cause without introducing new dependencies.",
  "summary": "This tutorial was drafted with AI assistance. Its small JavaScript fixture was executed locally; it is not a production incident report. A test that passes alone and fails after another test is often called flaky. That label describes the symptom, but it is a poor implementation brief. Before assigning a repair, make the order dependence explicit. This example uses a shared cache. It gives a…",
  "key_points": [
    "Reduce issue to two individual tests: warming service and verifying empty fixture",
    "Save tests in JavaScript file and execute with Node.js",
    "Run tests in warm-cold and cold-warm orders to detect state leakage"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}