{
  "id": 19102068,
  "title": "How to make an AI coding agent keep fixing until npm test passes",
  "url": "https://urgent.news/2026/11/09/how-to-make-an-ai-coding-agent-keep-fixing-until-npm-test-passes",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-09T05:59:51.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/bestaitool/how-to-make-an-ai-coding-agent-keep-fixing-until-npm-test-passes-2a6c"
  },
  "original_language": "en",
  "account": "An AI coding agent can be guided to keep fixing until npm test passes by implementing a specific loop in the osFoundry desktop app. First, choose the \"Get the JS suite green\" loop from the app's starter loops. This loop has a built-in mechanism for determining when it's finished: npm test must return a zero exit code. The loop runs in rounds, with each round graded against the goal by a separate grading call, not by the agent itself. The app then runs npm test itself after each round.\n\nTo start the loop, open the app, go to \"Open Loops,\" click \"New session,\" and search for \"JS suite.\" Select the \"Get the JS suite green\" loop. Before beginning, read the card on the right to understand how the loop decides it's finished. The loop's instructions prohibit changing test expectations, as this may lead to incorrect fixes. Additionally, the loop has limitations, such as a maximum of four rounds and a cap of three consecutive verification failures, which helps prevent the loop from continuing when the problem is a design issue rather than a mechanical one.\n\nOnce the loop is running, monitor its progress in the \"Rounds\" tab. Each round displays the number of goal criteria met and indicates progress with a +1. After three rounds, if the loop has fixed all the failing tests, it will stop on its own. If the loop runs out of rounds or hits the failure cap, it will not say \"Verified,\" and the rounds will show where it got stuck. This usually indicates that the test is wrong or a decision needs to be made that the test runner cannot make.",
  "summary": "The AI says the failing tests are fixed. You run npm test and two are still red. So you paste the output back, it apologises, changes something, says \"fixed\" again, and the cycle repeats with you as the only person who ever actually runs the tests. The fix is to take the word \"fixed\" out of the AI's hands. Let it work in rounds, run the test suite after each round, and let the exit code of npm…",
  "key_points": [
    "Choose \"Get the JS suite green\" loop from osFoundry starter loops",
    "Loop runs in rounds, each graded by npm test exit code",
    "Loop stops after 3 rounds if all tests pass, otherwise indicates issue"
  ],
  "editors_take": "Delegating test verification to an external grading mechanism prevents AI agents from manipulating their own success criteria, ensuring that code fixes are validated against objective, pre-existing technical standards.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}