How to make an AI coding agent keep fixing until npm test passes

An AI coding agent can be guided to keep fixing until npm test passes by implementing a specific loop in the osFoundry desktop app. First, choose the "Get the JS suite green" loop from the app's starter loops. This loop has a built-in mechanism for determining when it's finished: npm test must return a zero exit code. The loop runs in rounds, with each round graded against the goal by a separate grading call, not by the agent itself. The app then runs npm test itself after each round.

To start the loop, open the app, go to "Open Loops," click "New session," and search for "JS suite." Select the "Get the JS suite green" loop. Before beginning, read the card on the right to understand how the loop decides it's finished. The loop's instructions prohibit changing test expectations, as this may lead to incorrect fixes.

Additionally, the loop has limitations, such as a maximum of four rounds and a cap of three consecutive verification failures, which helps prevent the loop from continuing when the problem is a design issue rather than a mechanical one.

Once the loop is running, monitor its progress in the "Rounds" tab. Each round displays the number of goal criteria met and indicates progress with a +1. After three rounds, if the loop has fixed all the failing tests, it will stop on its own. If the loop runs out of rounds or hits the failure cap, it will not say "Verified," and the rounds will show where it got stuck. This usually indicates that the test is wrong or a decision needs to be made that the test runner cannot make.