Give Your Coding Agent Scripts, Not Prose: Build a Tool-Contract Doctor First

A coding agent with a folder of scripts and a contract test will outperform one with a beautifully written instruction file. The reason for this is that scripts fail loudly, whereas prose does not. Unfortunately, many AGENTS.md files appear to be onboarding documents for a new hire. While these may be pleasant to read, they are also prone to becoming outdated as the repository changes. Running the tests is the only way to ensure that the instructions are up-to-date.

The process from an empty repository to a green tool doctor that gates what the agent can call involves several stages. First, it is essential to determine whether the task is a decision or a procedure. Decisions should be written in prose, while procedures should be written in scripts. The agent needs to perform a repeatable step that is stable and report whether it was successful. The exit code should be verified in the Continuous Integration (CI) system, rather than relying on a testable test.

The contract between the agent and the tool should be clearly defined. The contract should include three failure classes and one JSON shape, along with one timeout. By defining the contract in this way, the agent can receive a stable surface instead of a hopeful description. The contract should consist of three exit codes: success, failure due to a missing required flag, and failure due to a missing path.

The happy path should emit parseable JSON with named keys, and the contract should include three failure classes and three exit codes. Vague failures are the reason agents retry indefinitely.

After writing the contract, the next step is to write the doctor that reads the contract. The doctor should validate the tool interfaces before handing them to the agent. The doctor should import the necessary libraries, set a timeout for the subprocess.run command, and run the appropriate command based on the case. If the command is not found, the doctor should return a problem indicating that the command was not found.

If the subprocess times out, the doctor should return a problem indicating that the command timed out. The doctor should then check the return code of the subprocess and compare it to the expected exit code in the contract. If the exit codes do not match, the doctor should append a problem indicating the mismatch. If the JSON output is expected, the doctor should attempt to load the JSON from the subprocess's stdout.

If the JSON cannot be loaded, the doctor should append a problem indicating that the stdout is not JSON.