Harness Engineering 101: How Coding Agents Actually Work

Harness engineering is the process of designing the software that runs around a model to optimize its performance in completing tasks. A key component of harness engineering is the agent, which is essentially the model combined with the harness. In a study published on arXiv in August, it was found that changing the harness can significantly impact a model's performance in bug fixing. The study started with 43 tasks and increased the number to 72 by adjusting the harness.

An agent harness comprises several elements, such as the loop that keeps the agent working, the tools it can call upon, the context window, the rules governing its actions, and the verification process for its work. Examples of harnesses include Claude Code and Codex CLI. The concept of harness engineering has evolved in three stages: prompt engineering, context engineering, and now harness engineering, which focuses on everything the model needs to act rather than just generate text.

The loop engineering, the next step in harness engineering, involves wrapping the harness in additional loops that re-run the agent based on certain conditions. This ensures the agent completes its tasks without human intervention. The pseudocode provided illustrates the core of a coding agent in a simplified form. It includes steps like initializing the context with system prompts and project memory, generating responses from the model, verifying the replies, and handling tool calls based on predefined policies.

The context limit, or the maximum number of tokens the model can handle, plays a crucial role in the agent's performance. In the study, the harness shortened older tool results as the context window filled, effectively increasing the available context to 262K tokens, which helped prevent the agent from finishing early due to context anxiety. This highlights the importance of managing the context limit in production environments, where every token has a cost.