Your AI agent is only as good as the harness around it

An AI agent’s effectiveness depends heavily on the infrastructure—its "harness"—surrounding it. While a well-trained model can provide impressive responses in a demonstration setting, real-world deployment requires additional structure. This harness includes several critical elements.

First, the harness defines tool contracts that limit what a model can do when it makes incorrect tool calls. For instance, specifying required input formats, output expectations, timeouts, and error classifications ensures the model’s requests are well-formed and unexpected issues are handled gracefully. One example tool contract is for updating a billing plan, including mandatory idempotency keys and clearly defined error states to prevent repeated charges from occurring.

Second, permissions and access controls are crucial. The model lacks inherent knowledge of business policies and must receive these rules from the surrounding system. For example, an agent drafting a customer support response needs not only the knowledge base but also context about account statuses and approval procedures. Permissions determine what actions an agent can perform, safeguarding against accidental or malicious actions, such as sending messages to incorrect recipients or accessing sensitive data.

Third, the harness includes mechanisms to monitor and analyze agent behavior. Trace records and test failures help teams identify issues early. If an agent fails to retrieve a required piece of information or encounters a tool error, the trace makes it possible to diagnose the problem quickly. This is essential because many production failures stem from interactions between the model and the underlying systems, where the model's reasoning is only part of the solution.

Finally, separating read and write operations adds an extra layer of safety. Read tools provide information without changing state, while write tools require additional confirmation and permission checks before executing. This separation ensures that changes are deliberate and verified by human oversight, reducing the risk of costly errors. By treating the model as a single component within a larger harness, developers can build robust systems that perform reliably in production.