Harnesses for Personal AI Agents: The Code Around the Model
Ask people which personal AI agent to use and the debate quickly turns to models: which is smartest, cheapest, or remembers the most. After working through a handful of recent podcast episodes about agents, I've come to think the better question, for builders and curious users alike, is a less glamorous one. What sits around the model? Claire Vo gives the cleanest definition I've heard, in an…
The debate surrounding personal AI agents often centers on selecting the most suitable model - the one with the highest intelligence, lowest cost, or greatest memory. However, Claire Vo suggests that the more pertinent question for builders and users is what lies "around" the model, referring to what she calls a "harness." A harness is simply the code surrounding an AI agent, which enhances its effectiveness.
It dictates what the agent sees, the tools it can utilize, and what constitutes completion. Vo argues that a harness becomes valuable when dealing with repetitive workflows that have consistent setups and expected outcomes. For instance, bug triage is a perfect use case for a harness. Sentry error notifications require reading stack traces, locating related code, checking recent deployments, opening tickets, and possibly drafting fixes.
A general chatbot would need to explain the context repeatedly, while a harness can wire the agent to the relevant services - Sentry, Linear, GitHub, and Vercel - and provide a simple terminal UI for human oversight. The harness applies a principle of constraint, allowing very specific agents with the right tools to effectively solve narrowly defined problems, often outperforming highly intelligent agents with no boundaries.
The concept of harnesses extends to consumer personal agents as well. When evaluating a new consumer agent, a suggested test is to give it a task with a verifiable outcome and examine three factors: progress reporting, asking before important actions, and the ability to stop. These factors correspond to progress reporting as an artifact, asking before acting within the tools layer, and the ability to halt the process as part of the run.
Greg Isenberg draws a parallel from the merchant's perspective, emphasizing trust as a ladder - initially allowing an agent to research, then prepare, and finally pay. The payment step is where most friction arises, and a simple threshold within the harness can significantly improve safety without extensive prompt engineering. Dario Amodei warns that agents require extremely high per-step accuracy around 99.9% due to the compounding nature of errors across long chains of actions.
To mitigate this, harnesses can shorten chains by pre-fetching context, checking outputs between steps, and halting when a human intervention is required. Amodei also notes that models will likely make mistakes far less often than humans, but these mistakes will be stranger, and deterministic checks around the model can help catch these anomalies.
When considering wrapping an agent around your own workflow, Vo recommends starting with clear success criteria, listing the tools involved and discarding half of them, starting with read-only mode, deciding where human intervention is necessary, logging every run as an artifact for debugging and measuring efficiency, and tracking cost per run to ensure the harness adds value.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
