Your Agent's Free-Text Output Is an API You Never Designed

Most agent demos fail at the same place, not because of the model but at the boundary where the model's output transitions to your program. Strings lack a schema, leading to code like this:

const reply = await llm.complete(prompt);

If the model writes "I would not approve this yet," your substring check might match "approve," resulting in unintended code execution.

An API emerges whenever you attempt to comprehend meaning from generated text. You don't consciously define this interface, but it exists nonetheless. This interface lacks versioning, validation, and logging capabilities. Prompt engineering can reduce failure rates but doesn't eliminate the parsing issue. Instead, ask the model for a decision, not a paragraph.

Typically, an agent step requires two different outputs: a description for humans and a decision for the program. The decision should come from a predefined set comprehended by your code. For example:

type ReviewDecision = {

status: 'pass' | 'review' | 'fail';

confidence: 'low' | 'medium' | 'high';

findings: Array<{ file: string; note: string; severity: 'info' | 'warn' | 'block' }>;

};

Now, failure modes become distinct: model-produced errors, contract violations, or implementation bugs. Validate before acting. Implementing a typed boundary ensures errors are recoverable before merging, paying or deploying.

Include evidence with the decision to make it trustworthy. Provide inputs the model saw, the allowed output space, the chosen value, and supporting evidence. This approach enables replayable records, helping identify issues when bad decisions are shipped. While typing the output doesn't guarantee correctness or replace other measures, it clarifies the control signal, making the system easier to reason about.