When Should You NOT Use an Agent?

Everyone is asking whether we should use agents, but the real question is when we should not. Teams often rush to include an agent because it's the trendy thing to do, only to later find they have reinvented a state machine with worse debugging capabilities. Agents are not inherently useful; they serve as a mechanism to defer decisions to runtime.

When your inputs, tools, and failure modes are well-understood, this deferral offers no benefits, only potential for nondeterminism. The core architectural question isn't whether to use an LLM or not; it's where the judgment boundary should sit. There are three main places where the boundary can land:

1. A single agent acts as a program. For a well-defined task with a known toolset, you don't need a loop at all. The LLM should be just one component within a deterministic pipeline, not the orchestrator.

2. When a loop is necessary, you're essentially building a runtime. A runtime comes with its own set of complexities, including observability, tool permissions, credit and rate limits, and the ability to explain actions post-execution. This isn't merely more AI; it's distributed systems with a language model acting as the cognitive layer.

3. The most expensive failure occurs when capabilities emerge that you didn't initially design for, such as open-ended tool calls, emergent side-effects, or scaling that leads to budget overruns or audit issues. Maturity in this context lies not in the cleverness of the model, but in the constraints surrounding it: entitlements, approval processes, observability, and evaluations that cover failure paths, not just the optimal ones.

Most systems labeled as "agent architecture" are actually a misplacement of the decision-making boundary, wrapped in additional layers. If you find yourself reaching for a framework, an orchestrator, or a runtime, pause and consider what you're actually deferring, and whether that deferral is truly effective or merely complicating debugging efforts. Here are a few heuristics I use before constructing an agent-based system:

- Can the task be broken down into steps with a known order? If so, construct a pipeline with the LLM as a step, not an agent.

- Does the agent call known tools with expected outputs? If yes, a single, well-governed agent on top of a deterministic core may suffice.

- Do you need to route the task mid-way to unexpected states? Only then should you consider building a real runtime, but only if you're prepared to handle its observability, limits, and audit trail as first-class work.

- Is the failure non-recoverable? In such cases, it's best not to place an agent in the loop at all. A single misapplied tool call inside a runtime can cause more damage than a slower, deterministic path ever could.

While the agent approach might seem more appealing, it is also much harder to explain three months down the line. Therefore, when scoping a new system, ask yourself what makes you reach for a deterministic pipeline instead of an agent—or the other way around. I'm interested in the rules teams actually follow, not the ones presented in talks.