Your agent's cost problem isn't the model. It's the steps you never measured.
A few weeks back I watched an agentic pipeline burn a month of budget in three days. The instinct everyone reached for was the obvious one: "the frontier model is too expensive, switch to a smaller one." That's the wrong diagnosis, and it's worth writing down why. When we finally tagged every token to the (step, tool, model) tuple that spent it, the picture was uncomfortable. The expensive model…
A recent incident showed how an agent's budget could be consumed in just three days. The immediate reaction was to switch to a cheaper model, but that isn't the real problem. By tracking every token used by each step, call, and model, the true culprits became apparent.
The large model wasn't the issue, but rather it was automatically used for every step, from simple to complex tasks. The problem was that there was no way to identify which specific step was causing the high cost. The expense was a tree structure, not just a single number.
In a single LLM call, cost is a single line item, but in a multi-step agent loop, it becomes a complex tree. Optimizing based solely on the total cost means making assumptions, usually opting for the smallest model everywhere. However, this approach could sacrifice quality in the steps that actually need the high-performing model, while ignoring the true waste.
Most agent frameworks allow setting a single model globally. This inflexible model choice gets applied to all steps, even those that could handle a smaller model efficiently. By logging per-step data like step id, tool, model, input/output tokens, latency, and retry count, the waste becomes obvious in minutes. Most expensive steps are rarely the ones you'd initially suspect, often being repeated retries due to tool errors rather than model missteps.
Failover to a different model for a single problematic step adds to costs when things break. A better approach is to route the failing step to a cost-effective fallback model. This is where the real cost optimization happens, not in the prompts or the model selection itself. Routing decisions at the layer where models are selected can both measure spend and take action, making it the crucial place for cost governance and compliance.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
