I trained my AI agent to burn less money. Here's what actually worked.
A few months ago I became convinced my AI assistant was bleeding money through incompetence. It felt true. Sessions ran long, things failed, and the meter was always spinning. So I did what I do with any suspected leak: I measured it. I streamed 45 days of local session transcripts through a script that classifies every failed tool call and prices the recovery turn that follows it. It covered 660…
A few months ago, the author became convinced their AI assistant was wasting money. To investigate, they measured the assistant's actions over 45 days using a script that identified failed tool calls and the subsequent recovery turns. The audit found 729 failed tool calls and 718 recovery turns, costing roughly 15.85 million units in total.
However, about 40% of the failures were genuine agent mistakes, while the remaining 60% were due to the environment. The environment caused issues like pages never reaching DOM-idle, screenshot injections timing out, and Windows locking files. The most significant factor was the cost of re-reading the session history, which accounted for roughly 80% of total spend.
To address this, the author made three changes: first, they archived sessions that exceeded a certain threshold, second, they checked the tiering instead of assuming it was optimal, and third, they made the measurement routine itself a habit. After implementing these changes, the weekly message count dropped significantly, and the total cache-read volume decreased over time.
The author concluded that while the AI agent was not the expensive part, their own habits were the biggest factor in the spend.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
