I measured one AI agent session token by token. 5 of 20 turns ate 94% of the cost.

In a recent account, an individual shared their experience measuring the actual cost of running AI agents. Rather than relying on estimates, they closely examined the usage per turn in a specific session, revealing surprising insights.

For a 20-turn session, the effective cost amounted to approximately 934,000 tokens. This figure took into account various factors, including cache reads and writes, as well as the costs associated with input and output operations. The minimum cost per turn was found to be 146,000 tokens, which encompassed the expenses of tools and plugins loaded during each turn.

Interestingly, the data showed that out of the 20 turns, five of them (25%) contributed to 94% of the total cache-write cost. These turns were not necessarily the most challenging ones; instead, they coincided with changes in the tool list. Specifically, an MCP server disconnecting and reconnecting triggered the invalidation of cached prefixes, necessitating the rewriting of the entire context. This process incurred twice the cost of a regular read operation.

The author drew three key conclusions from their findings. Firstly, the per-turn floor cost may be significantly higher than anticipated. Secondly, the expense lies not in repeating context but in losing and rewriting it. Lastly, maintaining a stable tool list proves to be more cost-effective than shortening prompts.

One crucial takeaway is that what appears green at first glance may not be as cost-effective upon closer examination. It is essential to measure the true impact and costs associated with AI agents in real-world scenarios rather than relying on assumptions or incomplete data. The author emphasizes the importance of measuring by effect on the actual runs.

However, it is important to note that the measurements presented in this account are specific to the particular session and setup. Therefore, it is strongly advised not to generalize these numbers or conclusions without conducting one's own measurements. The author has compiled a kit containing the methodology, lane templates, and a consumption guard—a small Python script that reads session transcripts and provides a detailed breakdown of costs. This kit can be accessed at the following link: https://spataro1.gumroad.com/l/ghjrgi.