{
  "id": 520698,
  "title": "Watch out for cache read costs",
  "url": "https://urgent.news/2026/08/10/watch-out-for-cache-read-costs",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-10T22:33:28.000Z",
  "source": {
    "name": "Lobsters",
    "slug": "lobsters",
    "url": "https://martinalderson.com/posts/watch-out-for-cache-read-costs/"
  },
  "original_language": "en",
  "account": "In the world of language models, cache read costs have gained prominence. OpenRouter's pricing tables often focus on input and output costs per million tokens, but cache read costs are actually more crucial. Agentic workloads, in particular, suffer from high cache read costs. To illustrate, consider a hypothetical agentic session with a starting context length of 60k tokens. Each tool call results in 500 tokens written and 5,000 tokens read. After 20 turns, cache reads make up nearly half the bill. As the session extends to 100 turns, the impact of cache reads becomes even more pronounced. The cost grows quadratically with the number of turns, while only 5.5k tokens are added per turn. This highlights the importance of reducing the number of tool calls per run, as it significantly impacts costs. Improving the agent's tools and reducing the number of turns can lower costs by around 16%. With the rapid increase in context window sizes, their memory footprint has decreased. Companies like DeepSeek have achieved impressive compression rates, allowing a 1-5GB KV cache to be stored in system memory or NVMe flash drives. This has led to a decrease in NVMe costs. DeepSeek offers cache reads at a discounted price compared to other providers, but rumors suggest they may increase this price soon. Cache reads effectively charge over and over again for a few GB of (V)RAM rental. Serving architectures allow booting the cache from VRAM to system RAM or NVMe, making it rent a few GB of system RAM at a substantial markup. At 100 turns, GPT 5.6 Sol costs $36.20, while Opus 5 costs $21.34 due to OpenAI's pricing structure. Anthropic, on the other hand, charges the same per-token rate regardless of the token count. This discrepancy can add 76% to the bill for GPT 5.6 Sol at 100 turns. Cache read costs are likely to become the main cost to consider for many agentic tasks. The innovation in cache sizes and pricing mechanisms has lagged behind, and the impact varies depending on the specific use case.",
  "summary": null,
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}