"Your cache hit rate is low" — true, and worth $0.16
Anthropic emailed our org last week: prompt cache hit rate is low, and caching repeated content "could save up to 64% of direct API spend." The email is careful with scope — direct API only, Claude Code excluded because it manages caching itself. I went to check what that 64% was made of. Our direct API traffic over the period: 16 calls, $0.25 total, average prompt 90 tokens. They come from…
Anthropic notified our organization that the prompt cache hit rate is low, and caching repeated content could save up to 64% of direct API spend. The email clarifies that this applies only to direct API calls, excluding Claude Code which manages its own caching. Direct API traffic over the period consisted of 16 calls, totaling $0.25 and an average prompt length of 90 tokens.
The source material reveals that these unique prompts could not be cached due to their brevity and lack of repetition. This results in a hit rate of 0%, demonstrating that the 64% figure in the email is accurate. The email overlooked certain traffic, which includes Claude Code session logs ingested into traceguard's trace store.
Over 71 days and 158 sessions, there were 58,753 API messages totaling 12,264,612,945 tokens. The table provided showcases the hit rate, input cost, and saved cost for various models, with Claude Code showing the highest savings at 83.9%. The analysis concludes that caching reduces the input side by 83.9%, cutting the overall bill by 1/5.5 compared to the scenario without caching.
Implementing keep-alive pings to maintain cache warmth is not recommended, as the counterfactual proves it would cost more than the savings it generates. To verify these findings, the author suggests running the command `python -m traceguard.routing_audit.cache_audit` on your own Claude Code transcripts or traceguard trace store.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
