{
  "id": 3552302,
  "title": "cached_tokens is 0 because your system prompt isn't stable",
  "url": "https://urgent.news/2026/08/26/cached-tokens-is-0-because-your-system-prompt-isnt-stable",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-08-26T17:05:51.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/chad_4ba286df4a1dc781a641/cachedtokens-is-0-because-your-system-prompt-isnt-stable-466h"
  },
  "original_language": "en",
  "account": "In the story \"cached_tokens is 0 because your system prompt isn't stable\" by Chad Priest, the author discovered a significant issue with their system prompt while using the Fireworks service. The problem stemmed from the system prompt not being stable, leading to the same prefix being used repeatedly, which resulted in full pricing for the same content. This issue cost the author more money than anticipated.\n\nThe author initially suspected session affinity, but it turned out not to be the main culprit. They then found that caching was automatic and total, requiring the prefix to be identical for the cache to work. They identified two main problems: the ~24KB workspace bootstrap rode turn 1 and vanished on turn 2, and the ~15K query-dependent memory recall was concatenated straight onto the system message.\n\nTo fix this, the author implemented a branch in the code, freezing the system prompt to make it cacheable. They then relocated the memory for system use to a late system turn, before the current user message. This allowed the cached prefix to be recognized and utilized effectively. The author also fixed the bootstrap issue by resending it every turn, resulting in a more efficient billing structure.\n\nIn conclusion, the key rule for a stable system prompt in a frozen-prefix design is to treat the system prompt as an invariant data structure, with any query-dependent elements placed within the message array. Understanding the discount provided by caching is crucial, as the effectiveness of a cache can vary significantly depending on the amount of input it can handle.",
  "summary": "I added a [cache] log line next to our usage tracker and read usage.prompt_tokens_details.cached_tokens off a warm conversation. It said 0 . Then 1 . Out of roughly 16,000 prompt tokens, every single turn. We were paying full price for the same prefix over and over, and the 273K-token board balloon I'd been blaming on the agent loop was just real money. What I was wrong about for the first day My…",
  "key_points": [
    "Cachedtokens remained at 0 due to unstable system prompt",
    "Author discovered issue with system prompt not being stable",
    "Fixed by freezing system prompt and relocating memory to late system turn"
  ],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}