Urgent.News

What's breaking now, across thousands of outlets.

More in AI

cached_tokens is 0 because your system prompt isn't stable

I added a [cache] log line next to our usage tracker and read usage.prompt_tokens_details.cached_tokens off a warm conversation. It said 0 . Then 1 .

  • Cachedtokens remained at 0 due to unstable system prompt
  • Author discovered issue with system prompt not being stable
  • Fixed by freezing system prompt and relocating memory to late system turn

More from Wednesday 26 August →