Prompt Caching, Explained: How Claude Reuses Your Prompt Prefix — and When It Doesn't Pay Off

Prompt caching is a feature offered by Anthropic to optimize the usage of Claude's language model in applications that interact with the model multiple times. Rather than processing the same prompt every time, Claude can reuse a previously cached prefix of the prompt, reducing the computational cost of the API requests. The cached prefix is built in a fixed order, starting with tool definitions, followed by the system prompt, and ending with the message history.

This prefix remains unchanged across requests, allowing Claude to read from the cache and write only new information.

There are two ways to enable prompt caching: automatically or manually. Automatic caching adds a single cache_control field to the top level of the request, and Claude handles the breakpoint placement. However, this may not always be optimal, as the breakpoint could end up on a block that changes every request, rendering the caching ineffective. Manual caching, on the other hand, allows developers to explicitly define breakpoints at specific points in the request, providing more control over the cached content.

To effectively utilize prompt caching, developers should order their prompts with static content first (tools, system instructions, background documents, and few-shot examples) and variable content last. This ensures that the cached prefix remains consistent across requests. Automatic caching is recommended for most cases, but developers should be aware of the minimum token thresholds for each model and ensure that the stable prefix meets these requirements.

Additionally, avoiding timestamps or per-request data in the cached blocks is crucial to prevent the creation of a different prefix hash for each request, thus maintaining the effectiveness of caching.

Prompt caching is not a magic bullet that makes Claude smarter or changes the model's output; instead, it optimizes the cost-efficiency of interacting with the model by minimizing the processing of duplicated content. When implemented correctly, prompt caching can significantly reduce the cost of using Claude's language model in multi-turn applications, making it a valuable tool for developers building AI-powered services.