Prompt caching diagnostics

Prompt cache diagnostics are tools that can help explain why a request reused fewer tokens than initially anticipated. By comparing a request to an earlier response, these diagnostics can identify changes to the model, tools, settings, or input that might have prevented the reuse of a prompt prefix. This functionality is currently available for GPT-5.6 and later supported models through the Responses API.

To investigate individual requests, use the diagnostics directly. For monitoring cache performance across an entire application, leverage the Prompt Caching Dashboard.

The diagnostics work by comparing the current request to an earlier response, specifically focusing on the exact prefix match required for reuse. This match depends on several factors, including the model being used, the service tier, and the tools employed in the request. Therefore, any changes to these settings can impact the ability of the system to reuse cached content.

The comparison response identifier (`comparison_response_id`) is used solely to request diagnostics and does not load the earlier conversation or change the caching behavior in any way.

When making requests, ensure that tool definitions and their order remain unchanged between calls to maintain cache reuse. The minimum cacheable length for the prefix is 1,024 tokens for GPT-5.6 and later models. If a tool change results in a cache miss, the outcome can be identified using the diagnostic output. Token counts may vary depending on the input, so it is crucial to keep tool definitions and ordering consistent to preserve reuse.

To compare consecutive turns, save each completed response's ID and pass it as `comparison_response_id` in the `prompt_cache_options` of the next request. For the first turn, omit the comparison ID. When setting `stream=True`, you can read the diagnostics from the `response.completed` event. The `prompt_cache_diagnostics.type` field will indicate whether a cache hit or miss occurred.

Even if new input requires additional processing, a cache hit can still be achieved if the reusable prefix from a previous response meets the model's requirements.

The diagnostic counts may not always align with actual usage or billing metrics. Therefore, always refer to the current response's usage fields to accurately measure cache reuse and associated costs. If a cache miss occurs, the `prompt_cache_diagnostics.reason` field will provide insights into the cause, along with suggested fixes. Some changes, such as switching models or compacting a conversation, might be intentional choices even if they reduce cache reuse.

Prompt cache diagnostics have no additional cost and do not count towards rate limits. However, any extra baseline or retry requests to the Responses API will be billed normally and still count toward rate limits. These diagnostics are fully compatible with Zero Data Retention, meaning OpenAI does not store raw prompts or model outputs for this feature.

Diagnostic records contain only configuration metadata, token-count estimates, and hashes to compare cache-sensitive content. These records are scoped to the organization, expire after a short period, and are used exclusively to explain cache hits or misses. Remember, setting `comparison_response_id` does not retrieve or persist the earlier response's content. For further details on data controls, refer to OpenAI's data management policies.