Cloudflare Workers caching: store values, not in-flight promises

Cloudflare Workers caching offers a method to store completed data values, rather than pending in-flight promises. The rationale behind this approach is to avoid sharing a pending operation, as the work behind that promise may be tied to the specific request that initiated it. The recommended strategy is to share completed values and let concurrent requests perform their own reads.

The article provides a JavaScript implementation of this technique, accompanied by tests that demonstrate its effectiveness in avoiding issues where a request reads data that hasn't completed, while another request still waits for an answer. Deduplicating concurrent work can be useful in many JavaScript applications, but it's crucial to understand that sharing a promise can lead to unintended consequences.

Cloudflare explicitly warns against the practice of performing I/O (Input/Output) on behalf of a different request, emphasizing that a completed JSON object is fundamentally different from a live response body. The key distinction lies in whether a later invocation needs the work or resources from an earlier one.

The solution involves separating the read from the reusable value, following these three steps:

1. Return an unexpired value if one exists.

2. On a miss, let this invocation perform and finish its own read.

3. Store the resulting data after that read succeeds.

This approach leads to two simultaneous misses, resulting in two independent reads, which is the intended behavior. By using this pattern, concurrent misses read independently, with neither caller waiting for the in-flight operation of the other.

The provided code example demonstrates a small standalone cache utility, suitable for public data with a TTL (Time To Live) and a limit on the number of entries. The example shows how a Worker can read a configuration file from its static assets binding, parse it, and cache the resulting data for subsequent requests.