Docker BuildKit Cache Setup That Actually Speeds Up CI

The client's Docker builds were taking an excessive nine minutes for every pull request. They had implemented --cache-from in their GitHub Actions workflow, but the BuildKit cache had never actually been utilized in the past three months. This was due to BuildKit pulling a stale :latest tag as cache source, which resulted in unnecessary full rebuilds. The most common failure mode of Docker BuildKit cache in CI setups is this invisible silent miss, which only becomes apparent upon timing the build.

BuildKit cache functions differently than the legacy Docker build cache. It computes a digest based on the base image digest, build context checksum for copied files, and the resolved command. This means that even if two builds have identical Dockerfiles, different base image digests will cause them to miss cache, despite the text being byte-identical. There are three cache backends relevant in CI: inline cache (BUILDKIT_INLINE_CACHE=1), registry cache (type=registry), and local/GHA cache (type=local or type=gha).

A common mistake is pulling --cache-from myimage:latest as the cache source, which leads to silent misses due to :latest drift. The second mistake involves placing COPY . . before npm ci or pip install, causing every commit to invalidate the dependency install layer. Additionally, relying solely on the CI runner's local disk for cache can be problematic, as GitHub-hosted runners are ephemeral and discard the warm cache after each job.

To optimize Docker BuildKit cache in CI, the structure of the Dockerfile should be ordered from least volatile to most volatile. Copy lockfiles first, install dependencies, and then copy the source code last. Use pinned digests for FROM statements to avoid cache invalidation from upstream tag changes. For the backend, registry cache is the most portable option, working irrespective of the runner that triggers the job.