Two-step control plane upgrades in GKE: How minor version rollbacks work under the hood
Kubernetes control plane minor version upgrades have historically been an all-or-nothing proposition. In standard Kubernetes clusters, upgrading a control plane from one minor version to the next—such as moving from 1.33 to 1.34—commits changes to storage schemas immediately. If an unexpected regression emerges after upgrading the API server, rolling back to the previous minor version was…
The article explains the process of two-step control plane upgrades in Google Kubernetes Engine (GKE). Historically, upgrading a Kubernetes control plane from one minor version to another resulted in immediate storage schema changes, making rollbacks difficult or impossible. GKE addressed this issue by introducing two-step upgrades, which decouple binary execution from API capability enablement.
This allows for a safer upgrade process with an observation period called a soak window. During the first step, the control plane binary is upgraded to the new minor version while emulating the previous version. If the soak window is completed without issues, the emulated version is finalized, permanently enabling the new API schemas and deprecations.
Rollbacks are no longer possible after this step. Automated orchestration is possible using Control Plane Rollout Service (CPRS) and Canary Analysis Service (CAS) for clusters configured for auto-upgrades. For manual upgrades, Google Cloud CLI and Terraform can be used to initiate the process with a custom soak duration. Verification of upgrade status can be done using the CLI, and rollbacks can be executed if necessary.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
