Your Kubernetes node is smaller than you think: 3 reservation traps in EKS, GKE and AKS
You buy a 16 GiB node. Kubernetes lets your pods use 11.9 GiB of it. The other 4 GiB go to kubelet reservations you never configured — and the managed services changed the rules recently, so most blog posts and calculators still show the old numbers. I went through the EKS nodeadm source and the current GKE and AKS docs and computed allocatable CPU and memory for every instance type. Three things…
Three notable "reservation traps" exist when sizing Kubernetes nodes in EKS, GKE, and AKS. Firstly, EKS reserves 4 GiB of memory on a 16 GiB node by default, due to kubelet reservations. However, EKS introduced prefix delegation in recent versions, which can reduce the reserved memory by up to a gigabyte per possible pod. For instance, with the default VPC CNI, a t3.medium node has 442 MiB reserved memory, leaving only 62% of RAM usable.
Enabling prefix delegation increases this to 1465 MiB reserved, resulting in 62% of RAM being usable. This can be advantageous on larger nodes but less so on smaller ones.
Secondly, GKE experienced a change in memory reservation starting with version 1.37. Previously, GKE reserved a progressive share of memory - 25% of the first 4 GiB, 20% of the next 4, and so on. With the introduction of Container-Optimized OS, the reservation formula changed to the minimum of the old value or 15 MiB multiplied by the maximum possible pods, plus 500 MiB.
This adjustment resulted in significantly reduced reserved memory for nodes with a large number of pods. For example, an n2-standard-16 node with 64 GiB memory saw its reserved memory drop from 5.5 GiB to 2.1 GiB when upgrading to GKE 1.37.
Lastly, AKS has a default reservation of 250 pods for every 16 GiB node when using Azure CNI Overlay. This reservation is equivalent to 25% of the node's RAM. For a D4s v5 node, this means 4 GiB of memory is reserved, leaving 74% of RAM usable. However, changing the maxPods value requires creating a new node pool and migrating workloads, as the value can't be altered on existing pools.
It's crucial to size pod requests against the node's allocatable resources, typically found via kubectl describe node, rather than relying on the maximum pod limit imposed by the CNI.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
