Sandboxes in Kubernetes without privileged: cgroup_writable and hostUsers: false
A pod that builds sandboxes — one that runs other people's code in its own namespaces, with a cgroup per request — usually runs as privileged: true . This post shows how that line can go away, and what was tested before it was trusted. The short version: containerd 2.1 has a runtime-handler option called cgroup_writable . In a pod with hostUsers: false , runc then hands the pod its own cgroup:…
Kubernetes pods that build sandboxes for running other people's code typically run with privileged access. However, a recent update to containerd 2.1 introduces a new runtime-handler option called cgroup_writable, which allows sandboxes to have their own cgroup without needing privileged access.
With hostUsers set to false, runc will give the sandbox its own cgroup, allowing it to create subgroups and manipulate them, but it cannot raise its own limit. This approach combines the benefits of User namespaces, an unmasked /proc, and a cgroup v2 subtree where the sandbox can write, while avoiding the risks associated with privileged access.
Containerd added cgroup_writable in its 2.1 release, which mounts /sys/fs/cgroup read-write for non-privileged containers. Runc then sets the cgroup's owner to the host uid that the container's uid maps to, along with adjusting systemd cgroup driver settings. This ensures that even if a container has elevated privileges within the sandbox, it cannot write to its own memory limit.
A test conducted on k3s 1.36.5 with containerd 2.3.4 and runc 1.4.2 confirmed that the sandbox can operate below its cgroup, without affecting its own memory limit. For this to work, a containerd drop-in configuration is needed on the node, along with a RuntimeClass named "zygo" and the pod configured with the appropriate settings.
This setup allows networked sandboxes to have /dev/net/tun access, while verifying that the sandbox can operate within its limits without exceeding them. This solution is compatible with Kubernetes 1.33+, containerd 2.1+, runc, and Linux 6.3+, though it is not tested with crun.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
