The DNS Drift That Wasn't DHCP: How Proxmox Cloud-Init Silently Injects Search Domains
Originally published at woitzik.dev Disclosure: This post contains Amazon affiliate links (marked with *). If you buy through them, I earn a small commission at no extra cost to you. I only link gear I actually own and use daily. Every DNS lookup from every pod in my k3s cluster was appending home.lan to the query. External lookups like registry-1.docker.io became registry-1.docker.io.home.lan ,…
A recent incident revealed an unexpected source of DNS latency in a Proxmox Kubernetes cluster deployment. External queries from pods experienced an average delay of 341 milliseconds, primarily due to the search domain "home.lan" being silently injected by Proxmox's cloud-init mechanism. Initially, the assumption was that DHCP was responsible, but further investigation uncovered that this domain was instead being set by Proxmox itself.
The root cause lay in how Proxmox's cloud-init generator populated the search domain in the VM's netplan configuration, which in turn influenced the kubelet's DNS resolution process. The investigation demonstrated that DNS configuration involves multiple layers, and an issue in one layer might not be immediately apparent when checking others.
To resolve the issue, the cloud-init configuration on the Proxmox nodes needed to be altered to unset the default search domain explicitly. This situation underscores the complexity of multi-layered DNS configurations and the importance of thoroughly examining all relevant configuration sources when diagnosing DNS-related issues.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
