When Nodes Start Rejecting Pods: Taints and Tolerations
TL;DR Taints go on nodes, tolerations go on pods. A tainted node rejects pods by default, and a toleration is what lets a specific pod back in — the opposite of node selectors and affinity, where the pod was the one doing the choosing. Three effects: NoSchedule , PreferNoSchedule , NoExecute . NoExecute is the odd one out. It can evict pods that are already running, not just block new ones from…
Taints and tolerations are mechanisms used in Kubernetes to control which pods can run on which nodes. A taint is applied to a node and functions like bug repellent, repelling pods from the node unless they have a specific tolerance. A toleration is applied to a pod and allows it to ignore certain taints, essentially granting the pod immunity from the taint's repelling effect.
Taints can have three different effects: NoSchedule, PreferNoSchedule, and NoExecute. NoExecute is unique in that it can evict pods that are already running on the node, not just block new ones from landing.
The main use case for taints and tolerations is to prevent pods from running on certain nodes by default, such as control-plane nodes or dedicated GPU nodes. This ensures that regular application pods do not compete with control plane resources or specialized hardware. Taints and tolerations are particularly useful when you want to reject pods from a node by default, unless the pod explicitly tolerates that taint.
In practice, taints are applied to nodes using commands like `kubectl taint nodes node-1 dedicated=gpu:NoSchedule`, while tolerations are specified in a pod's definition, for example:
```
apiVersion: v1
kind: Pod
metadata:
name: gpu-pod
spec:
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoSchedule
containers:
- name: nginx
image: nginx
```
When a taint is applied to a live node, it only affects future scheduling decisions and does not impact running pods. However, if the taint is NoExecute, it will immediately evict any pods without a matching toleration or after a set grace period if tolerationSeconds is specified. This is the key difference between taints and tolerations, as tolerations only remove restrictions, not express preferences for scheduling.
Common mistakes when using taints and tolerations include confusing a toleration with a preference for a node, and failing to consider the impact of NoExecute taints on running pods. It is crucial to check what pods are already running on a node and what tolerations they have before applying taints, especially if those pods are essential for the operation of your cluster. Additionally, DaemonSet pods automatically tolerate several built-in taints, which is a feature worth exploring further in the documentation.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
