Kubernetes
Node Labeling
Cloud Computing
DevOps
Containers

How to label Kubernetes node?

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Kubernetes node labels are key-value pairs attached to nodes that enable workload scheduling, organization, and selection. Labels are assigned with kubectl label nodes <node> key=value and used in Pod specifications via nodeSelector or nodeAffinity to control which nodes a Pod runs on. Common use cases include marking nodes by hardware (GPU, SSD), environment (production, staging), region, or team ownership. Labels do not affect node behavior — they only provide metadata for scheduling and querying.

Adding Labels to Nodes

bash
1# Add a single label
2kubectl label nodes worker-1 environment=production
3
4# Add multiple labels
5kubectl label nodes worker-1 tier=frontend team=web
6
7# Verify labels
8kubectl get nodes worker-1 --show-labels
9
10# Show specific label columns
11kubectl get nodes -L environment,tier

Viewing Node Labels

bash
1# Show all nodes with labels
2kubectl get nodes --show-labels
3
4# Filter nodes by label
5kubectl get nodes -l environment=production
6
7# Filter with multiple labels (AND logic)
8kubectl get nodes -l environment=production,tier=frontend
9
10# Set-based selectors
11kubectl get nodes -l 'environment in (production, staging)'
12kubectl get nodes -l 'environment notin (development)'
13kubectl get nodes -l 'gpu'      # Label exists (any value)
14kubectl get nodes -l '!gpu'     # Label does not exist

Updating and Removing Labels

bash
1# Update an existing label (requires --overwrite)
2kubectl label nodes worker-1 environment=staging --overwrite
3
4# Remove a label (append - to the key)
5kubectl label nodes worker-1 tier-
6
7# Remove labels from all nodes
8kubectl label nodes --all temporary-label-

Using Labels with nodeSelector

The simplest way to schedule Pods on labeled nodes:

yaml
1apiVersion: v1
2kind: Pod
3metadata:
4  name: gpu-pod
5spec:
6  nodeSelector:
7    gpu: "true"
8    environment: production
9  containers:
10    - name: ml-training
11      image: pytorch/pytorch:latest
12      resources:
13        limits:
14          nvidia.com/gpu: 1

The Pod only runs on nodes that have both gpu=true AND environment=production labels.

Using Node Affinity (Advanced)

Node affinity provides more expressive scheduling rules:

yaml
1apiVersion: v1
2kind: Pod
3metadata:
4  name: web-pod
5spec:
6  affinity:
7    nodeAffinity:
8      requiredDuringSchedulingIgnoredDuringExecution:
9        nodeSelectorTerms:
10          - matchExpressions:
11              - key: environment
12                operator: In
13                values:
14                  - production
15                  - staging
16              - key: tier
17                operator: NotIn
18                values:
19                  - backend
20      preferredDuringSchedulingIgnoredDuringExecution:
21        - weight: 80
22          preference:
23            matchExpressions:
24              - key: region
25                operator: In
26                values:
27                  - us-east-1
28  containers:
29    - name: web
30      image: nginx:latest
  • required: Pod must run on matching nodes (hard constraint)
  • preferred: Scheduler tries matching nodes but falls back if unavailable (soft constraint)

Labeling Nodes in a Manifest

yaml
1# node-labels.yaml — apply with kubectl
2apiVersion: v1
3kind: Node
4metadata:
5  name: worker-1
6  labels:
7    environment: production
8    tier: frontend
9    region: us-east-1
10    disk-type: ssd
11    gpu: "true"
bash
kubectl apply -f node-labels.yaml

Common Label Conventions

LabelPurposeExample Values
environmentDeployment stageproduction, staging, development
tierApplication tierfrontend, backend, database
regionGeographic regionus-east-1, eu-west-1
disk-typeStorage typessd, hdd
gpuGPU availabilitytrue, nvidia-a100
teamOwning teamplatform, ml, web
kubernetes.io/osOperating system (built-in)linux, windows
node.kubernetes.io/instance-typeCloud instance type (built-in)m5.xlarge

Automation with Scripts

bash
1#!/bin/bash
2# Label all GPU nodes
3for node in $(kubectl get nodes -o name | grep gpu); do
4    kubectl label "$node" gpu=true accelerator=nvidia --overwrite
5done
6
7# Label nodes by instance type from cloud provider metadata
8for node in $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}'); do
9    instance_type=$(kubectl get node "$node" -o jsonpath='{.metadata.labels.node\.kubernetes\.io/instance-type}')
10    if [[ "$instance_type" == *"gpu"* ]]; then
11        kubectl label node "$node" workload=ml --overwrite
12    fi
13done

Common Pitfalls

  • Forgetting --overwrite when updating a label: kubectl label nodes worker-1 env=staging fails if the env label already exists. Add --overwrite to update existing labels.
  • Using invalid label key/value characters: Label keys must match [a-zA-Z0-9._-] and be at most 63 characters. Values follow the same rules. Keys can optionally include a DNS prefix like company.com/role. Invalid characters cause a validation error.
  • Relying on labels for security isolation: Labels are metadata only — they do not enforce security boundaries. A misconfigured nodeSelector or a missing label does not prevent a Pod from being scheduled on the wrong node if the scheduler falls back. Use taints and tolerations for hard isolation.
  • Not labeling nodes consistently across the cluster: If some nodes have env=production and others have environment=production, selectors match different sets. Establish naming conventions and enforce them with automation or admission controllers.
  • Removing a label that active Pods depend on: Removing a label from a node does not evict Pods already running on it (IgnoredDuringExecution). However, new Pods with nodeSelector for that label will not be scheduled on the node. Plan label changes carefully.

Summary

  • Add labels with kubectl label nodes <node> key=value — use --overwrite to update
  • Remove labels by appending - to the key: kubectl label nodes <node> key-
  • Use nodeSelector for simple scheduling constraints (exact match)
  • Use nodeAffinity for complex rules (In, NotIn, Exists, required vs preferred)
  • Follow consistent naming conventions for labels across the cluster
  • Labels are metadata only — use taints/tolerations for hard scheduling constraints

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design