Introduction

When kubeadm join fails with http://localhost:10248/healthz connection refused, it means the kubelet is not running or not listening on port 10248. The most common cause is that the container runtime (Docker or containerd) is not running, or the kubelet service failed to start due to a swap-enabled system, cgroup driver mismatch, or missing configuration. Fix it by checking systemctl status kubelet, reading the kubelet logs, and resolving the underlying issue.

The Error

1[preflight] Running pre-flight checks
2[WARNING] ... connection refused
3error execution phase preflight: [preflight] Some fatal errors occurred:
4  [ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
5  [ERROR Port-10250-TCP]: Port 10250 is in use
6  [ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
7[preflight] If you know what you are doing, you can make check non-fatal with `--ignore-preflight-errors=...`

Port 10248 is the kubelet healthz endpoint. If the kubelet is not running, nothing responds on that port.

Step 1: Check Kubelet Status

1# Check if kubelet is running
2systemctl status kubelet
3
4# If it shows "inactive" or "failed", check the logs
5journalctl -xeu kubelet --no-pager | tail -50

Common log messages and their fixes:

Step 2: Disable Swap

Kubernetes requires swap to be disabled:

1# Disable swap immediately
2sudo swapoff -a
3
4# Disable swap permanently (survives reboot)
5sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
6
7# Verify swap is off
8free -h
9# Swap should show 0B for total

Step 3: Ensure Container Runtime Is Running

1# For containerd
2sudo systemctl start containerd
3sudo systemctl enable containerd
4systemctl status containerd
5
6# For Docker
7sudo systemctl start docker
8sudo systemctl enable docker
9systemctl status docker
10
11# For CRI-O
12sudo systemctl start crio
13sudo systemctl enable crio

Step 4: Fix cgroup Driver Mismatch

The kubelet and the container runtime must use the same cgroup driver (either cgroupfs or systemd):

1# Check containerd cgroup driver
2containerd config dump | grep SystemdCgroup
3# SystemdCgroup = true  means using systemd
4
5# Check kubelet cgroup driver
6cat /var/lib/kubelet/config.yaml | grep cgroupDriver
7# cgroupDriver: systemd

If they do not match, update the kubelet configuration:

1# /var/lib/kubelet/config.yaml
2apiVersion: kubelet.config.k8s.io/v1beta1
3kind: KubeletConfiguration
4cgroupDriver: systemd

Then restart kubelet:

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Step 5: Reset and Rejoin

If the node was previously joined and you need to start fresh:

1# Reset the node (removes all Kubernetes state)
2sudo kubeadm reset -f
3
4# Clean up network interfaces and iptables
5sudo rm -rf /etc/cni/net.d
6sudo iptables -F && sudo iptables -t nat -F && sudo iptables -t mangle -F
7
8# Re-enable and start kubelet
9sudo systemctl enable kubelet
10sudo systemctl start kubelet
11
12# Rejoin with the token from the control plane
13sudo kubeadm join <control-plane-ip>:6443 \
14  --token <token> \
15  --discovery-token-ca-cert-hash sha256:<hash>

Step 6: Generate a New Token If Expired

Tokens expire after 24 hours by default:

# On the control plane node
kubeadm token create --print-join-command
# Outputs the full kubeadm join command with a fresh token

Firewall and Port Requirements

Ensure these ports are open on the worker node:

1# Required ports for worker nodes
2sudo ufw allow 10250/tcp  # kubelet API
3sudo ufw allow 30000:32767/tcp  # NodePort services
4
5# Check if ports are open
6sudo ss -tlnp | grep -E '10248|10250'

Common Pitfalls

• Running kubeadm join without disabling swap first: Kubernetes does not support swap by default. The kubelet refuses to start if swap is active. Always run swapoff -a before joining and comment out swap entries in /etc/fstab for persistence.
• Mismatched cgroup drivers between kubelet and container runtime: If the container runtime uses systemd and kubelet uses cgroupfs (or vice versa), the kubelet crashes on startup. Both must use the same driver — systemd is recommended for modern distributions.
• Not running kubeadm reset before retrying a failed join: A failed kubeadm join leaves partial state in /etc/kubernetes/. Retrying without kubeadm reset causes preflight errors about existing files. Always reset before retrying.
• Using an expired join token: Tokens from kubeadm token create expire after 24 hours. If the join command was generated earlier, generate a new token with kubeadm token create --print-join-command.
• Forgetting to load required kernel modules: Kubernetes networking requires br_netfilter and overlay modules. Run modprobe br_netfilter overlay and add them to /etc/modules-load.d/k8s.conf for persistence. Without them, pod networking fails after the node joins.

Summary

• connection refused on port 10248 means the kubelet is not running — check with systemctl status kubelet
• Disable swap with swapoff -a and remove swap entries from /etc/fstab
• Ensure the container runtime (containerd/Docker/CRI-O) is running and enabled
• Match the cgroup driver between kubelet and the container runtime (use systemd)
• Run kubeadm reset before retrying a failed join, and generate a fresh token if needed