Kubernetes
service account
token reading error
troubleshooting
container security

Error reading service account token from /var/run/secrets/kubernetes.io/serviceaccount/token. Ignoring

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

The warning Error reading service account token from /var/run/secrets/kubernetes.io/serviceaccount/token. Ignoring appears when an application running inside a Kubernetes pod tries to read the automatically mounted service account token but fails. This is common when running applications locally outside of Kubernetes, or when the service account token volume has been explicitly disabled.

Why This Happens

By default, Kubernetes mounts a service account token into every pod at the path /var/run/secrets/kubernetes.io/serviceaccount/. This token allows the pod to authenticate with the Kubernetes API server. The error occurs when:

1. Running Outside Kubernetes

If you run your application locally (e.g., during development), the path /var/run/secrets/kubernetes.io/serviceaccount/token does not exist. Libraries like the Kubernetes client for Python, Java, or Go automatically attempt to detect if they are running in-cluster by reading this token:

python
1# Python kubernetes client — automatic in-cluster detection
2from kubernetes import client, config
3
4try:
5    config.load_incluster_config()  # Reads the service account token
6except config.ConfigException:
7    config.load_kube_config()       # Falls back to local kubeconfig

2. Service Account Token Mounting Disabled

Starting with Kubernetes 1.24, you can disable automatic token mounting:

yaml
1apiVersion: v1
2kind: Pod
3metadata:
4  name: my-pod
5spec:
6  automountServiceAccountToken: false  # Token will NOT be mounted
7  containers:
8    - name: app
9      image: my-app:latest

Or at the ServiceAccount level:

yaml
1apiVersion: v1
2kind: ServiceAccount
3metadata:
4  name: my-sa
5automountServiceAccountToken: false

3. Token Volume Not Yet Mounted

In rare cases, the kubelet has not yet mounted the projected volume when the application starts. This is a timing issue that typically resolves on its own.

4. Incorrect Permissions

The token file exists but the container process does not have read permissions:

bash
# Check file permissions inside the container
kubectl exec my-pod -- ls -la /var/run/secrets/kubernetes.io/serviceaccount/

How to Fix It

If Running Locally (Not in Kubernetes)

This warning is expected and can safely be ignored. Ensure your application falls back to local configuration:

python
1# Python example with proper fallback
2from kubernetes import client, config
3
4def get_k8s_client():
5    try:
6        config.load_incluster_config()
7        print("Running in-cluster")
8    except config.ConfigException:
9        config.load_kube_config()
10        print("Running locally with kubeconfig")
11    return client.CoreV1Api()
java
1// Java example with proper fallback
2try {
3    ApiClient client = ClientBuilder.cluster().build();
4} catch (IOException e) {
5    ApiClient client = ClientBuilder.defaultClient().build();
6}

If Running in Kubernetes and Token is Needed

Verify the service account token is mounted:

bash
1# Check if the volume is mounted
2kubectl describe pod my-pod | grep -A 5 "Mounts:"
3
4# Verify the token file exists
5kubectl exec my-pod -- cat /var/run/secrets/kubernetes.io/serviceaccount/token

Ensure automountServiceAccountToken is not set to false:

bash
1# Check the pod spec
2kubectl get pod my-pod -o yaml | grep automount
3
4# Check the service account
5kubectl get sa default -o yaml | grep automount

If You Do Not Need Kubernetes API Access

If your application does not need to talk to the Kubernetes API, you can safely suppress the warning. Disable token mounting to reduce the attack surface:

yaml
1apiVersion: v1
2kind: Pod
3metadata:
4  name: my-pod
5spec:
6  automountServiceAccountToken: false
7  containers:
8    - name: app
9      image: my-app:latest

Bound Service Account Tokens (Kubernetes 1.22+)

Modern Kubernetes versions use bound service account tokens with automatic rotation. Instead of a static secret, a projected volume provides a time-limited, audience-bound token:

yaml
1spec:
2  containers:
3    - name: app
4      volumeMounts:
5        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
6          name: kube-api-access
7  volumes:
8    - name: kube-api-access
9      projected:
10        sources:
11          - serviceAccountToken:
12              expirationSeconds: 3607
13              path: token

If you see token read errors with bound tokens, the token may have expired. The kubelet should rotate it automatically, but check kubelet logs if this persists.

Common Pitfalls

  • Treating the warning as an error: In most cases this is informational, not fatal. Applications that do not need the Kubernetes API should simply log and continue.
  • Hardcoding the token path: Always use the official client libraries rather than reading the token file directly. The libraries handle fallback and rotation.
  • Security implications of leaving tokens mounted: Unused service account tokens increase the blast radius of a container compromise. Disable mounting when the pod does not need API access.
  • RBAC issues masked by this error: If the token exists but RBAC denies access, you will see a different error (403 Forbidden). Do not confuse the two.

Summary

  • This warning means the service account token file cannot be read at the expected path
  • When running locally, it is expected — ensure your code falls back to kubeconfig
  • When running in Kubernetes, verify automountServiceAccountToken is not disabled
  • For pods that do not need API access, disable token mounting to improve security

Course illustration
Course illustration

All Rights Reserved.