Nginx
Ingress Controller
Kubernetes
EndpointSlice
Troubleshooting

Nginx Ingress Controller - Failed to watch v1.EndpointSlice

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

The log line Failed to watch *v1.EndpointSlice usually means the ingress controller cannot list or watch EndpointSlice objects from the Kubernetes API. In practice, the first suspects are missing RBAC permissions, a mismatched controller manifest, or a controller version that does not line up cleanly with the cluster version.

Why EndpointSlices Matter

Ingress controllers need backend endpoint data so they can route traffic to the right service pods. In current Kubernetes clusters, that information is carried through EndpointSlice resources in the discovery.k8s.io API group.

If the controller cannot watch those resources, it may still start and keep running, but it will not reliably learn backend changes. That means routing problems appear even though the controller pod itself looks healthy.

Read the Full Error Message

The short headline rarely tells the whole story. Pull the controller logs and inspect the lines around the failure:

bash
kubectl logs -n ingress-nginx deploy/ingress-nginx-controller

The follow-up text often reveals the actual class of problem:

  • 'forbidden usually means RBAC is missing'
  • 'the server could not find the requested resource suggests API compatibility or manifest drift'
  • repeated reflector warnings usually mean the controller keeps retrying and failing the same watch

Do not stop at the summary line. The extra text usually saves time.

Check RBAC First

The controller service account needs permission to get, list, and watch endpointslices in discovery.k8s.io.

First, confirm the service account the controller is using:

bash
kubectl get deploy -n ingress-nginx ingress-nginx-controller \
  -o jsonpath='{.spec.template.spec.serviceAccountName}'

Then test permissions directly:

bash
1kubectl auth can-i get endpointslices.discovery.k8s.io \
2  --as=system:serviceaccount:ingress-nginx:ingress-nginx
3
4kubectl auth can-i list endpointslices.discovery.k8s.io \
5  --as=system:serviceaccount:ingress-nginx:ingress-nginx
6
7kubectl auth can-i watch endpointslices.discovery.k8s.io \
8  --as=system:serviceaccount:ingress-nginx:ingress-nginx

If any of those answers is no, fix RBAC before chasing anything else.

A matching cluster role rule looks like this:

yaml
1apiVersion: rbac.authorization.k8s.io/v1
2kind: ClusterRole
3metadata:
4  name: ingress-nginx
5rules:
6  - apiGroups:
7      - discovery.k8s.io
8    resources:
9      - endpointslices
10    verbs:
11      - get
12      - list
13      - watch

That cluster role must also be bound to the controller service account through a ClusterRoleBinding.

Look for Chart or Manifest Drift

This failure often appears after one of these situations:

  • an old Helm chart was reused,
  • manifests were copied and edited by hand,
  • the cluster was upgraded but the ingress manifests were not,
  • or a security hardening pass accidentally removed one permission block.

Compare the live RBAC and deployment objects against the official manifests for the ingress-nginx version you actually run. Fixing the live ClusterRole by hand may restore service, but if the source manifests stay wrong, the next redeploy will reintroduce the problem.

Useful inspection commands are:

bash
kubectl get clusterrole ingress-nginx -o yaml
kubectl get clusterrolebinding ingress-nginx -o yaml
kubectl get deploy -n ingress-nginx ingress-nginx-controller -o yaml

These let you verify the permissions, the binding target, and the service account used by the deployment.

Compatibility Still Matters

If the problem is not RBAC, check version compatibility. EndpointSlices are standard in current Kubernetes, but a very old controller image, copied legacy manifests, or a heavily customized deployment can still carry outdated assumptions.

When the API server says the resource was not found, that is a hint to check the controller release, cluster version, and manifest source together instead of treating them as independent.

Confirm the Backend Objects Exist

After fixing the watch issue, verify that the relevant services actually have slices:

bash
kubectl get endpointslice -A
kubectl get endpointslice -n your-app-namespace \
  -l kubernetes.io/service-name=your-service

If the controller can watch EndpointSlices but the target service has none, the real problem may be elsewhere:

  • bad service selectors,
  • pods not ready,
  • or namespace mismatches.

That is a different failure mode, but it can look similar from the outside because traffic still does not reach the app.

Common Pitfalls

The biggest pitfall is treating this like an NGINX config issue. The failure is happening earlier in the Kubernetes watch layer, before backend state even reaches the proxy configuration logic.

Another mistake is patching the live cluster role and forgetting to fix Helm values or source manifests. That produces a fragile cluster that breaks again on the next upgrade or redeploy.

Teams also sometimes bind the correct role to the wrong service account or the right service account name in the wrong namespace. The names have to match the deployment exactly.

Finally, do not assume a running controller pod is a healthy controller. Kubernetes watches can be failing silently while the pod still looks ready enough to stay up.

Summary

  • 'Failed to watch *v1.EndpointSlice is usually an RBAC or manifest compatibility problem.'
  • Check the full log message first so you know whether the error is forbidden or API-related.
  • Verify get, list, and watch permissions on endpointslices.discovery.k8s.io for the controller service account.
  • Compare live RBAC and deployment objects with the official ingress-nginx manifests for your release.
  • After fixing the watch, confirm the relevant services actually have EndpointSlices and ready backends.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.