Kubernetes
nginx
ingress controller
bad gateway
troubleshooting

Kubernetes nginx ingress controller bad gateway

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

A 502 Bad Gateway from the NGINX Ingress Controller means NGINX could not get a valid upstream response from the Kubernetes service behind the ingress. The fastest way to fix it is to check the route in layers: ingress, service, endpoints, then pod health and container port.

What 502 Usually Means Here

In this setup, NGINX is the reverse proxy and your application is the upstream backend. A 502 often points to one of these problems:

  • the service targets the wrong container port
  • the service has no ready endpoints
  • the backend container is crashing or not listening
  • the backend protocol or timeout settings do not match the ingress expectation

Different root causes can produce the same browser error page, so the diagnosis needs to be systematic.

Check the Ingress and Service Wiring

A minimal setup looks like this:

yaml
1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: app-ingress
5spec:
6  ingressClassName: nginx
7  rules:
8    - host: app.example.com
9      http:
10        paths:
11          - path: /
12            pathType: Prefix
13            backend:
14              service:
15                name: app-service
16                port:
17                  number: 80
18---
19apiVersion: v1
20kind: Service
21metadata:
22  name: app-service
23spec:
24  selector:
25    app: app
26  ports:
27    - port: 80
28      targetPort: 8080

If the pod is really listening on 5000 while the service forwards to 8080, the ingress can still resolve the service name correctly and yet every request fails as 502.

Inspect Endpoints Immediately

This is one of the most useful checks:

bash
kubectl get svc app-service
kubectl get endpoints app-service
kubectl get pods -l app=app -o wide

If kubectl get endpoints shows no addresses, NGINX has nowhere to send traffic. That usually means the selector is wrong, the pods are not ready, or the deployment itself is unhealthy.

Test the Backend Without Ingress

Before digging into annotations, prove whether the service can answer traffic directly:

bash
kubectl port-forward svc/app-service 8080:80
curl http://127.0.0.1:8080/

If this fails, the backend problem is below the ingress layer. If this works but the public route still returns 502, focus on ingress config and controller behavior.

Read the Ingress Controller Logs

The controller logs often identify the real upstream failure:

bash
kubectl logs -n ingress-nginx deploy/ingress-nginx-controller

Look for messages about connection refused, no live upstreams, upstream timeouts, or SSL and protocol mismatches. Those log lines are far more specific than the browser response.

Watch for Protocol and Timeout Mismatches

Sometimes the service is alive, but NGINX and the backend disagree about how to speak to each other. Examples include:

  • backend expects HTTPS while ingress forwards plain HTTP
  • gRPC or WebSocket traffic is missing the required configuration
  • backend takes longer than the proxy timeout allows

These cases can still surface as 502 even when pods and services look normal at first glance.

Common Pitfalls

The most common mistake is checking only the ingress manifest and ignoring the service endpoints. Empty or wrong endpoints cause many 502 errors.

Another issue is port mismatch between service and container. The ingress points to the right service, but the service points to the wrong target port.

Teams also skip direct backend testing. If a port-forwarded request fails, the ingress controller is not the root problem.

Summary

  • A 502 from NGINX Ingress usually means the upstream Kubernetes service or pod is not responding correctly.
  • Verify the ingress rule, service port, target port, and endpoints in order.
  • Use port-forwarding to separate ingress issues from backend issues.
  • Check the ingress controller logs for upstream-specific failure messages.
  • Empty endpoints, wrong ports, unhealthy pods, and protocol mismatches are the most common causes.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.