Ingress
Kubernetes
Prometheus
Routing
Server Access

Ingress routing rules to access prometheus server

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Prometheus usually runs as an internal service inside a Kubernetes cluster, but teams often want browser access for dashboards, queries, and troubleshooting. The safe way to expose it is to route HTTP traffic through an Ingress resource instead of opening the pod directly.

Exposing Prometheus Through Ingress

An Ingress rule maps an external host or path to a Kubernetes Service. For Prometheus, that service usually points at port 9090. The Ingress controller, such as NGINX, Traefik, or HAProxy, watches the Ingress resource and configures the actual reverse proxy.

The basic flow is:

  1. Prometheus listens inside the cluster.
  2. A Service gives it a stable virtual address.
  3. An Ingress maps an external URL to that service.
  4. DNS points the hostname to the Ingress controller.

A minimal service and ingress pair looks like this:

yaml
1apiVersion: v1
2kind: Service
3metadata:
4  name: prometheus
5  namespace: monitoring
6spec:
7  selector:
8    app: prometheus
9  ports:
10    - port: 9090
11      targetPort: 9090
12---
13apiVersion: networking.k8s.io/v1
14kind: Ingress
15metadata:
16  name: prometheus
17  namespace: monitoring
18spec:
19  ingressClassName: nginx
20  rules:
21    - host: prometheus.example.com
22      http:
23        paths:
24          - path: /
25            pathType: Prefix
26            backend:
27              service:
28                name: prometheus
29                port:
30                  number: 9090

With that configuration, traffic sent to prometheus.example.com is forwarded to the prometheus service.

Host-Based And Path-Based Routing

Host-based routing is usually the cleanest option because Prometheus expects to serve content from a consistent root path. A dedicated host such as prometheus.example.com is easier than sharing a generic monitoring host with path rewrites.

If you must expose Prometheus under a path such as /prometheus, you need two aligned settings:

  • the Ingress must route /prometheus
  • Prometheus must know its external path prefix

Example:

yaml
1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: prometheus
5  namespace: monitoring
6  annotations:
7    nginx.ingress.kubernetes.io/rewrite-target: /$2
8spec:
9  ingressClassName: nginx
10  rules:
11    - host: monitoring.example.com
12      http:
13        paths:
14          - path: /prometheus(/|$)(.*)
15            pathType: ImplementationSpecific
16            backend:
17              service:
18                name: prometheus
19                port:
20                  number: 9090

Prometheus also needs matching startup flags:

yaml
args:
  - --web.external-url=http://monitoring.example.com/prometheus/
  - --web.route-prefix=/

Without these flags, static assets and redirects often break because Prometheus still thinks it lives at the root URL.

Adding TLS And Basic Protection

Prometheus is operationally sensitive. If you expose it publicly, use TLS and some form of authentication or network restriction.

yaml
1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: prometheus
5  namespace: monitoring
6spec:
7  ingressClassName: nginx
8  tls:
9    - hosts:
10        - prometheus.example.com
11      secretName: prometheus-tls
12  rules:
13    - host: prometheus.example.com
14      http:
15        paths:
16          - path: /
17            pathType: Prefix
18            backend:
19              service:
20                name: prometheus
21                port:
22                  number: 9090

If your controller supports authentication annotations, you can add them there. Another common pattern is to keep Prometheus internal and expose it only through a VPN, private load balancer, or identity-aware proxy.

How To Verify The Route

After applying the resources, verify each layer separately instead of guessing.

Check the service:

bash
kubectl -n monitoring get svc prometheus
kubectl -n monitoring describe svc prometheus

Check the ingress:

bash
kubectl -n monitoring get ingress prometheus
kubectl -n monitoring describe ingress prometheus

If DNS is not ready yet, you can still test by forcing the Host header:

bash
curl -H 'Host: prometheus.example.com' http://INGRESS_IP/-I

That tells you whether the controller is routing correctly before public DNS is updated.

Common Pitfalls

The most common mistake is routing to the wrong service port. Prometheus often listens on 9090, but some charts expose a differently named service or an alternate port mapping. Always inspect the actual service definition.

Another issue is path prefix mismatch. If you publish Prometheus below /prometheus but do not set --web.external-url, parts of the UI redirect to the wrong path or return 404.

Controller-specific annotations are also easy to misuse. A rewrite rule that works in NGINX may not work in Traefik. Keep your Ingress manifest aligned with the controller you actually installed.

Finally, avoid exposing Prometheus anonymously on the public internet. Metrics often reveal hostnames, workload names, and internal topology. Route protection is part of the deployment, not an optional afterthought.

Summary

  • Use a Kubernetes Service as the stable backend for Prometheus and route to it through an Ingress.
  • Prefer host-based routing such as prometheus.example.com because it is simpler than path rewriting.
  • If you use a path prefix, configure both the Ingress rule and Prometheus external URL settings.
  • Add TLS and access controls before exposing monitoring endpoints outside the cluster.
  • Validate service, ingress, and DNS separately so routing failures are easier to isolate.

Course illustration
Course illustration

All Rights Reserved.