How health check of Kubernetes work with Istio?
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
Kubernetes health checks still come from the kubelet even when Istio injects a sidecar into the pod. The complication is that Istio changes traffic flow inside the pod, so HTTP, TCP, and gRPC probes need special handling if you want liveness and readiness checks to keep reflecting application health accurately.
Start with Kubernetes Probes
Kubernetes supports liveness, readiness, and startup probes. A normal HTTP readiness probe might look like this:
Without a service mesh, the kubelet sends that request directly to the container through the pod IP and target port.
Why Istio Changes Probe Behavior
Istio sidecars intercept pod traffic with iptables rules. According to the Istio docs, that creates two common problems:
- with strict mTLS, the kubelet does not have an Istio certificate, so direct HTTP probes can fail
- TCP probes can become misleading because the sidecar makes the port appear open even when the application behind it is not ready
To solve this, Istio can rewrite HTTP, TCP, and gRPC application probes so the kubelet talks to the sidecar agent instead. The sidecar then forwards the health check to the original application endpoint.
Understand Probe Rewrite
When sidecar injection is enabled, Istio rewrites supported probes at pod-injection time. That is why the running pod may contain probe paths such as /app-health/... even if your deployment YAML originally used /ready or /healthz.
The kubelet is still the thing performing the probe. Istio is changing the route the probe takes so it still works inside the mesh.
This is also why debugging health failures in an injected pod should include inspecting the live pod spec, not just the original deployment manifest.
Disable Rewrite Only When You Mean To
Istio documents an annotation for disabling HTTP probe rewrite on a pod:
Use that only if you are sure the direct kubelet probe can still succeed with your mesh settings. In strict mTLS setups, disabling rewrite often causes healthy workloads to fail probes.
Choose Probe Types Carefully
Command probes are often the least affected by Istio because they run inside the container. HTTP probes are usually a good fit when rewrite is enabled. TCP probes deserve more caution because "the port accepted a connection" is not a very strong health signal once a proxy sits in front of the app.
If you want application-level confidence, HTTP or command probes are usually better than plain TCP probes in a sidecar-based environment.
Common Pitfalls
The biggest mistake is assuming Istio ignores Kubernetes health checks. It does not. The kubelet still performs them, but the mesh changes how they must be routed.
Another common issue is turning off probe rewrite without understanding why it was enabled. That can break health checks immediately in mTLS-protected environments.
People also trust TCP probes too much when a sidecar is present. A listening proxy is not the same as a healthy application.
Finally, when debugging readiness failures, inspect the injected pod spec and sidecar behavior instead of only reading the original YAML definition.
Summary
- Kubernetes health checks still come from the kubelet when Istio is present.
- Istio rewrites supported probes so they continue to work with traffic interception and mTLS.
- HTTP, TCP, and gRPC probes can be rewritten; command probes run inside the container.
- The annotation
sidecar.istio.io/rewriteAppHTTPProbers: "false"disables rewrite, but use it carefully. - In Istio environments, HTTP or command probes are usually more trustworthy than bare TCP probes.
Related reading
- How I create new namespace in Kubernetes
- How if I interact with different kubernetes clusters in different terminals sessions with out having to switch contexts all the the time?
- How is Docker Swarm different than Kubernetes?
- How is Python scaling with Gunicorn and Kubernetes?
- How is rancher different from Kubernetes
- How Kubernetes computes CPU utilization for HPA?
- How kubernetes HPA with 2 or more metrics behaves - especially the no.of replicas calculation?
- How much RAM can my Kubernetes pod grow to?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.