API Gateway: One Chokepoint, One Policy Plane, One Set of Tradeoffs
December 12, 2023
When people first hear "API Gateway" they picture a smart router. That undersells it. Routing is the smallest job the gateway does. Its real job is to be the one place in your system where every external request meets the same policies. The value is the funnel, not the features.
Walk through what a request actually picks up on the way through. First, authentication. The gateway verifies the JWT, looks up the API key, or terminates mTLS. Backend services then trust a clean internal header instead of reimplementing auth in twelve different languages. Second, rate limiting and quotas. A token bucket keyed on user, tenant, or IP lives at the edge and protects every downstream service at once. Third, request and response transformation. The mobile client wants a flat JSON shape. The backend speaks gRPC. The gateway translates. Fourth, response caching for safe verbs. A two-second cache on a hot read can drop backend QPS by an order of magnitude. Fifth, circuit breaking and retries. If payments starts returning 503s, the gateway can shed load and stop hammering it. Sixth, observability. One place to trace, one place to count, one place to sample. Logs, metrics, and tracing IDs get stamped on every request before fan-out.
Some gateways add request coalescing, where N identical in-flight requests for the same key share a single backend call. That is the difference between a thundering herd and a quiet origin.
The tradeoff is the thing most posts skip. Every request through one place is a single policy plane and a single chokepoint. I have seen a misconfigured rate-limit rule at the gateway take down every product in the company simultaneously, because every product shared the same gateway tier. A bad deploy of a Lua filter in an Envoy fleet, or a broken Kong plugin, or an AWS API Gateway throttling regression, hits 100% of your traffic the moment it rolls out. Treat the gateway like a database. Canary it. Version its config. Never let one team push directly to it.
When do you not need one? Internal service-to-service traffic does not belong here. East-west calls inside a mesh handle their own auth via mTLS and SPIFFE identities, their own retries via the sidecar, and their own observability via the mesh control plane. Putting an API Gateway between two internal services adds a hop, a failure mode, and a deploy coupling for no policy gain.
Use the gateway for the edge. Let the mesh own the interior.
The API Gateway's value is not what it does. It is that every request goes through one place. That makes policy uniform and observability cheap. It also makes the gateway your single biggest blast radius.
Originally posted on LinkedIn. View original.