A load balancer answers one question: given a thing arriving from a client, which backend should handle it? The whole design space follows from what that thing is, and that is the difference between layer 4 and layer 7.

Connections against requests

A layer 4 balancer decides per connection. It sees a TCP connection arrive, picks a backend, and from then on every byte in that connection goes to the same place. It does not know what a request is.

A layer 7 balancer decides per request. It terminates the connection, parses the protocol, and chooses a backend for each HTTP request, which means two requests on one client connection may go to different backends.

That single distinction determines everything else: what the balancer can route on, whether it can retry, how evenly it spreads load, how much it costs, and what it does to the client's view of the connection. The layer numbers themselves are a loose reference to the OSI model and are better read as "connection-level" and "request-level".

Where the balancer sits in the path

Most real systems have several balancing decisions stacked, and confusing them makes diagnosis much harder than it needs to be.

DNS makes a coarse decision by returning one or more addresses, often geographic. Anycast routing may then steer the packet to the nearest point of presence. An edge layer 4 device spreads connections across edge proxies. Those proxies are layer 7 and route requests to an internal service. Inside, a mesh sidecar or a client-side balancer picks a specific instance.

So a request may be balanced four or five times before reaching code, each at a different layer with different information. When traffic is uneven, the useful first question is which of those decisions is the one actually distributing the traffic you care about.

What layer 4 can see and what it cannot

A layer 4 balancer has the packet headers: source address and port, destination address and port, and protocol. That is the five-tuple, and it is all it has.

It therefore cannot route on a URL path, a hostname, a header, a cookie or a method, because all of those live inside the encrypted or at least unparsed payload. It cannot retry a failed request, because it does not know where one request ends and another begins. And it cannot distinguish a connection carrying one request from a connection carrying ten thousand.

What it can do is forward packets extremely cheaply, at line rate, with negligible added latency, and without needing the TLS private key. Those properties are why it remains the right answer for a large class of problems.

What layer 7 gains by terminating

A layer 7 balancer accepts the client's connection as a server, reads the request, and opens or reuses its own connection to a backend. It is a proxy, not a forwarder.

That gives it the request: method, path, host, headers, cookies, and body if it chooses to buffer. So it can route on any of them, which enables path-based routing to different services, host-based virtual hosting, header-based canary and A/B splits, and per-route policy.

It also gains the ability to act on failures. Because it knows request boundaries and whether a request is idempotent by method, it can retry a failed request on another backend transparently, which is the single most valuable layer 7 feature in practice. And it can add headers, rewrite paths, compress, cache, rate limit and enforce authentication, all of which require understanding the protocol.

The costs of terminating

Every one of those capabilities is paid for, and the costs are concrete rather than theoretical.

Latency rises, because the proxy must receive, parse and re-emit, and because there are now two connections and potentially two TLS handshakes rather than one forwarded flow. The added latency is typically under a millisecond for a well-tuned proxy and is not zero.

Throughput costs CPU, since parsing HTTP and terminating TLS are real work, so a layer 7 tier needs capacity planning of its own and becomes a scaling concern. The private key must live on the proxy, which has security implications. And the proxy becomes a stateful participant that can itself fail, buffer, or mishandle a protocol edge case.

Forwarding modes, and why the source address matters

Layer 4 balancers come in two modes that look similar and behave very differently.

In network address translation mode, the balancer rewrites the destination address, the backend sees the balancer as the client's address or the balancer's own, and the return traffic must flow back through the balancer. That return path is a capacity constraint, since response volume usually exceeds request volume.

In direct server return mode, the balancer rewrites only enough to deliver the packet, and the backend replies directly to the client. Return traffic bypasses the balancer entirely, which removes the bottleneck and is how very high throughput balancing is done. It requires the backends to share the service address and is more complex to operate.

Either way, the backend may no longer see the real client address, which breaks logging, rate limiting and geolocation unless something preserves it. The proxy protocol prepends the original addresses on a layer 4 connection, and X-Forwarded-For carries them at layer 7.

Choosing between them, concretely

The decision is usually straightforward once the requirements are named.

Choose layer 4 when the protocol is not HTTP, when the balancer must not hold the TLS key, when throughput is very high and per-connection cost must be near zero, or when end-to-end encryption to the backend is required. Database traffic, raw TCP services and pass-through TLS all sit here.

Choose layer 7 when routing depends on request content, when retries, rate limiting, authentication or header manipulation belong at the edge, when traffic should be split for a canary, or when the backends are HTTP services with uneven request costs.

The common production answer is both: layer 4 at the front for cheap, resilient connection spreading, and layer 7 behind it for everything that needs to understand requests.

The pinning problem, which drives most surprises

One consequence of the layer 4 model deserves naming now, because it explains a large fraction of real load balancing incidents.

Because a layer 4 decision is made once per connection, long-lived connections pin clients to backends. With HTTP/1.1 and short connections this is harmless, since connections turn over constantly. With HTTP/2, gRPC, WebSockets or a well-tuned connection pool, connections last for minutes or hours, so the balancer distributes connections once and then nothing moves.

The visible symptom is one backend hot while others idle, and scaling out having no effect at all, because existing connections never migrate. Recognising that shape instantly, and knowing it is a layer mismatch rather than a capacity problem, is what separates a five-minute diagnosis from an afternoon.

Who makes the decision, and where it runs

A further axis is worth separating from the layer question, because the two are often conflated.

The decision can be made by a dedicated device in the path, which is the classic load balancer; by the client itself, which resolves all backend addresses and chooses among them; or by a sidecar proxy running beside each workload. All three can be layer 4 or layer 7, and all three face the same algorithm and health checking questions.

What changes is operational: a device in the path is one thing to configure and one thing to saturate, a client-side implementation needs logic in every language the estate uses, and a sidecar puts a proxy's worth of CPU and memory next to every workload in exchange for uniform policy. None of this is implied by the layer, so it helps to decide the two separately rather than inheriting a topology from a layer choice.

A worked example of the difference

Consider one browser holding an HTTP/2 connection and issuing forty requests for a page.

Behind a layer 4 balancer, those forty requests land on exactly one backend, because the connection was assigned once. If one of them fails with a connection error, the browser sees the failure. If that backend is the slowest in the fleet, all forty requests are slow.

Behind a layer 7 balancer, the forty requests are distributed across the pool, each according to the current algorithm, so a slow backend affects a fraction of them. A failed request is retried on another backend and the browser never learns it happened. The same traffic, the same backends, and a materially different user experience, purely from where the decision is made.