Connection management is the least glamorous part of networking and the one that produces the largest latency improvements in real systems. The reason is that a new connection costs far more than most code assumes.

Counting the round trips

Before a single request byte moves, a new HTTPS connection has already spent several round trips.

A TCP handshake costs one round trip, since the client can send data with the third packet. A TLS 1.3 handshake costs one more, or two for TLS 1.2. So an HTTPS request on a fresh connection pays two round trips before the request is sent, plus one to receive the response.

On a warm connection the same request costs one round trip. So within a data centre at 0.5 ms the difference is 1.5 ms against 0.5 ms, which is a tripling but small in absolute terms. Across a region at 30 ms it is 90 ms against 30 ms, and across continents at 150 ms it is 450 ms against 150 ms. The multiplier is constant and the absolute cost scales with distance, which is why connection reuse matters most exactly where latency is already a problem.

The cost that is not latency

Round trips are the visible cost. Three others are less visible and often larger in aggregate.

TLS handshakes consume CPU, with the asymmetric cryptography in the handshake costing orders of magnitude more than the symmetric encryption of the data that follows. A server doing thousands of fresh handshakes per second can spend a significant fraction of its CPU on them, which is why session resumption exists.

Each connection consumes a file descriptor and kernel socket buffers on both ends, typically tens of kilobytes of memory per connection once send and receive buffers are counted. And each connection occupies an ephemeral port on the client, of which there are roughly 28,000 by default on Linux, which becomes a hard ceiling under churn.

Slow start, and why a new connection is also narrow

A new TCP connection does not only start late, it starts slow. Congestion control begins with a small window, commonly ten segments, and grows it over successive round trips.

So the first response is limited to roughly 14 kB before the sender must wait for acknowledgements, regardless of available bandwidth. A 100 kB response on a fresh connection takes several round trips to deliver even on a fast link.

A warm connection has already grown its window, so the same response may move in one burst. This is the second half of the connection reuse argument and the half that is usually forgotten: reuse saves handshake latency and it preserves the congestion window that was paid for.

The antipattern, and how common it is

The single most frequent connection bug is constructing a new HTTP client object per request.

Most client libraries keep their connection pool inside the client object, so a client created per request has an empty pool, opens a fresh connection, and discards it along with the client. Every request pays the full cold cost, and the pool configuration that looks correct in code never takes effect.

It is easy to write by accident, especially in a function that takes no client parameter, in a framework where the client is constructed in a handler, or in code ported from a language where the client is cheap. And it is easy to confirm: a connection count on the server, or a count of TLS handshakes, will be roughly equal to the request count instead of far below it.

Where reuse breaks even when the client is shared

A shared client is necessary and not sufficient, because several things silently prevent reuse.

A response body that is never fully read or closed leaves the connection unable to return to the pool, so it is discarded. This is the second most common cause after the per-request client and is particularly easy in error paths, where code checks the status, returns an error and never drains the body.

A Connection: close header from the server ends the connection after one response, which some servers send under load. A proxy or load balancer with a short idle timeout closes pooled connections from the other side. And a pool whose maximum size per host is smaller than the concurrency will force new connections for the excess, which looks like a reuse failure and is a sizing failure.

Session resumption, the partial remedy

When a new connection is unavoidable, TLS session resumption reduces its cost.

With a session ticket or a resumed session identifier, the handshake is shortened and the asymmetric cryptography is skipped, which removes most of the CPU cost and, with TLS 1.3 early data, can remove a round trip too.

It requires the server side to share session state across instances, through a shared ticket key or a session cache, which is often not configured in a load-balanced deployment. When it is not, every connection to a new backend is a full handshake, and resumption appears not to work. Verifying that resumption actually succeeds, rather than assuming it, is worth doing in any deployment where connection churn is unavoidable.

What this means for protocol choice

The cost structure explains why HTTP/2 and HTTP/3 are valuable beyond their framing.

HTTP/1.1 requires one connection per concurrent request, so a client making six parallel requests opens six connections and pays six handshakes. HTTP/2 multiplexes them onto one, so one handshake serves all of them and one congestion window is shared and grown.

That single difference is usually a larger win than the header compression or the binary framing, and it is the reason enabling HTTP/2 on an existing JSON API captures a large share of what a gRPC migration would deliver.

Measuring it before changing anything

Three measurements establish whether connection cost is a real problem in a given system, and they take minutes.

The reuse ratio, requests divided by connections accepted, measured at the server. A healthy internal service sees hundreds or thousands of requests per connection; a value near one means a per-request client somewhere.

The TLS full handshake rate against the resumption rate, which shows whether resumption is working. And the ephemeral port usage or TIME_WAIT count on busy clients, which shows whether churn is approaching a hard limit.

Doing this first avoids the common sequence where a team tunes pool sizes for a week while the actual bug is a client constructed inside a request handler.

Connection cost in the browser, briefly

The same arithmetic applies on the public internet with larger numbers, which is worth knowing even for backend work, because it decides what the edge must do.

A first page load from a client 150 ms away pays DNS resolution, a TCP handshake and a TLS handshake before the first byte of HTML, which is several hundred milliseconds of nothing. Every additional origin the page references repeats it.

This is why reducing the number of origins, serving assets from the same origin as the document, and terminating TLS close to the user all matter more than payload optimisation for a cold load. It is also why HTTP/3's zero round trip resumption is valuable at the edge and nearly irrelevant inside a data centre, where the handshake costs microseconds rather than hundreds of milliseconds.

The two questions to ask of any client

Given an unfamiliar codebase, two questions establish whether connection handling is sound.

Where is the client object created, and how many exist? A client per request is the cold-cost bug; a client per instance is correct; a client per thread is usually a sizing problem in disguise, since the pool limit now multiplies by the thread count.

And what closes the response? Following the success path and at least two error paths to the point where the body is drained or closed is the only way to confirm reuse actually happens, because the code that leaks a body looks entirely reasonable: it checks a status, logs, and returns an error.

Both answers are a few minutes of reading and they predict the system's connection behaviour better than any configuration file does.