TCP Congestion Control: Why Packet Loss Is the Whole Conversation

TCP does not know how much bandwidth is available. It guesses, and then it adjusts based on feedback.

The guess starts small. The congestion window, cwnd, begins at a few packets. For every round trip where every packet is acknowledged, cwnd doubles. That is slow start, and despite the name it is the most aggressive phase TCP has. The point is to find the ceiling fast.

Once cwnd crosses the slow start threshold, TCP switches to congestion avoidance: linear growth, one extra packet per round trip. This is the AIMD pattern, additive increase, multiplicative decrease. Climb gently. On the first sign of loss, cut the window in half. Two refinements layer on top. Fast retransmit triggers a resend after three duplicate ACKs instead of waiting for a full timeout. Fast recovery skips slow start after that and re-enters congestion avoidance directly.

The whole loop hinges on one assumption: packet loss means the network is congested. In the world TCP was designed for, that was true. Wired LAN, short paths, loss meant a router queue overflowed.

That assumption breaks on modern long-haul links.

Here is a failure I have watched eat throughput more than once. A service in us-east-1 talks to a database replica in eu-west-1. The cross-region path runs over optical equipment that has roughly 0.5% intrinsic bit error rate, completely unrelated to load. Every flow on that path loses about one packet in two hundred no matter what. CUBIC, the default Linux congestion controller, sees those losses and concludes the link is congested. It cuts the window, ramps up, loses a packet, cuts again. The window never grows large enough to fill the pipe. Throughput sits at maybe 3% of available bandwidth. iperf shows you a healthy link. Your application shows you a slow one.

Switch the socket to BBR and the picture changes. BBR does not use loss as its signal. It models bandwidth and round trip time directly, sending at the estimated bottleneck rate. Intrinsic loss becomes background noise. The window stays open. Throughput jumps an order of magnitude on the same physical path, with no change to the application code and no change to the underlying optics.

The deeper lesson is that the slow start and AIMD curves in any networking textbook are not laws. They are policy choices that made sense for a 1988 network. CUBIC adjusts the shape of the climb, BBR replaces the signal entirely, and proprietary controllers inside hyperscalers go further still. The TCP state machine is the same. The opinion about what loss means is not.

When you debug a slow long-haul connection, do not just look at cwnd. Ask what signal your sender is actually listening to, and whether the link is honest about that signal.