Exchanges and the Routing Algorithms

Topics Covered

Direct and Fanout

Direct, which is an equality test

The default exchange, which is a direct exchange

Fanout, which ignores the routing key entirely

Arithmetic for a fanout, with real numbers

Choosing between them is a question about the future

Where the copies come from, and what is shared

What the management UI will not tell you

The four built-in exchange types are four different matching algorithms, and the differences between them are differences in data structure and cost rather than in style. Starting with the two simple ones makes the expensive ones easier to judge.

Direct, which is an equality test

The simplest algorithm, and the one most topologies should use.

A direct exchange delivers a message to every queue whose binding key is exactly equal to the message's routing key. No wildcards, no pattern, no interpretation: a byte comparison on the whole string, case-sensitive.

The implementation is a hash lookup. RabbitMQ keeps bindings in Mnesia, and for a direct exchange the lookup is keyed on the exchange name plus the routing key, so the cost of routing is independent of how many bindings the exchange has. An exchange with ten bindings and one with ten thousand route at the same speed, which is not true of the other three types.

Equality still allows fan-out, because bindings are many to many. Three queues bound with the key invoice.created all receive a message published with that key, and the exchange does not care that the key is not unique. So the common belief that a direct exchange means one queue per key is wrong, and a direct exchange with duplicated binding keys is the cheapest way to get a selective broadcast.

The default exchange, which is a direct exchange

The one nobody declares and everybody uses.

Every vhost contains an exchange named with the empty string, of type direct, to which every queue is automatically bound using the queue's own name as the binding key. Publishing with an empty exchange name and a routing key of payments.worker therefore reaches a queue named payments.worker and nothing else.

The implicit binding cannot be removed and does not appear in the output of rabbitmqctl list_bindings for the exchange in the same way an explicit one does, which confuses people trying to audit a topology. It also means the default exchange has a binding for every queue in the vhost, so a vhost with 50,000 queues has a 50,000-entry table on an exchange nobody declared.

The reason to avoid it in application code is not performance but coupling: publishing to the default exchange names a queue, which is the one thing the object model was designed to prevent, and a publisher that names a queue cannot have its consumers rearranged without a publisher change.

A direct exchange is an equality test implemented as a hash lookup, so its routing cost is independent of binding count. Several queues may share one binding key, which makes selective broadcast cheap.

Fanout, which ignores the routing key entirely

The cheapest routing and the most expensive delivery.

A fanout exchange copies every message to every bound queue and does not look at the routing key at all. A publisher can set any key it likes and the result is identical, which is worth knowing because a team debugging a fanout exchange often spends time on a routing key that was never consulted.

Routing is therefore free, and delivery is not. The cost of a fanout publish is the number of bound queues, because each copy is an enqueue into a separate Erlang process with its own mailbox, its own index and, for a persistent message, its own write. A fanout exchange with 200 queues turns 1,000 publishes per second into 200,000 enqueues per second, which saturates a broker that a glance at the publish rate would call idle.

The copy is a reference rather than a duplicate of the body for a transient message, which saves memory, and a persistent message's body is written once to the shared message store with a reference per queue, which saves disk. What is not shared is the per-queue work: index entries, acknowledgement tracking and delivery.

One publish into a fanout exchange with 200 bound queues is 200 enqueues into 200 separate processes. The publish rate and the enqueue rate differ by the binding count, and only the first appears on the exchange.

Arithmetic for a fanout, with real numbers

Four multiplications, and they decide whether a cluster is adequate.

Take a fanout exchange with 200 bound queues, a publish rate of 1,000 per second, an average message of 2 KB, and persistent delivery into durable queues.

The enqueue rate is 1,000 times 200, so 200,000 per second. That is the figure that has to fit inside what the broker's schedulers can do, and since each enqueue lands in a different Erlang process, the work parallelises across cores reasonably well, which is the one piece of good news.

The network egress is 200 copies delivered to consumers, so 1,000 times 200 times 2 KB, which is 400 MB/s, or 3.2 Gb/s. That exceeds a 1 Gb interface by more than threefold, and it is the limit people meet first and attribute to the broker being slow.

The disk writes are one body write per publish plus one index entry per queue, so 2 MB/s of bodies and 200,000 index entries per second. Index entries are small and numerous, which is the write pattern a spinning disk handles worst and a network-attached volume handles with surprising latency.

Running those four numbers before building the topology takes five minutes and is the difference between choosing 200 queues deliberately and discovering the consequence in production.

Choosing between them is a question about the future

And the answer is usually direct.

The argument for fanout is that adding a consumer requires no routing decision, since binding a new queue is enough. The argument against is that every bound queue receives everything, so a consumer interested in 1% of the traffic pays for 100% of it, filtering in application code and discarding the rest.

That discarded 99% is not free. It is broker enqueue work, network bandwidth, consumer deserialization and consumer acknowledgement, and it is the dominant cost in most fanout topologies that grew past their original design.

A direct exchange with several binding keys per queue gives the same flexibility with none of that waste, because the filter runs once in the broker rather than once per consumer. The migration from fanout to direct is also easy in one direction and hard in the other, since adding keys to a direct topology is additive while removing traffic from a fanout topology means changing every publisher.

Where the copies come from, and what is shared

One body, many references, and the distinction is load-bearing.

When an exchange routes a message to several queues, RabbitMQ does not duplicate the body once per queue. For a transient message the body is a term in memory and each queue holds a reference to it, and for a persistent message the body is written once into the shared message store with a reference counted per queue. The body is freed when the last reference goes away.

So the memory and disk cost of a fanout is roughly the body once plus per-queue bookkeeping, not the body times the binding count, which is better than the intuition suggests. A 2 KB message into 200 queues is not 400 KB.

What is emphatically not shared is everything else. Each queue has its own index entry for the message, its own position in its own ordering, its own unacknowledged-delivery tracking once it is delivered, and its own delivery to a consumer over its own socket. Those are the costs that multiply, and they are per-message rather than per-byte, which is why a fanout of small messages hurts more than the byte figures suggest.

The practical reading is that fanout amplification is a message-count problem, not a data-volume problem. A topology that fans small messages widely is in the worst quadrant, and one that fans large messages narrowly is usually fine, which is the opposite of how capacity conversations normally start.

What the management UI will not tell you

Two numbers that have to be computed.

The management UI shows a message rate per exchange and per queue, and the exchange rate is the publish rate while the queue rates are the enqueue rates. For a fanout exchange those differ by the binding count, so an exchange showing 1,000 per second and a cluster doing 200,000 enqueues per second is not a contradiction, and the second figure is the one that predicts an outage.

The number the UI will not compute is the amplification factor itself. rabbitmqctl list_bindings gives the binding count per exchange, and multiplying it by the publish rate gives the enqueue rate the topology demands. Doing that arithmetic for every fanout exchange in an estate takes a few minutes and routinely finds one exchange responsible for most of the broker's work.

The related figure is the persistent write amplification: a persistent message into 200 queues is one body write and 200 index writes, so the disk cost scales with binding count even though the body does not. On a broker with a modest disk that is the limit that binds first.

Key Insight

Routing cost and delivery cost are different things and only one of them is visible in the exchange's message rate. A direct exchange routes by hash lookup at a cost independent of binding count, and a fanout exchange does no routing work at all, but a fanout publish into 200 bound queues is 200 enqueues into 200 separate Erlang processes. Multiply each fanout exchange's binding count by its publish rate to get the enqueue rate the topology actually demands, because that figure, not the publish rate, is what saturates a broker.