0%
RabbitMQ Deep Dive: AMQP, Queues, and Delivery Guarantees
The Broker Model
Queues and Storage
Delivery Guarantees
Flow and Scale
Delayed Messages and Scheduling
A queue is a structure whose cheapest operation is taking the oldest element, and a delay requirement asks it to take an element that is not the oldest.
What a queue is optimised for
One pointer at the head, and the cost model that follows.
A classic queue is a FIFO backed by an index whose head is the next message to deliver. Enqueue appends, dequeue reads the head, and both are constant-time with no search. The queue process holds the head position and never scans, which is what lets a single Erlang process push tens of thousands of messages per second.
A delay requirement breaks that in a specific way: the message that should be delivered next is the one whose deadline is earliest, and insertion order and deadline order are unrelated. Serving deadlines correctly needs a priority structure keyed on time, which means an ordered insert rather than an append and a comparison on every enqueue.
So a broker that supports delay is maintaining a second data structure with different complexity from the queue, and every implementation of delay in RabbitMQ is either that structure, built by the plugin, or a construction that avoids needing it by exploiting something the queue already does.
Message TTL, and what it actually evaluates
One setting, two forms, and a behaviour that surprises everyone once.
A TTL marks a message as expired after a period. It can be set per queue with the x-message-ttl argument, applying to every message in that queue, or per message with the expiration property, which is a string of milliseconds. An expired message is either discarded or, if the queue has a dead-letter exchange configured, dead-lettered, which is the hook every delay construction depends on.
The behaviour that surprises people is where expiry is evaluated. In a classic queue, a message's TTL is checked when it reaches the head of the queue. A message with a one-second TTL sitting behind a message with a one-hour TTL is not examined until the message in front of it is dealt with, so the expiry of a per-message TTL is bounded below by the deadline of everything ahead of it in the queue.
Per-queue TTL does not have that problem, because every message in the queue has the same TTL, so head order and deadline order coincide and the head is always the earliest deadline. That is the whole reason the delay constructions in the next section use a queue per interval rather than a per-message value, and it is a conclusion people reach by hitting the head-of-line case in production rather than by reading it.
Quorum queues, which evaluate differently
A divergence worth knowing before a migration.
A quorum queue evaluates per-queue TTL rather than per-message TTL in the same head-driven way, and historically did not support the per-message expiration property at all, with support arriving later and with caveats. A construction built on per-message TTL against a classic queue therefore does not necessarily survive being moved to a quorum queue, and the failure is that messages are not delayed rather than that an error is raised.
The deeper difference is that a quorum queue's expiry is a Raft operation: removing an expired message is a committed command rather than a local decision, so expiring a large batch of messages at the same instant is a write burst to the log. A delay construction that releases ten thousand messages at one deadline produces ten thousand dead-letter publishes plus the corresponding log commands, which is a load spike the design has to account for.
Expiry is cheap in a classic queue and is a replicated write in a quorum queue, which inverts the usual advice: the queue type that is better for durability is worse for a design that uses expiry as a scheduling mechanism, and that is a genuine tension rather than something to tune away.
What the TTL clock measures from
Not publish time, and the distinction is what makes a ladder possible.
A message's TTL runs from the moment it enters the queue that holds it, not from when it was published. That sounds like a technicality until a message is dead-lettered: on arriving in the next queue it is enqueued afresh, so the new queue's TTL applies in full with nothing carried over from the time it already spent elsewhere.
That is the property every multi-step delay construction depends on. A message that waits ten seconds in one holding queue and is dead-lettered into a one-minute holding queue waits seventy seconds in total, because each queue's clock starts on arrival. A chain of holding queues sums its delays, which is how a delay longer than any single rung is built and why an accidental chain produces a delay nobody intended.
The related argument people confuse with TTL is x-expires, which is not a message TTL at all: it deletes the queue itself after a period with no consumers and no activity. On a holding queue, which by design has no consumer, that argument is actively dangerous, because an idle delay line is exactly the thing x-expires was built to clean up and deleting the queue deletes its pending messages. A delay line must never carry x-expires, and a broad policy that sets it is one of the ways a working construction stops working.
The three mechanisms, and what each one is
A map of the lesson, since the rest of it is these three in detail.
The first is TTL plus dead-lettering: a holding queue per interval, which uses only queue primitives and therefore inherits everything a queue has, including replication if the holding queues are quorum queues, a depth metric, and a length limit. It cannot express a delay specific to a message.
The second is the delayed-message exchange plugin, which keeps pending messages in a node-local table with a timer each and gives arbitrary per-message delays. It inherits none of the queue's properties because what it holds is not in a queue, which is the trade in one sentence.
The third is not in the broker at all: scheduled items in a database with a worker that publishes rows as they come due. It is more code than either broker mechanism and it is the only one of the three that supports cancellation, enumeration, replay and an unbounded horizon. The deciding question is usually not which mechanism is nicest but which properties the pending set has to have, and the pending set is the thing a design most often fails to think about at all.
Why delay is a plugin rather than a feature
A design decision, and the cost budget behind it.
Nothing in AMQP expresses a time at which a message should become available. The protocol has publish, consume, acknowledge and reject, and a message property bag that carries expiration but nothing that carries a release time, so a broker that honours the protocol has no place to put the concept even if it wanted to.
The implementation reason is the broker's per-message cost budget. Every operation on the hot path is constant-time and the queue process is deliberately simple, because throughput comes from doing almost nothing per message. A deadline-ordered structure adds a comparison per enqueue and a data structure whose size is the pending set rather than the ready set, and a pending set is unbounded in a way a ready set is not, because nothing drains it but time.
So delay sits outside the core, as a plugin with its own store and its own properties, and that placement is the honest signal: it is not a queue feature that happens to be optional, it is a different mechanism wearing an exchange's interface. Reading it that way makes the rest of the lesson predictable, because everything the plugin lacks is something a queue provides and the plugin is not a queue.
The requirements delay is usually standing in for
Three distinct needs, which want three different mechanisms.
The first is a retry backoff, where a failed message should be reattempted in thirty seconds and then in five minutes. The delays are few, known in advance and identical for every message at a given attempt, which makes the queue-per-interval construction a natural fit and makes the plugin unnecessary.
The second is a per-message deadline, where a message carries its own arbitrary delay: remind this user in 73 minutes. The delays are unbounded in variety, which is what the plugin exists for and what the constructions cannot do without a queue per distinct delay.
The third is a recurring schedule, where something should happen every hour. A broker is the wrong tool for the third, because nothing in the model generates a message, and the standard mistake is to simulate it by having a consumer republish with a delay, which produces a chain that stops permanently the first time one link is lost. A recurring schedule needs a scheduler, and the broker's role is to deliver what the scheduler emits.
Per-message TTL is evaluated only when a message reaches the head of its queue, so a one-second deadline behind a one-hour deadline waits the full hour. Per-queue TTL does not have the problem because every message shares the same deadline, which makes head order and deadline order coincide. That single fact is why every delay construction built from TTL uses a queue per interval rather than a per-message value.