0%
RabbitMQ Deep Dive: AMQP, Queues, and Delivery Guarantees
The Broker Model
Queues and Storage
Flow and Scale
Patterns
Acknowledgements and Redelivery
A consumer has four ways to finish with a message and they differ in ways that are not obvious from their names. Choosing wrongly is how messages are lost or looped forever.
The delivery tag, on the other side
The same mechanism, counted by the broker this time.
Every delivery to a consumer carries a delivery tag, a 64-bit integer unique within the channel and monotonically increasing. Unlike the publish side, the broker assigns these and the consumer only echoes them back, which removes the counting problem entirely.
The tag identifies the delivery rather than the message. The same message redelivered after a requeue arrives with a different tag, which is what makes a tag safe to use as a key in a consumer's in-flight map and useless as a message identity.
Tags are per channel and reset on a new channel, so an acknowledgement sent on a different channel than the delivery arrived on is a channel error, 406 PRECONDITION_FAILED, and closes the channel. That is the single most common cause of a consumer that works until it reconnects.
basic.ack, and the multiple bit again
Symmetric with the publish side, and more dangerous here.
basic.ack with a delivery tag and a clear multiple flag acknowledges exactly that delivery. With the flag set it acknowledges every outstanding delivery up to and including that tag, which is a useful batching tool and a way to silently discard work.
The danger is that a consumer processing messages concurrently has outstanding tags that are not finished. Acknowledging tag 50 with multiple set when tags 44 and 47 are still being processed tells the broker all of them are done, and if the consumer then crashes, 44 and 47 are gone. The flag is only safe when the consumer processes strictly in order, which concurrent consumers do not.
It is still worth using in that ordered case, because one frame per hundred messages instead of one per message is a measurable reduction in channel traffic at high rates. The rule is simply that the batch must be a prefix of the work actually completed.
basic.consume against basic.get
Two ways to receive, one of which is almost always wrong.
basic.consume registers a consumer and the broker pushes deliveries to it as they become available, respecting the channel's prefetch window. basic.get is a synchronous poll: one request, one reply carrying either a message or get-empty.
The poll costs a full round trip per message and cannot be batched, so a consumer using it is bounded by network latency exactly as a publish-and-wait publisher is. At 1 ms it reaches a few hundred messages per second regardless of how much work the broker or the consumer could otherwise do, and the fix is not a tuning change.
basic.get also defeats every fairness mechanism. There is no registered consumer, so single active consumer does not apply, prefetch does not apply, and the management UI shows the queue as having no consumers while messages are being drained. A queue that appears to have no consumers and a falling depth is being polled, which is a useful diagnostic and usually a finding rather than a design.
The legitimate uses are narrow: draining a queue in a script, a one-off administrative read, or a request/response pattern where a reply is awaited once. Anything that runs continuously should register a consumer.
Acknowledging on the wrong channel
A specific error worth recognising on sight.
Because tags are scoped to the channel, an acknowledgement must travel on the channel the delivery arrived on. Sending it elsewhere is 406 PRECONDITION_FAILED with a message naming an unknown delivery tag, and the channel closes, requeuing everything outstanding on it.
This happens in two realistic ways. A consumer that reconnects and acknowledges work it was holding before the disconnection uses tags that no longer exist, and a consumer that keeps a pool of channels and acknowledges on whichever is free picks the wrong one roughly in proportion to the pool size.
The second is harder to see because it works under light load, when one channel is usually free and it is usually the right one. Load increases the chance of selecting the wrong channel, which makes this a bug that appears only in production and only under traffic, which is the worst combination available.
basic.nack and basic.reject, and the requeue field
Where a message goes when the consumer refuses it.
basic.reject carries a delivery tag and a requeue boolean. With requeue true the message goes back to the queue for redelivery; with it false the message is dead-lettered if a dead-letter exchange is configured and discarded if not, which is the sharpest edge in this whole area.
basic.nack is RabbitMQ's extension of basic.reject that adds the multiple flag, so it can reject a range. There is no other difference, and basic.nack is the one to use because the batching option costs nothing.
The discard-when-no-dead-letter behaviour deserves emphasis. A consumer that rejects a malformed message with requeue=false on a queue with no dead-letter exchange has deleted it, permanently, with no record anywhere. That is the correct protocol behaviour and it is the mechanism behind a surprising share of "we lost messages" incidents.
autoAck, which is not an acknowledgement mode
A different delivery guarantee wearing the same clothes.
Setting no-ack true on basic.consume, usually exposed as autoAck, tells the broker to consider a message delivered the moment it is written to the socket. The broker removes it immediately and the consumer sends nothing back.
This converts the pipeline to at-most-once. A consumer that crashes mid-batch loses every message already in its socket buffer and prefetch window, which under load is thousands rather than one. The loss is silent on both sides, since the broker believes it delivered them and the consumer never saw them.
It also removes flow control, because basic.qos does not apply when there is nothing to acknowledge. The broker pushes as fast as the socket accepts, so a slow consumer accumulates deliveries in memory until it dies, which is the second failure mode and the more common one in practice. autoAck is correct only where losing a message is cheaper than processing it twice, such as metrics sampling or cache invalidation hints, and it is chosen by accident far more often than deliberately.
What happens to unacknowledged messages
The state they sit in, and how long.
An unacknowledged message is removed from the ready portion of the queue and held against the consumer's channel, counted as messages_unacknowledged. It is not available to any other consumer and it is not lost; it is reserved.
There is no timeout on that state from the queue's point of view, so a consumer that neither acknowledges nor disconnects holds the message indefinitely. The only things that end the state are an acknowledgement, a reject, the channel or connection closing, the consumer being cancelled, or the broker's own consumer_timeout.
This is why a stuck consumer looks like a stuck queue. Messages are neither delivered nor redelivered, the queue's ready count sits still while its unacknowledged count stays high, and nothing in the broker's logs mentions it. Reading the two counts separately is the diagnosis, and treating messages as one number hides it.
Acknowledgement is a storage write, not a signal
Why acknowledging has a cost.
For a quorum queue, an acknowledgement is appended to the Raft log and replicated, exactly like a publish. For a classic queue it is recorded in the index. Acknowledging is therefore work proportional to the message rate and not a free notification.
That doubles the log traffic a quorum queue handles relative to the publish rate, and once deliveries are counted too, a queue at 5,000 messages per second is handling at least 15,000 log entries per second. This is the arithmetic behind a quorum queue's disk consumption being larger than the stored message volume suggests.
It is also the reason acknowledgement batching is worth something beyond network savings. One basic.ack with multiple set for a hundred messages is one log entry rather than a hundred, which on a busy quorum queue is a material reduction in the dominant cost. The batching decision is a storage decision wearing a protocol detail's clothing.
A consumer that rejects a message with requeue set to false on a queue with no dead-letter exchange has permanently deleted it, with no record anywhere. That is correct protocol behaviour and it is the mechanism behind a surprising share of reported message loss. Configure a dead-letter exchange before writing any reject path, or the error handling is a delete statement.