Every RabbitMQ problem that looks mysterious at the application level turns out to be a connection or channel problem, and the reason is that the AMQP 0-9-1 protocol puts two layers of multiplexing between an application and the broker, each with its own limits, its own lifecycle and its own way of failing.

A connection is a TCP socket with a protocol on top

One socket, and it is expensive enough that the count matters.

A RabbitMQ connection is a single TCP connection carrying the AMQP 0-9-1 protocol. On the broker side each connection costs a pair of Erlang processes, a receive buffer and a send buffer, and the memory figure usually quoted is around 100 KB per connection before any unusual buffer tuning, which means 10,000 connections is a gigabyte of broker memory doing nothing but holding sockets open.

That cost is why the protocol has channels at all. Opening a connection per publish, which an HTTP-shaped mental model suggests, is the single most common way to make a RabbitMQ cluster unhappy: the handshake is seven round trips including authentication, and a broker spending its time on handshakes is a broker not delivering messages.

The connection is also where TLS, authentication and the virtual host live. One connection authenticates once, as one user, into exactly one vhost, and none of those three can change for the life of the connection. An application needing two vhosts needs two connections, which is a design constraint rather than an inconvenience.

A channel is a lightweight session inside it

The multiplexing layer, and the part applications get wrong.

A channel is a logical, bidirectional stream multiplexed over the connection, identified by a 16-bit channel number. Publishing, consuming, declaring and acknowledging all happen on a channel rather than on the connection, and the broker tracks per-channel state: the consumers registered on it, the unacknowledged deliveries outstanding, the prefetch limit, and whether publisher confirms or transactions are enabled.

channel_max is negotiated at connection time and RabbitMQ's default is 2047, which is the practical ceiling rather than the protocol's 65,535. A client asking for more gets the server's value, because negotiation takes the lower of the two.

The rule that is not obvious from the API is that a channel is not thread-safe. Two threads publishing on one channel interleave their frames, and because a message is sent as a method frame followed by a header frame followed by body frames, interleaving corrupts the stream. Most client libraries either lock internally or document that you must not do this, and the failure when you do is a connection-level error rather than a clean exception.

Frames, which is what actually crosses the wire

Five types, and the layout explains several limits.

Every frame has the same envelope: a one-byte type, a two-byte channel number, a four-byte payload length, the payload, and a one-byte frame-end marker of 0xCE. The seven bytes of header plus one of terminator are eight bytes of per-frame overhead, which is why very small messages are inefficient in a way that has nothing to do with the broker's storage.

The types are method (type 1), carrying a class and method id plus arguments, content header (type 2), carrying the body size and the message properties, content body (type 3), carrying raw bytes, heartbeat (type 8), and the protocol header, which is not really a frame and is sent once.

A single published message is therefore at least three frames: basic.publish as a method frame, then a content header, then one or more body frames. A message larger than the negotiated frame_max is split across body frames, and frame_max defaults to 131072 bytes in RabbitMQ, negotiated down to the lower of client and server.

Laid out as a structure, the envelope is small enough to read:

1struct amqp_frame {
2    uint8_t  type;        /* 1 method, 2 header, 3 body, 8 heartbeat */
3    uint16_t channel;     /* 0 for connection-level frames */
4    uint32_t size;        /* payload length in bytes */
5    uint8_t  payload[];   /* size bytes */
6    uint8_t  frame_end;   /* always 0xCE */
7};

The channel field is two bytes, which is where the protocol's 65,535 channel ceiling comes from, and channel 0 is reserved for connection-level methods, which is why connection.start and connection.close do not belong to any channel. The size field is four bytes and declares the payload length, so a parser reads a fixed seven-byte header and then exactly that many bytes, followed by the one terminator byte. And the frame_end byte is a sanity check: if the byte following the payload is not 0xCE, the parser has lost its place and the only correct response is 501 FRAME_ERROR and a closed connection, because there is no way to resynchronise a stream whose framing is wrong.

That last point is worth sitting with, because it explains why protocol errors in RabbitMQ tend to kill a whole connection rather than one operation. A length-prefixed stream has no recovery point. Once the reader is off by a byte, every subsequent length it reads is garbage, so the broker cannot skip the bad frame and continue.

Heartbeats, and the failure they are there to catch

The setting most often disabled by accident.

TCP will happily hold a connection open that has no peer, because a half-open connection produces no packets and therefore no errors, and the default kernel keepalive is measured in hours. AMQP's answer is a heartbeat frame on an interval negotiated at connection time, with RabbitMQ's default 60 seconds and the convention that a peer missing two intervals is dead.

The consequence is that a crashed consumer's connection is detected in about 120 seconds, and until it is, every message delivered to it is outstanding and unacknowledged and is not redelivered to anybody. A queue appearing stuck with messages in the unacked state and no consumer activity is usually this, waiting out the heartbeat.

Lowering the interval shortens that window and costs more frames, which is almost always the right trade for a consumer fleet. The thing not to do is set it to 0, which disables heartbeats entirely, because then a dead peer is detected whenever the kernel decides to notice, which can be hours.

A subtlety worth knowing: heartbeats are suppressed while a connection is actively sending data, since traffic itself proves liveness. A connection that is blocked by flow control is not sending application data, and is still expected to send heartbeats, which is why a blocked connection that also stops heartbeating is a client bug rather than a broker one.

What a channel error does and what a connection error does

Two severities, and conflating them produces an application that silently stops working.

A channel exception closes the channel and leaves the connection open. The broker sends channel.close with a reply code and text, and everything the channel held is gone: its consumers are cancelled and its unacknowledged deliveries are requeued. The common codes are 404 NOT_FOUND for an absent exchange or queue, 403 ACCESS_REFUSED for a permissions failure, 406 PRECONDITION_FAILED for redeclaring a queue with different arguments, and 405 RESOURCE_LOCKED for consuming from an exclusive queue owned by someone else.

A connection exception closes the socket and every channel on it. The codes include 501 FRAME_ERROR, 503 COMMAND_INVALID, 504 CHANNEL_ERROR and 506 RESOURCE_ERROR, and they are mostly protocol violations, which means they are mostly client bugs.

The operational point is that an application which catches exceptions and continues, without reopening the channel, has an object that looks like a channel and is closed. Publishes on it fail, and in a client that publishes without confirms they fail silently.

Connection churn, which is measurable before it hurts

One metric, and it predicts an outage.

The management plugin and the Prometheus plugin both expose connection creation and closure rates. A healthy application fleet opens connections at deployment time and then holds them, so the steady-state rate is near zero, and a rate of hundreds per second is an application creating a connection per operation.

The reason to catch it early is that connection churn degrades non-linearly. Each handshake is cheap in isolation and the authentication step may hit an external backend, so a churn rate that is survivable at normal load becomes a thundering herd at the moment the fleet restarts, which is also when the broker is least able to absorb it.

The fix is a connection pool with channels per worker, which is what every mature client library offers and what every tutorial example omits, because a tutorial that opened a pool would be a worse tutorial and a better example.

The object model is four nouns, and the single most useful thing to internalise about it is that a publisher never names a queue. Everything else about RabbitMQ's routing flexibility, and most of its surprises, follows from that one piece of indirection.

A publisher publishes to an exchange

Never to a queue, and the apparent exception proves the rule.

basic.publish takes an exchange name and a routing key. It does not take a queue name, and there is no protocol method that publishes to a queue. The exchange receives the message, consults its bindings, and copies a reference to it into zero or more queues.

The apparent exception is the common tutorial line that publishes with an empty exchange name and a routing key equal to a queue name, and that does work. It works because every vhost has a default exchange, a direct exchange named with the empty string, to which every queue is automatically bound using its own name as the routing key. So the tutorial is still publishing to an exchange and relying on an implicit binding, which is worth knowing because the implicit binding cannot be removed and is the reason a queue is always reachable by its own name.

The consequence of the indirection is the property RabbitMQ is chosen for: a publisher can be written once and the set of consumers changed afterwards, by adding bindings, with no change to the publisher and no redeployment.

Zero queues is a successful publish

The default that loses messages, and it is a default.

If an exchange's bindings match no queue, the message is discarded. basic.publish is an asynchronous method with no reply, so the publisher is told nothing, and without publisher confirms it cannot even know the broker received it.

This is the single most common way messages disappear in RabbitMQ, and it is almost always a binding that was never created or a routing key with a typo. The broker is behaving correctly: an exchange is a routing function, and a function with no matching output has no output.

Two mechanisms exist to make it visible. The mandatory flag on publish asks the broker to return an unroutable message to the publisher via basic.return, which the client must have a handler for. And an alternate exchange, set with the alternate-exchange argument when declaring an exchange, sends anything unroutable to a second exchange, which is usually bound to a queue somebody actually watches.

The alternate exchange is the better of the two in production, because it does not depend on a publisher being connected or on the client library having a return handler wired up, and because it produces a queue whose depth is an alert.

A queue is the only thing that stores

Which decides where every resource limit applies.

Exchanges are stateless routing tables. They hold no messages, have no depth, and consume essentially no memory beyond their binding table. Queues hold messages, and therefore every question about memory, disk, durability, ordering and delivery is a question about a queue.

A queue is also, in the classic implementation, a single Erlang process, which is the fact that explains the most about RabbitMQ's performance profile. One queue is handled by one process, so one queue's throughput is bounded by one scheduler running one process, and a cluster with one very hot queue cannot use its other cores for it. Scaling throughput means more queues, not more nodes.

That single process also serialises everything about the queue, which is where the ordering guarantee comes from: messages enter and leave a queue in order, because one process handles them one at a time. The guarantee is per queue, and it survives exactly as far as the first redelivery.

Bindings, and the fact that they are many to many

The piece that makes topologies rather than pipes.

A binding joins an exchange to a queue, or an exchange to another exchange, with a routing key and optionally a table of arguments. One exchange can bind to many queues and one queue can be bound to many exchanges, and both multiplicities get used.

Many queues bound to one exchange is fan-out: every matching queue gets its own copy, and the copies are independent thereafter, so a slow consumer on one queue does not affect another.

One queue bound to many exchanges is less common and genuinely useful, typically for a monitoring or audit queue that collects from several unrelated flows.

Exchange-to-exchange bindings are a RabbitMQ extension rather than part of AMQP 0-9-1, and they let a topology be composed: a topic exchange bound to a fan-out exchange, say, so one routing decision feeds a group of queues that can change without touching the first exchange. The cost is that the path a message takes is no longer visible from one binding table, which makes debugging harder, so the extension is worth using sparingly.

What the broker charges for a binding

Because a publish searches the table, and the table has a size.

Every publish evaluates the exchange's bindings, which means the binding table is a data structure the broker searches on the hot path. A direct exchange is a hash lookup on the routing key and is effectively free. A fan-out exchange ignores the key and copies to everything bound, so its cost is the number of bound queues. A topic exchange matches a pattern and uses a trie, so its cost grows with the depth of the routing key rather than the number of bindings. A headers exchange compares an argument table against every binding and is the one with no index at all.

The number that matters in practice is not the binding count but the copy count, because routing is cheap and copying is not. A fan-out exchange with 500 bound queues turns one publish into 500 enqueues across 500 separate Erlang processes, so a publisher sending 1,000 messages per second is asking the broker to do 500,000 enqueues per second. That is the usual reason a cluster with a modest publish rate is saturated.

The design response is to question whether every one of those queues needs its own copy. A fan-out to 500 consumers is often really a fan-out to 5 groups of 100 workers, which is five queues rather than 500, and the hundred-fold reduction in enqueue work is the whole difference between a healthy cluster and an unhappy one.

Auto-delete, exclusive, and the queues that disappear

Two flags that are useful and are often set by accident.

An exclusive queue belongs to the connection that declared it, is invisible to every other connection, and is deleted when that connection closes. It is the right tool for an RPC reply queue and for anything per-client, and it is a trap in a worker pool, where a second consumer attempting to attach gets 405 RESOURCE_LOCKED.

An auto-delete queue is deleted when its last consumer disconnects, having had at least one. That condition is more subtle than it reads: a queue declared and never consumed from is not deleted, and a queue whose consumers all briefly drop during a deployment is deleted along with everything in it, which is the usual way an auto-delete queue loses messages.

Both flags participate in the declaration equivalence check, so neither can be changed later without deleting the queue, and both are set unintentionally by client libraries whose convenience helpers default them to true. Checking the actual flags in rabbitmqctl list_queues name durable auto_delete exclusive is worth doing once per environment, because the answer is often not what the code appears to say.

Declaration is idempotent until it is not

The error everybody meets once.

queue.declare and exchange.declare are idempotent: declaring something that already exists with the same arguments succeeds and returns its current state, which is why clients can declare on every startup and usually should.

Declaring something that exists with different arguments fails with 406 PRECONDITION_FAILED and closes the channel. The arguments compared include durability, auto-delete, exclusivity and the whole x- argument table, so adding x-message-ttl to a queue that already exists without it is a channel error rather than a modification.

There is no queue.modify in the protocol. Changing a queue's arguments means deleting and recreating it, which means losing its messages, which is why a queue's arguments are closer to a schema decision than a configuration one.

The way out, and the reason it exists, is a policy. Policies are applied by the broker to queues matching a name pattern and can set many of the same properties, including TTL, length limits and the dead-letter exchange, without redeclaring anything. Anything that might need to change later belongs in a policy rather than in the declaration.

Durability, which is three separate decisions

Conflated constantly, and the conflation loses messages.

A durable exchange survives a broker restart as a definition. It says nothing about messages.

A durable queue survives a broker restart as a definition, including its bindings. It says nothing about whether the messages in it survive.

A persistent message, meaning one published with delivery_mode set to 2, is written to disk. A persistent message in a non-durable queue is lost on restart because the queue is gone, and a transient message in a durable queue is lost on restart because the message was never written.

Surviving a restart requires all three: durable exchange, durable queue, persistent message. Getting two of the three is the standard way to be surprised, and the pairing that catches people most is a durable queue full of transient messages, which looks durable in the management UI and is not.

Even all three is not a guarantee of zero loss, because a persistent message is acknowledged to the publisher before it is necessarily fsynced, which is the subject of the publisher confirms lesson.

Reading the protocol's method classes is worth an hour because the AMQP 0-9-1 vocabulary is what every client library, every management screen and every log line uses, and because the split between synchronous and asynchronous methods explains which operations can fail silently.

The handshake, in order

Seven steps, and knowing them makes a connection failure readable.

The client opens a TCP connection and sends the literal protocol header: the bytes AMQP, then 0x00, then the major, minor and revision, which for AMQP 0-9-1 is 0x00 0x09 0x01. This is not a frame and gets no channel number.

The broker replies connection.start, advertising its supported authentication mechanisms and locales. The client answers connection.start-ok with a mechanism, usually PLAIN, and the credentials, plus a client properties table that is where a client library's name and version come from in the management UI.

Then connection.tune from the broker, proposing channel_max, frame_max and heartbeat, and connection.tune-ok from the client, which is where all three are settled at the lower of the two proposals. Then connection.open naming the vhost, and connection.open-ok.

A failure at start-ok is authentication, and RabbitMQ deliberately reports it as a connection close with 403 rather than distinguishing a bad user from a bad password. A failure at open is the vhost, either missing or not permitted for that user, which is a different problem that produces a similar-looking error.

Synchronous and asynchronous methods

The distinction that decides what can fail silently.

Most methods are synchronous: they have a matching reply and the client waits for it. queue.declare returns queue.declare-ok carrying the queue name and its current message and consumer counts. exchange.declare, queue.bind, basic.consume, basic.qos and basic.get all have an -ok.

A handful are asynchronous, with no reply at all, and basic.publish is the important one. A publish is sent and that is the whole interaction, which is why an unroutable message is not an error and why a publish to a cluster that is about to lose the message succeeds.

basic.ack, basic.nack and basic.reject are also asynchronous, in the other direction, which means a consumer's acknowledgement is fire-and-forget. If the channel closes between processing and acking, the message is requeued and processed again, and the consumer has no way to learn that happened.

This asymmetry is the source of RabbitMQ's delivery semantics: at-least-once is what the protocol naturally provides, and the mechanisms layered on top, confirms in one direction and acknowledgements in the other, are what narrow the window.

The classes worth knowing by name

Six, and they map onto what you do.

connection and channel manage the two layers of the previous section, including channel.flow which the broker historically used to ask a publisher to pause and which modern RabbitMQ replaces with connection blocking.

exchange and queue are declaration and binding: declare, delete, bind, unbind, plus queue.purge.

basic is the data path and is the class with the most methods: publish, deliver, consume, cancel, get, ack, nack, reject, qos, recover, return. Almost every runtime behaviour in RabbitMQ is a basic method.

tx is AMQP's transaction class, select, commit and rollback, and it is a trap discussed in its own lesson. confirm is a RabbitMQ extension, confirm.select, that enables publisher confirms, and it is the mechanism that tx should have been.

Push and pull, and why one of them is a mistake

Two ways to get a message, and they are not equivalent.

basic.consume registers a consumer on a channel, after which the broker pushes messages as basic.deliver frames whenever they are available and the consumer's prefetch window allows. This is the push model and it is the correct one: the broker decides when to send, deliveries arrive without a request, and throughput is limited only by the prefetch window and the consumer's speed.

basic.get is the pull model: one synchronous request returns one message or basic.get-empty. It looks convenient and it is a performance disaster, because it is a full round trip per message with no pipelining, so a consumer polling with basic.get at a 1 ms network round trip is capped at about 1,000 messages per second regardless of how fast the broker or the consumer is.

Worse, a polling loop with basic.get on an empty queue produces constant traffic and constant broker work for nothing, and the usual mitigation of adding a sleep converts it into a latency problem.

basic.get is correct for exactly one shape of task: a scheduled job that drains a queue once and exits. Everything else should use basic.consume, and a codebase using basic.get in a loop is a codebase with a performance bug it has not measured yet.

basic.qos, which is where throughput is decided

One method, three arguments, and the one that matters is not the obvious one.

basic.qos sets the prefetch window: how many unacknowledged deliveries the broker will allow outstanding on a channel before it stops sending. It takes a prefetch_size in bytes, which RabbitMQ does not implement and ignores, a prefetch_count, and a global flag.

The default is no limit at all, which means a consumer that attaches with manual acknowledgement and no basic.qos call receives the entire queue as fast as the socket allows. A queue of two million messages delivered to one consumer's in-memory buffer is how a consumer gets killed by the kernel's out-of-memory handler within seconds of starting, and the broker is doing exactly what it was asked.

The global flag decides whether the count applies per consumer or per channel, and RabbitMQ's interpretation differs from the specification's: false, the default, means per consumer, so a channel with four consumers and a prefetch of 10 has up to 40 deliveries outstanding. true means the limit is shared across the channel. Most applications want the default and most applications do not know which they have.

Consumer tags and basic.cancel

The handle on a consumer, and the two ways it goes away.

basic.consume returns a consumer tag, a string that identifies that consumer on that channel. The client may supply one, and if it does not the broker generates something like amq.ctag- plus a random suffix. Every basic.deliver carries the tag, which is how a channel with several consumers knows which one a delivery belongs to.

basic.cancel from the client stops deliveries to that consumer, and the important detail is what happens to messages already in flight: they are not withdrawn, so a client that cancels and stops reading leaves unacknowledged deliveries outstanding until the channel closes. A clean shutdown is cancel, then drain and acknowledge what has already arrived, then close.

The broker can also send basic.cancel to the client, unprompted, when the queue a consumer is attached to is deleted. This requires the client to have advertised the consumer_cancel_notify capability, which every modern library does, and a client that ignores the notification sits waiting on a queue that no longer exists, consuming nothing and reporting no error. That silent-idle-consumer failure is common enough to be worth an explicit handler.

Reading the wire when the client is lying

Because sometimes the only honest source is the bytes.

RabbitMQ has a rabbitmq_tracing plugin and, underneath it, a firehose mechanism: setting rabbitmqctl trace_on publishes a copy of every message entering and leaving the broker to the amq.rabbitmq.trace topic exchange, with routing keys publish.<exchange> and deliver.<queue>. That is the authoritative answer to where a message went, and it costs real throughput, so it belongs in a staging environment or in a short production window.

rabbitmqctl list_queues, list_bindings, list_consumers and list_channels answer most questions without it, and the one most people do not know is list_bindings, which prints the actual binding table and settles routing arguments quickly.

For protocol-level problems there is no substitute for a packet capture, and the frame layout from the previous section is what makes one readable: the type byte, the channel number, and the 0xCE terminator are enough to find frame boundaries by eye.

The habit worth building is to describe a problem in method names rather than in library terms. "The consumer stops after a while" is a different investigation from "deliveries stop arriving and no basic.cancel was received", and the second sentence is most of the way to an answer. Client libraries wrap these methods in idiomatic names that differ from language to language, and the protocol vocabulary is the only thing the broker's logs, the management UI and the documentation all share.

A message's content header frame carries fourteen predefined properties, and the useful distinction among them is not what the documentation says they mean but which ones the broker actually interprets. Three change the broker's behaviour, one is validated, and the remaining ten are metadata the broker carries and ignores.

The three the broker acts on

delivery_mode, expiration and priority, and each one has a trap.

delivery_mode is 1 for transient and 2 for persistent, and persistent means the broker writes the message to disk. The trap is that it says nothing on its own: a persistent message in a non-durable queue is lost on restart anyway, and a persistent message is acknowledged to a publisher potentially before its write is fsynced.

expiration is a per-message time to live, and it is a string containing a number of milliseconds, not an integer. A client passing an integer gets a 406 or has it silently coerced depending on the library. The trap is where expiry is evaluated: a message at the head of a queue is checked and discarded, but a message behind an unexpired one is not, so an expired message deeper in a queue continues to occupy it and continues to count toward the queue's depth until it reaches the head.

priority is an integer and only means anything if the queue was declared with x-max-priority. On an ordinary queue it is ignored entirely, which is a common surprise: the property is set, the publisher believes priority is in effect, and the queue is first in first out.

The one the broker validates

user_id, which is a security feature people turn into a bug.

If user_id is set, RabbitMQ checks it against the authenticated user of the publishing connection and rejects the message with 406 PRECONDITION_FAILED if they differ. The purpose is to let a consumer trust the claimed identity of a publisher, since the broker has already verified it.

The bug it becomes is a shared service account: an application publishing on behalf of end users cannot put the end user in user_id, because the connection authenticated as the service. The end user belongs in headers or app_id, and user_id is for the rarer case where the connection's identity is the thing a consumer needs.

The ten it carries and ignores

Which makes them conventions, and conventions need enforcing.

content_type and content_encoding describe the body and are purely advisory: the broker does not parse the body and does not care. correlation_id and reply_to are the request-reply convention, used by every RPC-over-RabbitMQ implementation and implemented entirely in application code.

message_id, timestamp, type and app_id are identification and provenance. cluster_id is deprecated and unused. And headers is a free-form table, which is the one with a special property: it is the only one a headers exchange can route on, which makes it the escape hatch for routing decisions that do not fit a dotted routing key.

Because the broker ignores them, these are organisational agreements rather than features. A platform that wants message_id and timestamp on everything has to enforce that in a shared publishing library, and the enforcement is worth having, because message_id is the natural idempotency key and timestamp is what makes an age-of-oldest-message measurement possible.

The x- headers, which are a different thing

Easy to confuse with properties, and they behave differently.

Entries in the headers table whose names begin with x- are reserved by convention, and the broker adds several of its own. The one everybody meets is x-death, which the dead-lettering machinery adds to record why and when a message was dead-lettered, which queue it came from, and how many times it has gone round.

x-death is the only built-in way to count retries, and its structure matters: it is an array of entries, one per dead-lettering reason and queue pair, each with a count. A retry ladder that reads the wrong entry's count either loops forever or gives up immediately, which is the single most common bug in a RabbitMQ retry topology.

The x- prefix also appears in queue and exchange arguments, x-message-ttl, x-max-length, x-dead-letter-exchange, x-max-priority, x-queue-type, and these are not message properties at all despite sharing the prefix. They are part of the declaration and participate in the equivalence check that produces 406 on a mismatched redeclare.

How properties are encoded, and why that costs something

A bit flag field, then only what is present.

The content header frame begins with the class id and weight, then the 64-bit body size, then a 16-bit property flags field in which each bit says whether the corresponding property is present. The values follow in order, only for the bits that are set, each encoded according to its type: a short string with a one-byte length prefix for content_type and the rest, an octet for delivery_mode and priority, a 64-bit timestamp, and a field table for headers.

So an absent property costs a bit and a present one costs its length plus a prefix, which makes the properties cheap individually and the headers table the only one that can get large. A field table is a four-byte length followed by name-value pairs, each name length-prefixed and each value tagged with a type byte, and tables nest. Several kilobytes of headers on every message is a real bandwidth and storage cost, and it is invisible in any metric that reports message counts.

The practical rule is that headers is for routing attributes and small metadata, not for the parts of a payload somebody did not want to put in the body. A tracing context, an idempotency key and a tenant identifier are the right size. A serialized object is not.

Which properties survive a hop, and which do not

The question that breaks retry topologies and tracing.

A message dead-lettered from one queue into another keeps its body and its properties, with two exceptions that matter. Its expiration is removed if the dead-lettering reason was expiry, which is what stops a message expiring again immediately in the retry queue. And x-death is added or extended in the headers table, which is the retry counter.

A message republished by an application, rather than dead-lettered by the broker, keeps nothing automatically. Every property has to be copied by hand, and the usual bug is a retry implementation that consumes, waits, and republishes while dropping correlation_id, which breaks the reply path, or dropping message_id, which breaks every consumer's deduplication at exactly the moment duplicates become likely.

The routing key a message arrived with is also not a property. It is an argument of basic.deliver rather than part of the message, and if a consumer needs to know it after a hop, it has to be copied into a header deliberately. x-death records the original routing key, which is the one place the broker preserves it, and reading it from there is more reliable than remembering to copy it.

Message size, and where it stops being free

Three thresholds, and only one of them is documented as a limit.

A message's body is just bytes to the broker, and the protocol imposes no maximum beyond the four-byte body-size field. RabbitMQ 3.8 and later enforce a max_message_size which defaults to 134217728 bytes, 128 MB, and earlier versions effectively allowed up to 2 GB with predictable consequences.

The first real threshold is frame_max at 131072, above which a body is split across frames, which is a small cost. The second is the embedding threshold for persistent messages: RabbitMQ writes small message bodies inline in the queue index and larger ones into a shared message store, with the boundary at queue_index_embed_msgs_below, default 4096 bytes. Crossing it changes a message from one write to two lookups, which is a real and invisible performance cliff.

The third is memory. A queue holds message bodies in memory until it decides to page them out, so average message size multiplied by queue depth is the memory a backlog costs, and a queue of 100,000 messages at 100 KB each is 10 GB, which is a memory alarm rather than a backlog.

The practical guidance is the same as everywhere else: keep messages small and put large payloads in object storage with a reference in the message, which turns a 10 GB backlog into a 10 MB one.

RabbitMQ is usually described as an AMQP broker, and the description hides the fact that AMQP 0-9-1 and AMQP 1.0 are unrelated protocols that share a name, and that RabbitMQ speaks at least five protocols besides. Knowing which one an application is using decides what guarantees it has.

AMQP 0-9-1 and AMQP 1.0 are different protocols

Not versions of each other, which the numbering actively conceals.

AMQP 0-9-1 is the protocol this lesson has described: it has a broker-side object model with exchanges, queues and bindings, and the client declares and binds them. The routing topology is part of the protocol.

AMQP 1.0 is a peer-to-peer messaging protocol with no broker model at all. It defines connections, sessions and links, and a link attaches to a named node whose meaning is entirely up to the implementation. There are no exchanges in AMQP 1.0, no bindings, and no declaration: the protocol specifies how two peers move messages, not what a broker does with them.

The standards history is the reason. AMQP 0-9-1 was finalised in 2008 as a complete broker specification. The working group then redesigned the protocol from scratch, and AMQP 1.0, ratified as an OASIS and later ISO standard in 2012, deliberately dropped the broker model to allow non-broker implementations. The two are not wire-compatible and a client for one cannot speak to a server expecting the other.

What that means for RabbitMQ

Two implementations, and one of them became native recently.

RabbitMQ's native protocol is AMQP 0-9-1, and its whole object model, management interface and documentation are expressed in those terms. AMQP 1.0 support existed for years as a plugin that mapped 1.0 links onto 0-9-1 concepts, and RabbitMQ 4.0 made AMQP 1.0 a natively supported protocol alongside 0-9-1 rather than a translation layer.

The practical consequence is that an AMQP 1.0 client talking to RabbitMQ addresses queues and exchanges through an address syntax rather than by declaring them, so the topology must exist already, created by an 0-9-1 client, by the management API or by configuration. An AMQP 1.0 application therefore cannot be self-configuring in the way an 0-9-1 application can, and that is a deployment constraint rather than a bug.

It also means that a team told to use AMQP because it is the standard may be reaching for either protocol, and the two choices lead to different code, different libraries and different capabilities. The question to ask is not whether a system speaks AMQP but which AMQP.

The other protocols, and the trap they share

Four more, and each is a plugin with a port.

MQTT on 1883, or 8883 with TLS, is the protocol of choice for devices and mobile clients because its frames are tiny and its session model tolerates intermittent connectivity. RabbitMQ maps MQTT topics onto a topic exchange, replacing / with ., and an MQTT subscription becomes a queue.

STOMP on 61613 is a text-based protocol that is trivial to implement and therefore popular in browsers and scripts, mapped onto the same object model.

Web STOMP and Web MQTT expose those two over WebSockets, which is how a browser talks to RabbitMQ directly.

And the stream protocol on 5552 is RabbitMQ's own binary protocol for streams, which is not a mapping onto the 0-9-1 model at all and is covered in the streams lesson.

The trap they share is that a message published over MQTT and consumed over AMQP 0-9-1 lands in an ordinary queue, so its durability, acknowledgement and dead-lettering behaviour are the 0-9-1 behaviour described in this lesson, not MQTT's. An MQTT publisher's QoS level does not translate into a RabbitMQ guarantee, and a team reasoning about QoS 1 semantics while the broker is applying queue durability rules is reasoning about the wrong system.

Virtual hosts, which are not namespaces

A stronger boundary than the name suggests.

A vhost is a completely separate object space: its own exchanges, queues, bindings, policies, permissions and user grants. Two vhosts can each hold a queue called orders and they have nothing to do with each other, and there is no way to route a message from one vhost to another. An exchange cannot bind to a queue in a different vhost, which makes a vhost a hard partition rather than a prefix.

The default vhost is /, which is why connection strings end in what looks like a trailing slash and why a URL-encoded %2F turns up in management API calls. Permissions are granted per user per vhost as three regular expressions, for configure, write and read, matched against resource names, and the defaults granted by the management UI's convenience button are .* for all three, which is full control.

The boundary is strong in the object model and weak in the resource model, and that is the trap. All vhosts on a node share the same Erlang VM, the same memory limit, the same disk and the same schedulers, so a runaway queue in a staging vhost takes down the production vhost next to it. A vhost is an isolation mechanism for names and permissions, not for capacity, and multi-tenancy that needs capacity isolation needs separate clusters.

How to tell which protocol a connection is using

Because the answer is rarely in the application's configuration.

rabbitmqctl list_connections name protocol user vhost prints the protocol per connection, and the values are informative: {0,9,1} for AMQP 0-9-1, {1,0} for AMQP 1.0, MQTT 3.1.1, STOMP 1.2 and so on. The management UI shows the same thing on the connections tab, along with the client library's name and version, which comes from the client properties table sent in connection.start-ok.

That client properties table is more useful than it looks during an incident. It carries the library name, its version, the platform and often the application name, so a broker under unexpected load can usually be traced to a specific deployment without asking anybody. Setting a meaningful connection_name property in every service is a five-line change that pays for itself the first time.

The port is the other giveaway, since each protocol has its own listener: 5672 for AMQP 0-9-1 and 1.0, 5671 for either with TLS, 1883 and 8883 for MQTT, 61613 for STOMP, 5552 for streams and 15672 for the management HTTP API. A protocol that is not listening is a plugin that is not enabled, which is the first thing to check when a client cannot connect at all.

Where RabbitMQ's model differs from a log

Worth stating once, because the rest of the course depends on it.

In a log-based broker, a message is appended to a partition and stays there until retention removes it, consumers track their own position, and the same message can be read by many independent consumers at different times.

In RabbitMQ's 0-9-1 model, a message is routed into queues at publish time and removed from a queue when it is acknowledged. The broker, not the consumer, tracks what has been delivered. Replay is not a feature, because there is nothing left to replay from: once acknowledged, the message is gone.

This single difference generates almost every other one. Fan-out in RabbitMQ means copying at publish time, so adding a consumer group after the fact means the historical messages do not exist for it. Ordering is per queue rather than per partition. Throughput scales with queue count rather than partition count. And a backlog is a memory and disk problem in the broker rather than a retention setting, because the broker is holding messages it intends to hand over rather than a log it intends to keep.

Streams, added in RabbitMQ 3.9, bring the log model into RabbitMQ deliberately, which is why they get their own lesson rather than being treated as another queue type.

What comes next

The next lesson takes exchanges apart. The four built-in types, direct, fan-out, topic and headers, with the matching algorithm each one implements and its cost; the * and # wildcards in topic routing keys and what they actually match; the binding table as a data structure the broker has to search on every publish; and the exchange types that come from plugins, including consistent hashing, which is how a RabbitMQ topology gets the partition-like sharding that the single-process queue makes necessary.