Python programming
RabbitMQ
Content-based routing
Message queuing
Software Development

Content-based routing with RabbitMQ and Python

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Content-based routing means sending a message to different consumers based on what the message represents. In RabbitMQ, that usually does not mean the broker inspects the JSON body directly. Instead, you expose the routing information through a routing key or message headers and let the exchange route from that metadata.

That distinction matters. RabbitMQ is very good at metadata-based routing, but if you truly need to inspect arbitrary payload content, your application usually has to do that before publishing or in a separate routing service.

Pick the Right Exchange Type

RabbitMQ offers several exchange types, but two are most relevant here:

  • 'topic when routing is encoded in a routing key'
  • 'headers when routing is based on message headers'

If the question is "route invoices to accounting and alerts to operations," a topic exchange is often enough. If the question is "route only messages whose department is finance and priority is high," a headers exchange is a cleaner fit.

RabbitMQ Does Not Parse the Body for You

This is the point many introductions blur. If you publish a JSON message body, RabbitMQ does not automatically inspect fields such as "type" or "region" and route on them. You have to choose one of these patterns:

  • copy relevant values into the routing key
  • copy relevant values into headers
  • build an application-level router that reads the body and republishes

For most systems, putting routing fields into headers is explicit and maintainable.

Example With a Headers Exchange

Assume you want to route reports by department and priority. First create a consumer that receives only high-priority finance messages:

python
1import json
2import pika
3
4connection = pika.BlockingConnection(pika.ConnectionParameters("localhost"))
5channel = connection.channel()
6
7channel.exchange_declare(exchange="reports", exchange_type="headers", durable=True)
8channel.queue_declare(queue="finance_high", durable=True)
9channel.queue_bind(
10    exchange="reports",
11    queue="finance_high",
12    arguments={
13        "x-match": "all",
14        "department": "finance",
15        "priority": "high",
16    },
17)
18
19def callback(ch, method, properties, body):
20    payload = json.loads(body)
21    print("finance_high received:", payload)
22    ch.basic_ack(delivery_tag=method.delivery_tag)
23
24channel.basic_consume(queue="finance_high", on_message_callback=callback)
25print("Waiting for finance_high messages")
26channel.start_consuming()

Now publish messages with matching headers:

python
1import json
2import pika
3
4connection = pika.BlockingConnection(pika.ConnectionParameters("localhost"))
5channel = connection.channel()
6
7channel.exchange_declare(exchange="reports", exchange_type="headers", durable=True)
8
9messages = [
10    (
11        {"department": "finance", "priority": "high"},
12        {"report_id": 1, "text": "Quarter close warning"},
13    ),
14    (
15        {"department": "hr", "priority": "low"},
16        {"report_id": 2, "text": "New onboarding packet"},
17    ),
18]
19
20for headers, payload in messages:
21    channel.basic_publish(
22        exchange="reports",
23        routing_key="",
24        body=json.dumps(payload),
25        properties=pika.BasicProperties(
26            headers=headers,
27            delivery_mode=2,
28            content_type="application/json",
29        ),
30    )
31
32connection.close()

Only the first message reaches finance_high, because the binding requires both header values to match.

When a Topic Exchange Is Better

If the routing dimensions are naturally hierarchical, a topic exchange may be simpler. For example, routing keys like report.finance.high or report.ops.low are easy to bind with patterns such as report.finance.*.

That looks like this:

python
1channel.exchange_declare(exchange="reports_topic", exchange_type="topic")
2channel.queue_bind(
3    exchange="reports_topic",
4    queue="finance_all",
5    routing_key="report.finance.*",
6)

The benefit is simplicity. The tradeoff is that routing fields now live inside the routing key instead of explicit headers.

Durability and Delivery Semantics

Routing is only part of the design. For a production queue, you also need to think about persistence and acknowledgments.

  • declare durable exchanges and queues when messages must survive broker restarts
  • publish persistent messages with delivery_mode=2
  • use manual acknowledgments for consumers that do real work

Those choices do not create exactly-once processing, but they do make the pipeline more robust.

Common Pitfalls

The biggest pitfall is assuming RabbitMQ can route on arbitrary JSON body fields by itself. It cannot. If the routing decision depends on the body, your publisher or a separate routing component has to expose that information to the broker.

Another common mistake is using auto_ack=True in examples and then copying that into production. If the worker crashes after receiving a message, the message is already gone.

Teams also forget to declare the same exchange and queue properties everywhere. If one process declares a durable queue and another declares it differently, RabbitMQ will reject the mismatch.

Finally, do not overload the routing design with too many dimensions. If the rules become hard to reason about, introduce clearer message contracts or a dedicated routing service.

Summary

  • RabbitMQ routes on exchange metadata such as routing keys and headers, not arbitrary message bodies.
  • 'headers exchanges work well when routing depends on named attributes.'
  • 'topic exchanges work well when routing can be encoded into structured routing keys.'
  • Durable queues, persistent messages, and manual acknowledgments are important for production reliability.
  • If you need true payload inspection, route in application code before or after publishing.

Course illustration
Course illustration

All Rights Reserved.