RabbitMQ
Multiple Consumers
Message Queue
Distributed Systems
Consumer Singleton

rabbitmq multiple consumers on a queue- only one get the message

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

If multiple consumers are attached to the same RabbitMQ queue, each message is normally delivered to only one of them. That is the expected behavior, not a bug.

RabbitMQ queues implement the competing-consumers pattern. A queue distributes work among consumers; it does not broadcast one copy of each message to every consumer.

One Queue Means Work Sharing

When several consumers read from a single queue, RabbitMQ hands each message to one consumer. Over time the broker spreads work across consumers, often in a roughly round-robin way, though acknowledgments and prefetch settings affect the exact pattern.

That makes a single queue ideal for background jobs such as:

  • image processing
  • email sending
  • report generation
  • task workers that should not duplicate work

Here is a minimal consumer in Python:

python
1import pika
2
3def callback(ch, method, properties, body):
4    print("received:", body.decode())
5    ch.basic_ack(delivery_tag=method.delivery_tag)
6
7connection = pika.BlockingConnection(pika.ConnectionParameters("localhost"))
8channel = connection.channel()
9channel.queue_declare(queue="jobs")
10channel.basic_qos(prefetch_count=1)
11channel.basic_consume(queue="jobs", on_message_callback=callback)
12channel.start_consuming()

If you start two copies of this program, they will share the messages from jobs. Each individual message still goes to only one consumer.

If You Want Every Consumer to Get the Message

To broadcast a message to multiple consumers, do not attach them all to the same queue. Give each consumer its own queue and bind those queues to the same exchange.

That changes the topology from:

  • one exchange
  • one queue
  • many competing consumers

to:

  • one exchange
  • many queues
  • one consumer per queue, or several per queue if each queue is its own work pool

For example, two services that both need the same event should usually consume from separate queues bound to a fanout or topic exchange. Then RabbitMQ stores one copy of the message in each queue, and both services receive it independently.

Why "Only One Consumer Got It" Sometimes Looks Wrong

The confusion usually comes from mixing up queue semantics and pub-sub semantics.

If the mental model is "I published one event and all listeners should see it," then a single shared queue is the wrong design. RabbitMQ is doing exactly what a queue is supposed to do: giving the next job to one worker.

Another source of confusion is acknowledgments. If a consumer receives a message but crashes before acknowledging it, the broker can redeliver that same message to another consumer later. That does not mean both consumers processed it successfully. It means the first delivery was not completed.

The Role of Prefetch and Ordering

RabbitMQ does not guarantee perfectly alternating delivery between consumers. If one consumer is faster, or if prefetch allows a consumer to buffer multiple unacknowledged messages, the distribution may look uneven.

Setting basic_qos(prefetch_count=1) often helps create fairer work distribution because a consumer receives a new message only after acknowledging the previous one.

Even with fair dispatch, message order is not guaranteed across multiple consumers. If strict ordering matters, a single active consumer or a more carefully partitioned design may be necessary.

Common Pitfalls

  • Expecting one queue to behave like a broadcast channel.
  • Attaching many consumers to one queue when each service actually needs its own copy of every event.
  • Ignoring acknowledgments and then misreading redeliveries as duplicate successful processing.
  • Assuming round-robin delivery is exact even when prefetch and consumer speed differ.

Summary

  • Multiple consumers on the same RabbitMQ queue compete for messages.
  • Each message normally goes to one consumer, not all consumers.
  • If every consumer should receive the message, use multiple queues bound to an exchange.
  • Acknowledgment failures can cause redelivery, which is different from intentional fan-out.
  • Prefetch settings influence how evenly work is distributed.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design