RabbitMQ RPC across multiple rabbitMQ instances
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
RabbitMQ is a widely used open-source message broker that helps in decoupling applications by providing a platform to send and receive messages efficiently. While RabbitMQ is often used for managing message queues, it can also support an architectural pattern known as Remote Procedure Call (RPC). RPC allows a client to execute a procedure on a different server, which in traditional architectures can lead to tightly coupled systems. However, with RabbitMQ, RPC can leverage messaging to achieve loose coupling.
Understanding RPC with RabbitMQ
RPC in RabbitMQ can be implemented using two queues: one for sending commands (or requests) and another for receiving responses. Typically, the procedure or function that the client wants to run on the server is encapsulated as a message and sent to a request queue. The server listens on this queue, processes the message, and sends the result back on the response queue attached to a correlation ID unique to the request.
Extending RPC across Multiple RabbitMQ Instances
Extending RPC to work across multiple RabbitMQ instances involves linking these instances in some manner. There are multiple ways to achieve this, and the choice may depend on specifics like network layout, security requirements, and overall architectural goals. Below are two common methods:
- Federation: RabbitMQ federation extends a queue or exchange over multiple brokers. This method is suitable for networks where clusters are distributed across longer distances and not all nodes are always connected.
- Shovel: RabbitMQ Shovel is a plugin that acts as a bridge between brokers, forwarding messages from a queue on one broker to another. Unlike federation, it requires a more stable network connection but can handle higher data rates.
Each method of connecting RabbitMQ instances has its pros and cons, and the choice between them should be made according to the specific requirements of the infrastructure and the nature of the RPC operations.
Implementing Cross-Broker RPC
Suppose you have two RabbitMQ brokers: Broker A and Broker B. Broker A receives an RPC request from a client that needs to be processed by a service connected to Broker B. Here’s a step-by-step guide on how to implement this using RabbitMQ Shovel:
- Setup RabbitMQ on both brokers.
- Configure Shovel on Broker A to forward messages from the local RPC request queue to a remote RPC request queue on Broker B.
- Process the requests on Broker B and publish the responses back on a response queue that Broker A is subscribed to.
- Return response from Broker A to client.
This setup leverages the strengths of RabbitMQ and Shovel to distribute processing across multiple brokers while maintaining the integrity and responsiveness of RPC communications.
Technical Considerations
When implementing RabbitMQ RPC across multiple instances, several considerations must be kept in mind:
- Latency: Networking delays between brokers will add to the RPC response time.
- Reliability: Ensure that both request and response messages are delivered even in the case of network or broker issues. Configuring message acknowledgments and durable queues can help.
- Security: Messages may need to be encrypted, and connections should be secure, particularly if brokers are distributed across the internet.
- Monitoring: It is vital to monitor the queues and message flow to troubleshoot and ensure the health of the RPC system.
Example Configuration
Here’s a sample configuration overview by using RabbitMQ’s Shovel plugin:
| Component | Configuration Summary |
| Source Broker | Configure outgoing Shovel to forward RPC requests. This involves setting up a source queue, destination broker and destination queue. |
| Destination Broker | Configure services to consume requests and publish responses. Monitor response queue for ensuring service health. |
| Client | Send request to Source Broker and listen on a dedicated callback queue for the response. |
Conclusion
Implementing RPC across multiple RabbitMQ instances allows for flexible, scalable, and reliable service architectures. Using RabbitMQ tools like Federation or Shovel can greatly help in distributing workload and can enhance the robustness of an application infrastructure. With proper configuration, monitoring and security in place, RabbitMQ can provide a robust solution for handling distributed RPC calls.
Related reading
- RabbitMQ same message to each consumer
- RabbitMQ Scaling queues with the consistent hash exchange
- Rabbitmq server connection closing abruptly
- Rabbitmq server drops connection when client takes more than 60 seconds to acknowledge a message
- RabbitMQ single active consumer with passive failover consumers
- RabbitMQ Topic exchanges 1 Exchange vs Many Exchanges
- RabbitMQ set_permissions syntax
- RabbitMQ settings disappear on restart. Why?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.