Why Aeron multicast is much slower than raw Tcp implementation
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Aeron and TCP/IP are two different communication protocols largely used in systems where data transmission speed and reliability are crucial. Specifically, in the context of financial trading systems or real-time data streaming services, choosing the right protocol can significantly affect system performance. This article will explore why Aeron, when set up with multicast configurations, may appear slower compared to a raw TCP implementation.
Understanding Aeron Multicast
Aeron is a high-performance, reliable multicast and unicast messaging system designed for low-latency applications, often used in Algorithmic trading, gaming, and large-scale data distribution systems. Multicast is a method where data is transmitted to multiple recipients using the least network bandwidth by sending a single copy of data, which is then distributed to other subscribers in the network.
Key Differences with TCP
TCP (Transmission Control Protocol) is a connection-oriented protocol, ensuring reliable, ordered, and error-checked delivery of a stream of bytes between applications running on hosts communicating over an IP network. Unlike Aeron’s multicast, which may target multiple subscribers simultaneously, TCP is fundamentally intended for point-to-point communication.
Performance Constraints of Aeron Multicast Compared to TCP
1. Transmission Mechanism
- Aeron Multicast: Relies on UDP (User Datagram Protocol) which is inherently faster because it is non-connection based and does not guarantee message delivery. It simply sends out packets to the multicast group without ensuring all members receive the packet, which can lead to packet loss.
- TCP: Provides built-in mechanisms for recovery, error-checking, and flow control. Each packet must be acknowledged by the recipient, and if not, it will be retransmitted. This can be slower but ensures data integrity and order.
2. Network Configuration and Traffic
- Aeron Multicast: Suitable for environments where multiple recipients are listening to the same source. However, in crowded networks, multicast traffic can lead to increased collisions and packet loss, necessitating high overhead for managing loss recovery and re-transmission through additional tools.
- TCP: More efficient in a point-to-point setting as it controls congestion and manages packet flow between the sender and the receiver, reducing the likelihood of the network becoming overwhelmed.
3. Use Cases and Environment Consideration
- Aeron Multicast: Optimally used in controlled network environments where there is minimal risk of packet loss and the network infrastructure supports efficient multicast routing.
- TCP: Preferred in environments where data integrity and order are crucial across distributed systems not designed to handle multicast efficiently.
Practical Implications
In practice, the apparent slowness of Aeron in multicast mode compared to TCP can be attributed to:
- The additional complexity in managing multicast groups and ensuring data integrity across multiple recipients.
- Higher packet loss rates, especially in uncontrolled or complex network topologies.
- The need for Aeron to implement additional mechanisms (e.g., retransmission or sequence numbering) that can inherently slow down the raw speed advantage offered by UDP.
Comparative Table: Aeron Multicast vs. TCP Implementation
| Feature | Aeron Multicast | Raw TCP Implementation |
| Reliability | Lower inherently, relies on application layer for enhancement | High, built-in mechanisms for delivery assurance |
| Suitable Environment | Controlled networks with multicast support | Any network topology |
| Speed of Transmission | Fast, but can be offset by loss recovery overhead | Consistently moderate due to reliability checks |
| Data Delivery Guarantee | No inherent guarantee, depends on network and application layer | Strong delivery, order and error-checking guarantees |
Conclusion
While Aeron multicast may offer impressive raw speed potential over UDP, real-world usage often demands adjustments that can nullify these advantages, particularly in diverse and uncontrolled network environments. TCP, being inherently robust, remains reliably moderate in speed but consistently slower in scenarios demanding high reliability and data integrity. Therefore, the choice between Aeron multicast and TCP should be influenced by specific application needs, network environment, and performance requirements.
Related reading
- Why can envoy sidecar control my traffic?
- Why can't an AWS lambda function inside a public subnet in a VPC connect to the internet?
- Why can't I connect AWS RDS instance from EC2 instance in another VPC after peering
- Why consumer hangs while consuming messages from Kafka on DC/OS using Client API for Java?
- Why are Docker container images so large?
- Why are hash table expansions usually done by doubling the size?
- Why do I always get 403 when fetching data with RestTemplate?
- Why do I get 411 Length required error in my HttpWebRequest?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.