Why or why not use RequestVote RPC as heartbeat in Raft implementation?
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
In the Raft consensus algorithm, two of the primary RPC (Remote Procedure Call) types used are RequestVote and AppendEntries. The AppendEntries RPC is traditionally employed for two main purposes: to replicate log entries and to serve as a heartbeat mechanism. The purpose of this article is to explore why using RequestVote RPC as a heartbeat would be a suboptimal choice in Raft's implementation and to discuss the roles these RPCs play within the Raft protocol.
Understanding RequestVote and AppendEntries RPCs
Before delving into the nuances of why RequestVote should not be used as a heartbeat, let's define these RPCs clearly:
- RequestVote RPC: This is primarily used during the Leader Election phase in Raft. A candidate node sends this RPC to all other nodes in the cluster to request their votes to become the leader. This message includes information like the candidate’s log index and term to help other nodes decide whether the candidate is sufficiently up-to-date to be considered for leadership.
- AppendEntries RPC: This RPC is used both for log replication and as a heartbeat. As a log replication tool, it contains log entries that the leader sends to the followers to be appended to their logs. As a heartbeat, it is sent periodically (empty of log entries) to maintain the authority of the leader and to prevent new elections.
Technical Reasons Against RequestVote as Heartbeat
- Overhead and Confusion: Using
RequestVoteRPCs as a heartbeat would introduce unnecessary overhead and confusion. EveryRequestVoteRPC requires nodes to evaluate and potentially update their current term and reset their leader election timers, which is more computationally and conceptually excessive compared to the simplicity of anAppendEntriesheartbeat that simply maintains leader authority. - Elections Storms: More frequent issuance of
RequestVoteRPCs could lead to frequent election storms where multiple nodes keep considering themselves as potential leaders and initiate votes. This could destabilize the cluster especially in scenarios of network delays or partitions. - Clarity and Separation of Concerns: Raft protocol's clarity comes from the well-defined roles of its RPC mechanisms. Mixing these roles can lead to increased complexity in understanding and maintaining the system. Keeping
AppendEntriesfor both log replication and as a heartbeat while reservingRequestVotesolely for elections maintains a separation of concerns that simplifies reasoning about the system. - Increased Risk of Split Brain: Frequent and unnecessary elections initiated by misusing
RequestVoteas a heartbeat could increase the chances of split-brain scenarios, where clusters may end up with more than one leader, leading to data inconsistencies.
Example Scenario
Consider a cluster of 5 nodes where RequestVote is used instead of AppendEntries for heartbeats. If the network faces intermittent failures, the frequent RequestVote RPCs could lead to nodes flipping terms and leaders rapidly, without any stable leadership, causing constant flux and inefficiency in log replication and overall system performance.
Summary Table
| Feature | AppendEntries as Heartbeat | RequestVote as Heartbeat |
| Complexity | Low | High |
| Stability | High | Low |
| Consistency | Maintains consistent leader | Prone to inconsistencies |
| Overhead | Minimal | High |
| Purpose Fulfillment | Fulfills dual-purpose efficiently | Mixes concerns leading to inefficiency |
Conclusion
In summary, using RequestVote RPC as a heartbeat in a Raft implementation disrupts the algorithm's efficiency and stability. The separation of concerns provided by distinct RPCs for leader election and log replication/heartbeat ensures that Raft can be a reliable and understandable system. This clarity helps in maintaining consistency across distributed systems, proving essential in environments requiring robust data management and fault tolerance. Hence, sticking to AppendEntries as a heartbeat is considered a best practice in implementing Raft.
Related reading
- why should a producer write to odd number of servers in case of a distributed message queue
- Why training speed does not scale with the batch size?
- Why we need partition tolerance [CAP]
- Wildfly 17 Distributed Infinispan Cache Session not found
- Why prefer start end - start / 2 over start end / 2 when calculating the middle of an array?
- Why Q.head Q.tail 1 represents the queue is full in CLRS
- Will Hazelcast IMap block when waiting for an entry in transit?
- Will replicated data in cluster use same memory space in every system?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.