In a distributed data storage system, consistent hashing is a popular method used to efficiently distribute data across multiple nodes. This method helps in minimizing the amount of data transfer when nodes are added or removed from the system. However, one of the challenges in such a setup is handling READ and WRITE requests during rehashing, especially for failed nodes. In this article, we delve deep into how these requests are managed in the face of node failures and rehashing.

Understanding Consistent Hashing

Consistent hashing maps a data item to a point in a circular hash space (sometimes called the "ring"), and each node in the system is responsible for a specific segment of that ring. When a node is added or removed, only a small proportion of keys, typically those around the hash ring near the new or removed node, are remapped.

Node Failures and Impact on Data

When a node in a consistent hashing-based system fails, its portion of the hash ring must be covered by other nodes to ensure data availability and system reliability. Typically, each key is not just stored on a single node; replicas of the key are stored on multiple nodes. This replication strategy enhances fault tolerance.

REHASHING Process

Rehashing occurs when nodes are added to or removed from the distributed system. Each node's responsibilities might change because the range of hashes it manages could expand or shrink. During rehashing, the system must reassess and possibly relocate data to ensure that all items are correctly mapped to their respective nodes.

Handling READ/WRITE Requests During Rehashing

Handling requests during rehashing, particularly when the involved nodes are failed, needs careful consideration. Below we discuss how READ and WRITE requests can be managed:

1. READ Requests:
   • When a READ request is issued to a failed node (node responsible for the data's hash), the system will reroute the request to one of the replica nodes.
   • Consistent hashing often extends with a mechanism for maintaining an updated list of node statuses and their ranges in the ring. By querying this list, any node in the system can redirect a READ request to an appropriate active node.
   • Techniques like read-repair are used during READ operations, which helps in updating stale replicas on-the-fly when a divergence in data versions is detected.
2. WRITE Requests:
   • WRITE requests during the rehashing must also be routed to replica nodes.
   • Writes follow the principle of write quorum, wherein the WRITE must succeed in a majority of the replica nodes for the WRITE to be considered complete.
   • This quorum approach ensures that even if there’s a failure during rehashing, the system maintains data integrity and availability.

Example Scenario

Consider a cluster with 5 nodes and each piece of data is replicated 3 times. If node 2 fails and is being replaced during rehashing, WRITE and READ requests for data primarily managed by node 2 would redirect to its replica nodes (say, nodes 1 and 3). Even in the presence of node failure and active rehashing, data can still be accessed and written to reliably.

Summary Table

The following table summarizes how data requests are handled during rehashing when a node fails:

Conclusion

Handling READ and WRITE requests in a consistent hashing-based datastore during rehashing and node failures is crucial for maintaining data availability and integrity. Techniques like replication, read-repair, and maintaining updated node lists ensure that even as nodes come and go, the system can continue operating without data loss or significant performance degradation. As distributed systems scale and handle more critical applications, refining these strategies remains a key area of research and development.