0%
Redis Deep Dive: Encodings, Memory, and Failover
The Server Model
Data Structures Inside
Memory Management
Persistence
Correct Use
Cluster, Hash Slots and Moving Them Around
Cluster divides the keyspace into a fixed number of slots and assigns ranges of slots to nodes. Every behaviour in the rest of this lesson follows from that single decision, so it is worth understanding exactly what the slot is and why it exists.
Sixteen thousand three hundred and eighty-four
Every key belongs to one of 16384 slots, computed as the CRC16 of the key modulo 16384. The number is fixed at compile time and the same in every deployment, which means the mapping from a key to a slot is a pure function that any client can compute without asking anybody.
That property is what makes the design work. A client that knows which node owns which slots can route every request correctly with no lookup, no proxy and no coordination. The whole routing layer is a local computation plus a cached table of 16384 entries, which is small enough to hold trivially and cheap enough to consult on every command.
Why slots rather than hashing to nodes
Hashing keys directly to nodes means the mapping changes whenever the node count changes, and almost every key moves. Hashing to a fixed number of slots and then assigning slots to nodes means adding a node moves only the slots you choose to move, and nothing else is disturbed.
This is the standard indirection that every sharded system eventually adopts, and 16384 is a deliberate choice: large enough that slots can be distributed evenly across any practical number of nodes, small enough that a bitmap of slot ownership fits in two kilobytes and can be gossiped between nodes constantly. Redis caps the cluster size at 1000 nodes partly for this reason, which still leaves sixteen slots per node at the limit.
Slot ownership
Each slot has exactly one owning primary at any moment. A node's configuration includes the set of slots it owns, expressed as ranges, and every node in the cluster holds the full mapping for every slot, not just its own.
That shared map is what lets any node answer a misrouted request correctly rather than failing. Send a command for a slot a node does not own and it replies with a redirection naming the node that does, which it knows because it holds the whole map. The cluster is therefore self-describing from any node, and CLUSTER SLOTS or the newer CLUSTER SHARDS dumps that map in a form a client can consume directly.
The cluster bus
Nodes talk to each other on a second port, the client port plus 10000, so 16379 for a node serving 6379. The protocol there is binary and not the ordinary Redis protocol, and it carries gossip, failure detection, configuration updates and the slot map.
That extra port is a frequent deployment problem. A firewall or security group that opens only 6379 produces a cluster where clients work and nodes cannot see each other, which manifests as nodes permanently in a handshake state or as a cluster that never reaches an ok state. Opening both ports for every node pair is part of the setup, and the symptom when it is missed looks like a Redis problem rather than a network one.
What the gossip carries
Each node pings a few random other nodes every second, and the ping carries its own view: which nodes it knows, what state it believes each is in, which slots it owns, and its configuration epoch. Replies carry the same from the other side.
So the cluster converges on a shared view without any central coordinator, and a node that has been away learns the current topology from whoever it next talks to. The configuration epoch orders conflicting claims: if two nodes both claim a slot, the one with the higher epoch wins, which is the mechanism that resolves the aftermath of a failover without a negotiation.
Hash tags
A key containing braces hashes only the substring between the first { and the next }. So user:{1000}:profile and user:{1000}:sessions both hash on 1000 and therefore land in the same slot, on the same node.
This is the only tool Cluster gives you for keeping related keys together, and it is therefore the central design decision when modelling data for a cluster. It is also a loaded gun, which the next section explains.
What a slot is not
A slot is not a unit of storage, a unit of memory accounting or a unit of locking. It is purely a routing label. Keys in the same slot are not stored together in any special way and are not any faster to access together, except that they are guaranteed to be on the same node.
That guarantee is the entire value. Multi-key commands, transactions and scripts require every key to be in one slot, so the slot is the boundary of atomicity in a cluster, and it is a routing concept doing double duty as a consistency concept. Recognising that the slot is just a label explains why the restriction is what it is: Redis is not refusing a cross-slot operation out of caution, it is refusing because the keys are genuinely on different machines.
Enabling cluster mode
cluster-enabled yes turns an instance into a cluster node, which cannot be undone without a restart and which changes the instance's behaviour in several ways: only database 0 exists, SELECT is refused, and commands for unowned slots are redirected rather than served.
A node started in cluster mode with no slots assigned is idle but functional, which is how nodes join. Creating a cluster is therefore starting several such nodes, introducing them to each other, and assigning the slots, which redis-cli --cluster create does in one command. The point to remember is that cluster mode is a property of the instance decided at startup, not a mode the client opts into.
Primaries and replicas in a cluster
A cluster node is either a primary owning slots or a replica of a primary. Replication between them is the ordinary replication of the previous lessons, with the same handshake, backlog and offsets, so everything already covered applies unchanged.
What Cluster adds is that the failover decision is made by the primaries voting, rather than by a separate Sentinel process, which is why a cluster needs at least three primaries to function: a majority of primaries must be reachable to authorise anything. A three-primary cluster with no replicas has no failover capability at all, and a three-primary cluster with three replicas is the smallest genuinely available configuration, which is six nodes.
What to take from the model
Three things. The key-to-slot function is fixed and client-computable, which is why there is no proxy. Slot ownership is a shared map every node holds, which is why any node can redirect correctly. And the slot is a routing label that doubles as the atomicity boundary, which is the source of every restriction in the next section.
The rest of the lesson is routing, moving slots between nodes, adding and removing nodes, and operating the result.
The slot is purely a routing label, not a unit of storage, memory accounting or locking. Its only guarantee is that keys sharing a slot are on the same node, and that is why multi-key commands, transactions and scripts are restricted to one slot: Redis is not being cautious, the keys are genuinely on different machines.
Loading problem...