0%
Redis Deep Dive: Encodings, Memory, and Failover
The Server Model
Data Structures Inside
Memory Management
Persistence
Replication and Cluster
Scripts and Functions, Logic That Runs Beside the Data
A script is the answer to almost every limitation of the previous lesson. It runs on the server, with the data in front of it, as one atomic unit, which makes read-then-decide a single round trip with no retries.
EVAL and the two argument lists
EVAL script numkeys key [key ...] arg [arg ...] runs a Lua body. The numkeys argument splits the remaining arguments into the keys the script will touch, available as KEYS, and everything else, available as ARGV.
Both tables are one-indexed, as Lua tables are, so the first key is KEYS[1]. The split looks like a formality and is not: it is how the server knows which keys a script touches, which matters for cluster routing and for any proxy or tooling that reasons about keys. A script that constructs key names from ARGV instead of declaring them in KEYS works on a standalone instance and is wrong.
redis.call and redis.pcall
Inside the script, redis.call('GET', KEYS[1]) runs a Redis command and returns its reply converted to a Lua value. redis.call raises a Lua error if the command errors, which aborts the script and returns the error to the client. redis.pcall returns the error as a value instead.
So redis.call is the right default, because an unexpected command error should abort rather than be ignored, and pcall is for the cases where an error is an expected outcome to be handled. The distinction matters because a script that swallows errors with pcall and carries on writing can leave exactly the partial state transactions are criticised for.
Atomicity, and where it comes from
A script runs as a single unit with nothing interleaved, for the same reason a transaction does: the server executes one thing at a time, and a script is one thing. No other client's command runs during it, and no partial state is visible.
This is stronger than a transaction in the way that matters, because the script can read, branch and write within that unit. The whole difficulty of the previous lesson, that no queued command yields a value, disappears: inside a script every redis.call returns immediately and the next line can act on it. That is the entire reason scripts exist.
The script cache and EVALSHA
Every script passed to EVAL is cached on the server under the SHA1 of its body, for the lifetime of the process. EVALSHA sha numkeys ... runs a cached script by digest, which avoids sending the body on every call.
For a script of any size this is a real saving, since a few hundred bytes of Lua on every invocation is bandwidth and parsing that buys nothing. Most client libraries handle it automatically: they compute the digest, try EVALSHA, and fall back to EVAL if the server replies NOSCRIPT. That fallback is essential and the next section on operations explains when it fires.
SCRIPT LOAD, EXISTS and FLUSH
SCRIPT LOAD caches a script and returns its digest without running it, which is the clean way to register scripts at application startup. SCRIPT EXISTS checks one or more digests. SCRIPT FLUSH empties the cache entirely.
The cache is not persisted and is not replicated, so every restart, and every new replica, starts empty. That is the single most important operational fact about EVALSHA: the digest your application holds is valid only as long as that specific server process has seen the body, which is why the NOSCRIPT fallback is not optional and why SCRIPT LOAD at startup is a convenience rather than a guarantee.
How values convert between Lua and Redis
Redis replies become Lua values: a string stays a string, an integer becomes a number, an array becomes a one-indexed table, a nil reply becomes false, and a status reply becomes a table with an ok field. Going the other way, Lua numbers are truncated to integers, false becomes nil, true becomes 1, and a table becomes an array that stops at the first nil.
Two of those are traps. Numbers being truncated means returning 3.7 gives the client 3, so any fractional value must be returned as a string. And a table stopping at the first nil means a sparse table returns only its leading run, which silently truncates results. Both are easy to hit and neither produces an error.
Returning errors and status replies deliberately
return redis.error_reply('message') returns a proper Redis error to the client, and return redis.status_reply('OK') returns a status reply. Returning a plain string returns a bulk string, which is usually what you want for data.
Using error_reply for an expected rejection, rather than returning a sentinel value, is a judgement call. An error makes the client library raise, which is appropriate when the rejection is exceptional, and awkward when it is routine. For something like a rate limiter where rejection is the common case, returning a number is kinder to the calling code than raising on most invocations.
What is in the sandbox
The Lua environment includes cjson for JSON, cmsgpack for MessagePack, struct for binary packing, bit for bitwise operations, redis.sha1hex, and redis.breakpoint and redis.debug for the debugger. The standard library is present apart from what was deliberately removed.
What is absent is everything that would reach outside: no io, no os beyond a couple of harmless functions, no sockets, no require and no loading of external modules. A script cannot call another service, read a file or consult the clock in a way that would make it non-deterministic. Those removals are what make the sandbox safe to run on a server that cannot afford surprises.
Scripts and the single thread, restated
Because a script holds the server for its whole duration, its cost is borne by every other client. A script doing a hundred redis.call invocations is a hundred commands' worth of pause for everybody, and a script with a loop over a large collection can be far worse.
This is the central operational constraint and the whole of the fourth section. The mental model to carry is that a script is one command as far as the server's scheduling is concerned, so the discipline that applies to a single command applies to the whole script: it should be fast, bounded, and free of anything that scales with the size of the data.
A first complete example
A conditional decrement, the stock problem from the previous lesson, in four lines: read the key with redis.call('GET', KEYS[1]), treat a missing key as zero with tonumber(v) or 0, return -1 if it is at or below zero, and otherwise return redis.call('DECR', KEYS[1]).
That is the whole thing, and it is worth comparing against the WATCH version: one round trip against at least two, no retry loop, no backoff, no livelock under contention, and the logic in one place rather than split between a read and a conditional write. The script is also shorter than the client code it replaces, which is not always true and is pleasant when it is.
What to take from execution
Three things. A script is atomic in the strong sense and can read, branch and write inside that atomicity, which is what a transaction cannot do. The KEYS declaration is load-bearing rather than decorative, because the server uses it for routing. And the script cache is per process, unreplicated and not persisted, so EVALSHA must always have an EVAL fallback.
The script cache lives in one server process's memory, is never replicated and never persisted. A restart, a failover to a replica or a newly added node all start with an empty cache, so a digest your application has held for months can suddenly return NOSCRIPT. Every EVALSHA call needs an EVAL fallback, and SCRIPT LOAD at startup is a convenience rather than a guarantee.