Streams arrived in Redis 5.0 and are the only type designed after the others had been in production for years, which shows: they address the things lists and pub/sub could not do, and they are the one type whose design anticipates consumers failing.

Entries and their identifiers

An entry is a set of field and value pairs, like a hash, appended to the stream with XADD and given an identifier. The identifier is two 64-bit integers written as <milliseconds>-<sequence>, where the milliseconds come from the server clock and the sequence distinguishes entries added within the same millisecond.

Passing * as the identifier asks the server to generate one, which is what almost all code should do. An explicit identifier is accepted, and the requirement is that it be strictly greater than the stream's current last identifier, so identifiers are monotonic by construction and a stream cannot be written out of order.

Why the identifier shape matters

That structure is the packing that the sorted sets lesson warned against attempting by hand, implemented properly: two separate 64-bit integers rather than a single double, so there is no precision limit, no collision, and no arithmetic to get wrong. A stream can take 18 quintillion entries in one millisecond before the sequence overflows.

It also means the identifier is a timestamp, so a range query over identifiers is a range query over time. XRANGE key 1698000000000 1698003600000 is an hour of entries, and since the sequence may be omitted, a bare millisecond value means the start of that millisecond as a lower bound and its end as an upper bound. That is a deliberate convenience and it is why range queries over time read naturally.

Reading ranges

XRANGE key start end returns entries in order, with - and + as the minimum and maximum identifiers, and an optional COUNT. XREVRANGE is the same reversed, taking its bounds in the opposite order, which is the usual trap. Since 6.2 a bound prefixed with an open parenthesis is exclusive, which is what makes paging work: fetching the next page as XRANGE key (lastId + COUNT 100 continues cleanly without re-reading the boundary entry.

Before exclusive bounds existed, paging meant incrementing the sequence number of the last identifier by hand, which worked and was ugly. Any code still doing that can be simplified, and any new code should use the exclusive form.

How a stream is stored

Entries are not stored one per allocation. A stream is a radix tree keyed by entry identifier, whose leaves are macro nodes, each a listpack holding many consecutive entries. The radix tree gives logarithmic lookup of a node by identifier, and the listpack packs the entries inside it compactly.

Two configuration values control the node size: stream-node-max-entries, defaulting to 100, and stream-node-max-bytes, defaulting to 4096. A node seals when either is reached and a new one starts. This arrangement is why a stream's memory per entry is low, in the region of tens of bytes of overhead rather than the hundred-ish a sorted set member costs, and it is also the reason for the deletion behaviour below.

Field names are repeated, and what to do about it

Within a macro node, the listpack stores entries with a small optimisation: when consecutive entries have the same field names, the names are stored once for the node and the subsequent entries hold only values. That makes a stream of uniformly shaped entries dramatically cheaper than one where each entry has different field names.

The practical advice follows directly. Keep the field set identical across entries of a stream, and keep field names short, because they are stored once per node rather than once per entry only when they match exactly. A stream where half the entries have an extra optional field loses the optimisation for every node containing both shapes, so a consistently present field with an empty value is cheaper than a sometimes-absent one.

Deletion, and why it is not what you expect

XDEL removes an entry, and the memory is not necessarily reclaimed. The entry is marked deleted within its macro node's listpack, and the node is freed only when every entry in it has been deleted, because rewriting a listpack to compact it on every deletion would be expensive.

So a stream with scattered deletions holds nodes that are mostly tombstones, and XLEN reports the live count while the memory reflects the nodes. This is the correct trade for a log, where deletion is meant to be rare, and it is a surprise to anybody treating a stream as a queue they delete from. The lesson is that a stream should be trimmed from the front, which frees whole nodes, rather than deleted from the middle.

Appending, and the options on XADD

XADD key * field value field value is the minimal form, and three options change its behaviour in ways worth knowing. NOMKSTREAM, from 6.2, refuses to create the stream if it does not already exist, which is what a producer should use when the stream is meant to be created deliberately by the consumer's XGROUP CREATE MKSTREAM rather than by whichever side happens to run first.

The trimming options, MAXLEN and MINID, append and trim in the same command, which is covered in detail later and is mentioned here because it is the right default and a producer written without it is a producer that depends on a trimming job existing.

An explicit identifier instead of * is for one case: replaying or migrating a stream, where the identifiers must be preserved. Outside that, generating identifiers client-side invites a clock disagreement between producers, which presents as writes rejected for being non-monotonic, and the rejection is the correct behaviour rather than a bug to work around.

How many streams, and how to name them

A stream is one key, so the granularity of a stream is a design decision made at naming time and awkward to change afterwards. One stream per event type gives consumers a natural filter and means a consumer interested in one type reads only that type; one stream for all events gives a single ordered history and makes consumers filter in application code.

The rule that serves most designs is one stream per bounded context that has its own retention and its own consumers, named with a stable prefix such as orders:events. Splitting finer than that multiplies the trimming, the monitoring and the consumer groups; splitting coarser than that makes every consumer read everything and makes one noisy event type the capacity constraint for all of them.

The commands that inspect

XLEN is the live entry count. XINFO STREAM key reports the length, the radix tree shape, the first and last entries, the last generated identifier and the number of consumer groups, and XINFO STREAM key FULL adds the entries and the full group state, which is the command to reach for when debugging.

XINFO GROUPS and XINFO CONSUMERS describe the consumer group machinery covered later. Together these are unusually good introspection by Redis standards, and the habit of running XINFO STREAM FULL before forming a theory saves a great deal of guessing.

Where a stream sits among the types

A stream is the only Redis type that is a log: entries are appended, retained, and read by many independent consumers without being consumed destructively. A list can be used as a queue, and reading from it removes the element. Pub/sub delivers to whoever is connected and retains nothing. A stream retains, which means a consumer can start late, re-read, or fall behind and catch up.

That retention is the whole point and it is also the cost: the stream grows until something trims it, and deciding what trims it is a design decision that cannot be deferred. A stream with no trimming policy is an unbounded memory commitment, and that is the single most common way a stream-based design fails in production.