Redis Streams Trimming: The Tilde in MAXLEN Is Not Free

Redis Streams grow forever unless you trim them. Most teams reach for XADD MAXLEN ~ N because it shows up in every tutorial and because it is much cheaper than the exact form. The tilde is the catch. It means Redis will trim toward N when it can do so by dropping a whole radix tree node, and it will skip trimming when the work would cross a node boundary. The result is that a stream told MAXLEN ~ 100000 routinely sits anywhere between 100K and roughly 200K entries, depending on how the underlying nodes line up.

The exact form, XADD MAXLEN = N, gives you a hard ceiling at the price of trimming work on every write. XTRIM cleans up after the fact and can be run from a janitor process, which is useful if you do not want trim cost on the producer path. MINID is the time-aware sibling: trim everything older than a given stream ID, which usually maps to a wall-clock window. None of these are interchangeable. They each pick a different thing to be sloppy about.

The production failure I keep seeing involves a team that picked MAXLEN ~ 100K for an order-events stream and assumed memory was bounded. Their normal throughput sat at twenty thousand writes a minute and the stream stayed near the cap. Then a partner integration ran a backfill and their producer burst to two hundred thousand writes a minute. The approximation held about 200K entries instead of 100K, the instance crossed maxmemory, and allkeys-lru started evicting whichever keys were coldest. The coldest keys were old stream entries the slow consumer had read but not yet acknowledged. The PEL still referenced IDs that no longer existed in the stream. When the consumer recovered and tried to XACK, the work was simply gone. Redis returned no error. Downstream order confirmations vanished.

The fix had two parts. Streams that carry work the business cannot lose were moved to a dedicated Redis with maxmemory-policy noeviction, so backpressure surfaces as a write failure the producer must handle instead of as silent eviction. Within that instance, critical streams switched to MAXLEN = N for hard count bounds, and the audit stream switched to MINID so retention tracks elapsed time instead of message count. The tilde is fine for metrics streams nobody will miss. It is not fine for anything you would page on.

Redis will do exactly what you asked. The trick is knowing what you actually asked for.