Negative Caching: When Caching Nothing Saves the Database

Most cache discussions assume the data exists. Negative caching is about what to do when it does not. A client asks for user 42, the database returns nothing, and the cache writes null under that key. The next thousand requests for user 42 never touch the database. That is the win.

The reason this matters is not theoretical. Any public endpoint that takes an id from the URL is a probe surface. Scanners walk integer ranges. Stale links reference deleted records. A bad client retries a missing id in a loop. Without negative caching, every miss is a database query. With it, the first miss is a query and the next ten thousand are constant-time lookups in Redis. The database stays warm and the bad traffic costs you almost nothing.

The trade is that the cache now lies in a new direction. Before, a miss meant "go ask the database, it has the truth." With negative caching, a hit can mean "the database said no five minutes ago, here is that answer." If the data changed between then and now, you are serving wrong information confidently.

The failure mode looks like this. A user signs up at 10:00:00, and the signup flow inserts a row in users. The recommendation service probed for that user id at 09:59:50, missed, and cached null with a one-hour TTL. For the next hour, every request through the recommendation cache sees "user does not exist," even though the row is right there. The user files a support ticket. The engineer queries the database, sees the row, and cannot reproduce because they bypassed the cache. The TTL expires, and the system heals itself with no commit attached.

Two mitigations, used together.

Short TTLs on negative entries. The cost of a re-miss is one database query, and the cost of staleness is user-visible. The asymmetry favors a much shorter TTL than your positive entries. A typical setup uses positive TTLs of five to thirty minutes and negative TTLs of thirty to ninety seconds.

Event-driven invalidation. On every write that could resolve a previous miss, publish an invalidation. The signup flow deletes cache:user:42 in the same transaction that inserts the row. This is the only mitigation that brings the staleness window to zero, and the only one that works when the negative TTL has to be long.

A bloom filter in front of the cache is a third option at very high read volumes. It answers "definitely not in the database" with zero database load and no staleness, because new writes update it at insert time. It costs memory and accepts a small false-positive rate.

Treat the absence of a value as a first-class cache entry, give it a TTL short enough to forgive itself, and invalidate it on the write that proves it wrong.