Caching & Performance

Animated explainers on caching & performance.

All Caching & Performance Databases Kafka & Streaming Distributed Systems System Design Networking & Load Balancing ML & Agentic AI

May 17, 2026

Cache Miss Amplification: Why a 99% Hit Rate Can Still Take You Down

A hit saves one read. A miss can fan out into dozens. When the miss path is 10x more expensive than the hit path, even a tiny miss rate can saturate the origin and bring down the service.

Caching & Performance

May 12, 2026

Tail Latency Is What Your Users Actually Feel

Average latency tells a friendly story while p99 quietly burns the user experience. Here is why fan-out amplifies tails, where the slowness comes from, and how to fight back.

Caching & Performance

May 9, 2026

Logs, Metrics, Traces: What Each One Is Actually For

The three pillars of observability answer different questions and have very different cost curves. Mix them up and you either go blind or set your storage bill on fire.

Caching & Performance

May 8, 2026

How Distributed Tracing Actually Stitches a Request Together

A trace is just three IDs and a propagation rule. Here is how trace_id, span_id, and parent_id flow through HTTP headers, why one forgetful service breaks the whole picture, and how sampling keeps it affordable.

Caching & Performance

Apr 27, 2026

Viral Traffic: One Hot Object, A System-Wide Fan-Out

A viral post is not just more requests. It is a single key that suddenly dominates cache shards, replica reads, fan-out writes, and notification queues all at once.

Caching & Performance

Apr 24, 2026

How a Slow Database Takes Down an App That Is Otherwise Fine

Your database does not need to crash to crash your app. A single slow query can hold the connection pool hostage and turn one bad index into a full outage.

Caching & Performance

Apr 22, 2026

Pod Eviction and QoS: Why Kubernetes Picks Which Workloads Die First

When a node runs out of memory, the kubelet starts killing pods in a specific order. QoS class and resource requests decide who goes first.

Caching & Performance

Apr 19, 2026

HPA vs Cluster Autoscaler: Two Loops, One Traffic Spike

HPA scales pods. Cluster Autoscaler scales nodes. They run in series, and the handoff between them is where most scaling outages happen.

Caching & Performance

Apr 13, 2026

How Metrics, Traces, and Logs Compose Into One Investigation

Metrics tell you something broke. Traces narrow which hop is slow. Logs explain the why. The order matters, and so does the cardinality discipline that keeps any of them affordable.

Caching & Performance

Apr 11, 2026

One Request, Three Layers: Where Work Actually Belongs

A user sees one click. Your system sees a fanout across edge, service, and data layers. Putting work at the wrong layer is how outages start.

Caching & Performance

Apr 5, 2026

One Page Load, Fifty Backend Calls: The Tail Latency Math of Synchronous Fan-Out

A single user request becomes a tree of internal RPCs. The wider the tree, the more your p99 of one dependency becomes the median of the whole request. Here is the math and the fixes.

Caching & Performance

Mar 31, 2026

TCP Congestion Control: Why Packet Loss Is the Whole Conversation

Slow start, AIMD, CUBIC, and BBR. Classic TCP uses packet loss as its only signal, and on a lossy long-haul link that assumption falls apart.

Caching & Performance

Mar 30, 2026

The TCP Handshake and Teardown: Where Your Sockets Actually Go to Die

Three-way handshake, four-way teardown, TIME_WAIT, and the 2MSL wait. The state machine matters because exhausted ephemeral ports look like network failures.

Caching & Performance

Mar 30, 2026

Head-of-Line Blocking: Why HTTP/3 Switched From TCP to QUIC

HTTP/1.1, HTTP/2, and HTTP/3 each handle head-of-line blocking differently. Why HTTP/3 had to abandon TCP and rebuild on UDP with independent QUIC streams.

Caching & Performance

Mar 29, 2026

CDN Cache Key Design: The Knob That Decides Your Hit Rate

Your CDN hit rate is downstream of one design decision: what goes into the cache key. Path, query, headers, and device class all fragment or unify the cache.

Caching & Performance

Mar 27, 2026

Cache Stampedes: When One Expired Key Takes Down Your Database

A hot key expires, a thousand requests miss in parallel, and they all rebuild the same value at the same time. The cure is single-flight locks, early refresh, and stale-while-revalidate.

Caching & Performance

Mar 27, 2026

Hot Keys: When Consistent Hashing Stops Being Fair

Consistent hashing spreads keys evenly. Real traffic follows a power law, so one celebrity key can pin a single shard at 100 percent CPU while the rest of the cluster naps.

Caching & Performance

Mar 17, 2026

Health Checks That Don't Take Down Your Whole Fleet

A bad health check is worse than no health check. Here is how active and passive probes work together, and why deep checks can turn a hiccup into an outage.

Caching & Performance

Mar 17, 2026

Load Balancer Health Checks: The Difference Between Healthy and Ready

A TCP probe tells you the port is open. It does not tell you the app can serve traffic. Good health checks combine active probes, passive signals, and dependency-aware readiness.

Caching & Performance

Mar 16, 2026

HTTP Cache Headers: The Freshness Control Panel

Cache-Control, ETag, and Vary decide who stores your response, how long it stays fresh, and when it must revalidate. Get them wrong and your CDN serves yesterday's data for a year.

Caching & Performance

Mar 7, 2026

Cache TTL Tuning: The Knob That Sits Between Hit Rate and Staleness

TTL is not a default value, it is a system design decision. Per resource TTLs, randomized jitter, and write through invalidation pick the right point on the freshness curve.

Caching & Performance

Mar 6, 2026

TLS Handshake and Session Resumption: Why Your P99 Climbs After Every Deploy

TLS 1.2 needs two round trips, TLS 1.3 needs one, and 0-RTT resumption needs none. Session tickets only help if every load balancer shares the same key.

Caching & Performance

Mar 6, 2026

DNS Resolution in the Browser and Beyond: When TTL is a Suggestion

DNS resolution touches browser cache, OS resolver, recursive resolver, root, TLD, and authoritative servers. TTLs govern caching at each step, except when a resolver ignores them.

Caching & Performance

Mar 3, 2026

Tail Latency and Hedged Requests: Why p50 Looks Fine While Users Complain

p50 says the system is healthy. Users say it feels random. The gap is the tail, and the fix is a careful one because the obvious mitigation can finish the outage.

Caching & Performance

Feb 23, 2026

Negative Caching: When Caching Nothing Saves the Database

Caching the absence of a value protects the database from invalid-key floods. Here is why short TTLs alone are not enough, and how a long negative TTL can hide a new record from production for hours.

Caching & Performance

Feb 22, 2026

Concurrency Control Under Load: When Adding Threads Makes Things Slower

More threads and more replicas feel like the safe response to a slow service. Little's Law and queue depth tell a different story: smaller pools with fast rejection often beat larger pools that queue quietly.

Caching & Performance

Feb 22, 2026

Redis Connection Pooling Is a Correctness Problem, Not a Performance One

A connection per request kills throughput and exhausts file descriptors. A pool sized to peak request rate hides the bug. The right size tracks peak concurrency, with validation on borrow.

Caching & Performance

Feb 15, 2026

Hot Partition Mitigation: When One Key Brings the Cluster to Its Knees

Salting, pre-splitting, time-bucketing, and a cache tier. Each fixes a different kind of hot partition. Picking the wrong one trades one bottleneck for another.

Caching & Performance

Feb 6, 2026

Write Amplification: The Hidden Multiplier That Kills SSDs Early

Write amplification is the ratio of bytes the disk writes to bytes your app writes. B-trees sit around 4 to 6. LSM trees commonly hit 10 to 30. SSDs add another 2 to 3x on top. If you do not measure it, your endurance budget evaporates.

Caching & Performance

Jan 31, 2026

Hot Partitions and Hot Keys: The Tax You Pay for Going Viral

One celebrity user can melt a single shard while the rest of the cluster sits idle. Here is how to detect hot keys early and the four fixes that actually work in production.

Caching & Performance

Jan 27, 2026

The Cache Hierarchy: Performance Is Which Layer You Actually Hit

L1 is 1ns, L2 is 4ns, L3 is 10ns, RAM is 100ns, local Redis is 1ms, remote Redis is 5ms, a database call is 50ms, cross-region is over 100ms. Every layer is 10 to 100x slower than the one above. Optimize at the layer that moves the metric.

Caching & Performance

Jan 21, 2026

Why B-Trees and LSM Trees Both Exist: The Disk Physics Behind the Split

B-trees and LSM trees solve different problems forced on us by disk physics. Random seek cost, sequential bandwidth, and why spinning platters and SSDs picked different winners.

Caching & Performance

Jan 11, 2026

Redis Eviction Policies: Eight Choices, and Only One Is Right for Your Workload

noeviction, allkeys-lru, allkeys-lfu, volatile-lru, volatile-lfu, volatile-ttl, allkeys-random, volatile-random. Each picks a different victim. Mixing cache and session data in one instance with the wrong policy logs your users out.

Caching & Performance

Jan 10, 2026

Cache Stampede: When TTL Expiry Becomes a Self-Inflicted DDoS

A single key's TTL ticks to zero and a thousand concurrent requests slam the database at the same instant. Here is how singleflight, stale-while-revalidate, and jitter keep that from becoming an outage.

Caching & Performance

Jan 9, 2026

Synchronized TTL Is a Time Bomb You Lit at Deploy Time

When a thousand keys all expire on the same second, every dependent request misses at once. The origin sees a synchronized cliff, not a smooth load curve. Jitter is the cheap fix.

Caching & Performance

Jan 5, 2026

Redis Is a Toolbox, Not a String Store: Pick the Structure That Fits the Access Pattern

Sorted sets, hashes, bitmaps, HyperLogLog, GEO, and lists each map to a different system design pattern. Choosing wrong is not a style preference, it is a CPU and memory bill you pay every request.

Caching & Performance

Dec 23, 2025

Traffic Spikes Are Not a Capacity Problem

More servers does not save you from a flash sale or a viral post. Spikes break the shape of the load, not its size. Front-door limits, async paths, and load leveling are what actually hold the line.

Caching & Performance

Dec 3, 2025

Anatomy of a Trading Order: What Place Order Actually Does

From API call to confirmed fill, an order touches risk checks, an OMS, a matching engine, market data fan-out, and settlement. Every hop has a sub-millisecond budget and a way to ruin it.

Caching & Performance

Dec 12, 2023

API Gateway: One Chokepoint, One Policy Plane, One Set of Tradeoffs

An API Gateway is not a router with extra features. It is the one place where every request meets the same policies. That is both the value and the cost.