Archive

Short, animated explainers on the systems that power the internet. Caching, databases, Kafka, distributed consensus, load balancers, and the LLM infrastructure underneath modern AI products.

All Caching & Performance Databases Kafka & Streaming Distributed Systems System Design Networking & Load Balancing ML & Agentic AI

System Design

May 18, 2026

The Complexity of Simple Writes

A user's single write action may trigger multiple downstream operations in a system, complicating consistency and overall system design.

Caching & Performance

May 17, 2026

Cache Miss Amplification: Why a 99% Hit Rate Can Still Take You Down

A hit saves one read. A miss can fan out into dozens. When the miss path is 10x more expensive than the hit path, even a tiny miss rate can saturate the origin and bring down the service.

Databases

May 16, 2026

Distributed Query Execution: One Slow Shard Owns Your P99

A SELECT across shards becomes a fanout plan with parallel scans, shuffles, and a coordinator merge. Tail latency is dominated by the slowest shard, and cross-shard joins quietly break linear scaling.

Distributed Systems

May 14, 2026

Quorum Writes: How Distributed Systems Trade Latency for Consistency

Why waiting for every replica is a mistake. Quorum writes use the W + R > N rule to balance latency, availability, and consistency in distributed databases.

Databases

May 13, 2026

Why Your Database Choice Shapes Every Other Design Decision

Caching, queueing, replication, partitioning: every system design choice bends to your database. Here is why databases shape everything downstream of them.

Caching & Performance

May 12, 2026

Tail Latency Is What Your Users Actually Feel

Average latency tells a friendly story while p99 quietly burns the user experience. Here is why fan-out amplifies tails, where the slowness comes from, and how to fight back.

System Design

May 11, 2026

Analytics Event Fan-Out: One Click, Seven Consumers, A Hundred Edge Cases

One user action becomes one event becomes seven downstream pipelines. The cost of that fan-out is schema evolution, lag, dedup, and per-consumer ordering.

Distributed Systems

May 10, 2026

Three Kubernetes Probes, Three Different Jobs

Startup, readiness, and liveness probes look similar and do completely different things. Mixing them up is how a flaky dependency turns into a fleet-wide restart loop.

Caching & Performance

May 9, 2026

Logs, Metrics, Traces: What Each One Is Actually For

The three pillars of observability answer different questions and have very different cost curves. Mix them up and you either go blind or set your storage bill on fire.

Caching & Performance

May 8, 2026

How Distributed Tracing Actually Stitches a Request Together

A trace is just three IDs and a propagation rule. Here is how trace_id, span_id, and parent_id flow through HTTP headers, why one forgetful service breaks the whole picture, and how sampling keeps it affordable.

Kafka & Streaming

May 7, 2026

Kafka Consumer Group Rebalancing: How Partitions Get Reassigned

Kafka consumer groups stay running even when one consumer crashes. Here is how the group coordinator rebalances partitions and keeps the stream flowing.

Kafka & Streaming

May 6, 2026

How Kafka Replicates Data and Elects a New Leader When a Broker Dies

Kafka survives broker failures through partition replication and ISR-based leader election. Here is how the High Watermark and the controller keep data safe.

Kafka & Streaming

May 6, 2026

Kafka In-Sync Replicas Deep Dive: When ISR Shrinks and Writes Quietly Stop

The ISR is not a static list. It expands and contracts based on replica.lag.time.max.ms, and when it falls below min.insync.replicas your producers stop writing.

Kafka & Streaming

May 5, 2026

Kafka Exactly-Once Semantics: The Three Pieces You Actually Need

Kafka EOS is not one feature. It is the idempotent producer, transactions, and `read_committed` consumers working together across the full consume-process-produce loop.

Databases

May 4, 2026

Event Sourcing: Store the Events, Derive the State

Event sourcing turns your database into an append-only log of facts. State becomes a function of history. The cost is schema evolution forever and replay times that grow with your business.

Kafka & Streaming

May 3, 2026

CQRS with Async Projection: Two Models, One Truth

CQRS splits the write model from the read models and lets each be optimized in isolation. Async projections introduce lag, multiple read shapes, and a new operational concern: projection drift.

Kafka & Streaming

May 2, 2026

MapReduce Explained: Why a 20-Year-Old Idea Still Runs Your Batch Jobs

MapReduce is map, shuffle, reduce. Three phases, two functions, and a scheduler that makes embarrassingly parallel work cheap. Here is why Spark replaced it and where it still wins.

Kafka & Streaming

May 1, 2026

Tumbling vs Sliding vs Session Windows: Picking the Right Shape for Stream Aggregates

Tumbling windows give you clean billing buckets. Sliding windows give you smooth anomaly detection. Session windows give you user behavior. Pick by what question you are asking.

Kafka & Streaming

Apr 30, 2026

Exactly-Once Stream Processing: How Checkpoints and Idempotency Work Together

Exactly-once stream processing is not one feature. It is checkpoints plus idempotent sinks. Here is why you need both, and what breaks if you skip either.

Databases

Apr 29, 2026

The Six Database Questions Every System Design Eventually Asks

Writes, reads, indexing, transactions, replication, partitioning. The technology stack changes every five years, but the six core database questions never do.

System Design

Apr 28, 2026

Gray Failures: When the System Is Up and Broken at the Same Time

A node that returns 200s slowly is more dangerous than one that is fully down. Gray failures pass health checks, fool load balancers, and quietly poison everything that depends on them.

Caching & Performance

Apr 27, 2026

Viral Traffic: One Hot Object, A System-Wide Fan-Out

A viral post is not just more requests. It is a single key that suddenly dominates cache shards, replica reads, fan-out writes, and notification queues all at once.

Caching & Performance

Apr 24, 2026

How a Slow Database Takes Down an App That Is Otherwise Fine

Your database does not need to crash to crash your app. A single slow query can hold the connection pool hostage and turn one bad index into a full outage.

Caching & Performance

Apr 22, 2026

Pod Eviction and QoS: Why Kubernetes Picks Which Workloads Die First

When a node runs out of memory, the kubelet starts killing pods in a specific order. QoS class and resource requests decide who goes first.

Distributed Systems

Apr 21, 2026

Kubernetes Ingress: One Door, Many Rooms, And the Controller That Actually Opens It

Ingress is the HTTP router at the cluster edge. The resource is just YAML. The Ingress Controller is what does the work. Get the class label wrong and nothing picks it up.

Distributed Systems

Apr 20, 2026

How Kubernetes Service Discovery Hides a Lot of Moving Parts Behind One DNS Name

A pod talks to payments.default.svc.cluster.local and it just works. Here is the CoreDNS, Service, and Endpoints machinery hiding behind that one stable name.

Caching & Performance

Apr 19, 2026

HPA vs Cluster Autoscaler: Two Loops, One Traffic Spike

HPA scales pods. Cluster Autoscaler scales nodes. They run in series, and the handoff between them is where most scaling outages happen.

Distributed Systems

Apr 17, 2026

Three Replicas, One Node, Zero Availability: Why HA in Kubernetes Is About Placement

Replica count alone does not buy you HA. Where the replicas land does. Anti-affinity, topology spread, and PodDisruptionBudgets are the controls that turn three pods into actual redundancy.

Distributed Systems

Apr 15, 2026

What Actually Happens When a Kubernetes Node Dies

Automatic recovery sounds fast. In reality, a node failure costs you seconds-to-minutes of unavailability per pod, and the default timers explain why.

Distributed Systems

Apr 14, 2026

The Kubernetes Service Is the Whole Reason Pods Can Be Cattle

The Service object looks like a small piece of YAML, but it is the abstraction that lets pods churn freely without breaking every caller. Here is what it actually hides.

Caching & Performance

Apr 13, 2026

How Metrics, Traces, and Logs Compose Into One Investigation

Metrics tell you something broke. Traces narrow which hop is slow. Logs explain the why. The order matters, and so does the cardinality discipline that keeps any of them affordable.

Databases

Apr 12, 2026

Why One User Write Turns Into a Graph of Side Effects in Modern Backends

A single Place Order request looks like one write. In production it fans out into a graph of cache invalidations, index updates, events, and audit logs.

Caching & Performance

Apr 11, 2026

One Request, Three Layers: Where Work Actually Belongs

A user sees one click. Your system sees a fanout across edge, service, and data layers. Putting work at the wrong layer is how outages start.

Distributed Systems

Apr 10, 2026

The Rolling Deploy That Returns 5xx: Why Readiness Probes Decide When Traffic Moves

A rolling update is not really about creating and deleting pods. It is about deciding when traffic shifts. Readiness probes are the gate, and getting them wrong burns users every release.

Distributed Systems

Apr 9, 2026

Pods Are Cattle: How Kubernetes Self-Healing Actually Works

Kubernetes does not restart dead pods. It creates new ones to satisfy a desired replica count. That distinction is the whole reason StatefulSets exist.

System Design

Apr 9, 2026

When Failover Is What Actually Causes the Outage

A region degrades, traffic shifts away, and the healthy regions collapse under the surge. Multi-region resilience is not about losing a region. It is about whether the survivors can carry the traffic.

Databases

Apr 7, 2026

Pick the Database That Matches the Workload, Not the One on Your Resume

OLTP, OLAP, key value, document, time series, columnar, search. Each storage engine is optimized for a specific access pattern. Match the workload first, the brand name second.

Distributed Systems

Apr 7, 2026

Circuit Breaker Pattern Deep Dive: The Half-Open State Is Where the Bugs Live

A circuit breaker does not heal a failing dependency, it contains the blast radius. The closed, open, and half-open states each have their own failure modes worth knowing.

Kafka & Streaming

Apr 5, 2026

Kafka Consumer Lag: When Your System Is Up but No Longer Real Time

Consumer lag is the gap between the log-end offset and the committed offset. It is a per-partition metric, and the worst kind of incident: not down, just slow.

Caching & Performance

Apr 5, 2026

One Page Load, Fifty Backend Calls: The Tail Latency Math of Synchronous Fan-Out

A single user request becomes a tree of internal RPCs. The wider the tree, the more your p99 of one dependency becomes the median of the whole request. Here is the math and the fixes.

Kafka & Streaming

Apr 5, 2026

Backpressure: How Healthy Systems Push Back Before Their Queues Eat Them

Systems rarely die from being busy. They die from accepting work they cannot finish. Backpressure is the mechanism that makes overload visible early.

Distributed Systems

Apr 4, 2026

Why Retries Need Idempotency: The Real Fix for Duplicate Payments and Orders

Retries recover from failures, but they can also duplicate them. Idempotency keys are what turn a dangerous retry into a safe one across payments and orders.

Databases

Apr 4, 2026

Read After Write: When Your Own Update Vanishes on Refresh

Async replication scales reads but lets users see their own writes disappear. Read-your-writes, monotonic reads, session pinning, and the failures that follow.

System Design

Apr 3, 2026

Cascading Timeout Failures: Why Inverted Deadlines Multiply Load Under Stress

When timeouts grow as you go deeper into the call graph, slow downstreams keep doing work the client gave up on. Deadline propagation is the fix.

ML & Agentic AI

Apr 2, 2026

Inside a Coding Agent: The Loop That Actually Matters

Coding agents are not better prompts. They are a tight loop: the model proposes a tool call, the environment runs it, the result lands in context, the model decides what to do next.

Networking & Load Balancing

Apr 2, 2026

Load Balancing Fundamentals: L4, L7, and Why Round-Robin Lies

Round-robin assumes every backend is identical. In a mixed-instance Kubernetes cluster it is not, and equal request counts produce unequal CPU pressure.

Networking & Load Balancing

Apr 1, 2026

REST vs gRPC Streaming: When Persistent Connections Help and When They Hurt

gRPC streaming sounds like a free win over REST. It is not. Long-lived streams interact badly with load balancers, deploys, and connection draining.

Caching & Performance

Mar 31, 2026

TCP Congestion Control: Why Packet Loss Is the Whole Conversation

Slow start, AIMD, CUBIC, and BBR. Classic TCP uses packet loss as its only signal, and on a lossy long-haul link that assumption falls apart.

Caching & Performance

Mar 30, 2026

The TCP Handshake and Teardown: Where Your Sockets Actually Go to Die

Three-way handshake, four-way teardown, TIME_WAIT, and the 2MSL wait. The state machine matters because exhausted ephemeral ports look like network failures.

Caching & Performance

Mar 30, 2026

Head-of-Line Blocking: Why HTTP/3 Switched From TCP to QUIC

HTTP/1.1, HTTP/2, and HTTP/3 each handle head-of-line blocking differently. Why HTTP/3 had to abandon TCP and rebuild on UDP with independent QUIC streams.

Caching & Performance

Mar 29, 2026

CDN Cache Key Design: The Knob That Decides Your Hit Rate

Your CDN hit rate is downstream of one design decision: what goes into the cache key. Path, query, headers, and device class all fragment or unify the cache.

Caching & Performance

Mar 27, 2026

Cache Stampedes: When One Expired Key Takes Down Your Database

A hot key expires, a thousand requests miss in parallel, and they all rebuild the same value at the same time. The cure is single-flight locks, early refresh, and stale-while-revalidate.

Caching & Performance

Mar 27, 2026

Hot Keys: When Consistent Hashing Stops Being Fair

Consistent hashing spreads keys evenly. Real traffic follows a power law, so one celebrity key can pin a single shard at 100 percent CPU while the rest of the cluster naps.

Databases

Mar 26, 2026

Inverted Indexes: How Full-Text Search Stays Fast at a Billion Documents

An inverted index flips the problem: instead of scanning documents for terms, you look up terms and get the documents. That is the whole trick behind millisecond search.

Databases

Mar 25, 2026

Multi-Leader Replication: The Conflict Problem Nobody Wants to Own

Multi-leader cuts cross-region write latency, then hands you conflicts. LWW silently loses data, app-level merge takes work, and CRDTs only fit certain shapes.

Databases

Mar 22, 2026

Leader Follower Replication: The Default That Hides a Cliff

Leader follower replication is the boring default for Postgres, MySQL, and MongoDB. Sync vs async, read scaling, failover, and the lag-driven data loss waiting in production.

Databases

Mar 21, 2026

Multi-Tenancy: Three Isolation Models and the Tenant That Stalls Them All

Shared schema, schema per tenant, database per tenant. Each model trades operational cost for blast radius. Row-level security alone does not stop the noisy neighbor.

Databases

Mar 19, 2026

Time-Series Databases: Why a Generic Row Store Falls Over at Metric Scale

Time-series data is append-only, write-heavy, and queried by time range. That access pattern demands columnar layout, delta-encoded timestamps, and downsampling, not a B-tree.

Networking & Load Balancing

Mar 18, 2026

L4 vs L7 Load Balancing: When Speed Beats Smartness

Layer 4 forwards packets blind and fast. Layer 7 inspects each request and routes by content. Here is how to choose, and why most production systems use both.

Caching & Performance

Mar 17, 2026

Health Checks That Don't Take Down Your Whole Fleet

A bad health check is worse than no health check. Here is how active and passive probes work together, and why deep checks can turn a hiccup into an outage.

Caching & Performance

Mar 17, 2026

Load Balancer Health Checks: The Difference Between Healthy and Ready

A TCP probe tells you the port is open. It does not tell you the app can serve traffic. Good health checks combine active probes, passive signals, and dependency-aware readiness.

Caching & Performance

Mar 16, 2026

HTTP Cache Headers: The Freshness Control Panel

Cache-Control, ETag, and Vary decide who stores your response, how long it stays fresh, and when it must revalidate. Get them wrong and your CDN serves yesterday's data for a year.

System Design

Mar 15, 2026

Token Bucket Rate Limiting: Bursts, Refills, and the Redis Trap

Token bucket allows bursts up to B and a sustained rate of R. Centralizing the bucket in Redis makes every API call a network hop, which scales until it doesn't.

Networking & Load Balancing

Mar 14, 2026

HTTP/1.1 vs HTTP/2 vs HTTP/3: A Story About Removing Different Kinds of Blocking

Each HTTP version fixed the head-of-line blocking the previous one could not. HTTP/2 multiplexed at the app layer, HTTP/3 moved to QUIC to fix it at the transport layer.

Distributed Systems

Mar 11, 2026

IP Routing and Longest Prefix Match: Why the Most Specific Route Always Wins

Routing tables are tries, not hash maps. The longest matching prefix wins because specificity equals authority, and BGP propagates that authority across the internet.

Distributed Systems

Mar 11, 2026

ARP and the L2/L3 Boundary: Why Your VIP Failover Sometimes Takes Four Minutes

ARP bridges IP addresses to MAC addresses on the local subnet. Gratuitous ARP is how failover tells the LAN about a new owner, except when a switch decides to ignore it.

Networking & Load Balancing

Mar 10, 2026

Why Backend Engineers Need the OSI Model: Naming the Layer Saves Hours

The OSI model is not trivia. Each layer corresponds to a class of failure mode, and naming the layer first turns guesswork into a directed search.

Kafka & Streaming

Mar 9, 2026

Finding the Top-K Heavy Hitters in an Unbounded Stream

Count-Min Sketch plus a min-heap solves Top-K for unbounded streams. Here is how the approximation works and where the overestimates actually bite you.

Databases

Mar 8, 2026

MongoDB: When to Embed, When to Reference

Embed for one-to-few, bounded, read-together data. Reference for shared or unbounded data. The 16MB document limit and write hotspots decide the rest.

Caching & Performance

Mar 7, 2026

Cache TTL Tuning: The Knob That Sits Between Hit Rate and Staleness

TTL is not a default value, it is a system design decision. Per resource TTLs, randomized jitter, and write through invalidation pick the right point on the freshness curve.

ML & Agentic AI

Mar 6, 2026

Search Relevance vs Latency: Why the Best Ranker Can Tank Your Conversion

Search is two stages: retrieval and ranking. Each has a latency budget, and a 4% NDCG win can still cost you 7% of conversions if you spend it all on a slow reranker.

Caching & Performance

Mar 6, 2026

TLS Handshake and Session Resumption: Why Your P99 Climbs After Every Deploy

TLS 1.2 needs two round trips, TLS 1.3 needs one, and 0-RTT resumption needs none. Session tickets only help if every load balancer shares the same key.

Caching & Performance

Mar 6, 2026

DNS Resolution in the Browser and Beyond: When TTL is a Suggestion

DNS resolution touches browser cache, OS resolver, recursive resolver, root, TLD, and authoritative servers. TTLs govern caching at each step, except when a resolver ignores them.

Distributed Systems

Mar 5, 2026

Split Brain and Fencing Tokens: What Happens After the Election

Electing a new leader is the easy part. Preventing the old leader from writing once it wakes up from a long pause is what fencing tokens are for.

Distributed Systems

Mar 5, 2026

Raft Consensus: Leader Election, Log Replication, and a Slow-Disk Failure

Raft breaks consensus into leader election, log replication, and safety. Why majority quorum is magic, and how a slow follower took down a 3-node cluster.

Distributed Systems

Mar 4, 2026

Saga Pattern for Distributed Transactions: Compensations Need Idempotency Keys Too

Two-phase commit collapses under coordinator failures. Sagas replace global atomicity with compensating actions, but only if every forward step and every compensation is idempotent.

Caching & Performance

Mar 3, 2026

Tail Latency and Hedged Requests: Why p50 Looks Fine While Users Complain

p50 says the system is healthy. Users say it feels random. The gap is the tail, and the fix is a careful one because the obvious mitigation can finish the outage.

Kafka & Streaming

Mar 2, 2026

At-Least-Once vs Exactly-Once: There Is No Third Option

Three delivery semantics exist on paper, but exactly-once is really at-least-once plus idempotency. Pick the one your business logic can survive.

Distributed Systems

Mar 2, 2026

Causal Consistency vs Linearizability: Choosing the Right Order

Linearizability gives one global timeline at consensus cost. Causal consistency preserves happened-before without coordination. The tradeoff and one production trap.

Distributed Systems

Mar 1, 2026

Vector Clocks: Detecting Concurrent Writes Without a Global Clock

Vector clocks track happened-before across multi-writer replicas and flag true concurrency. Plus the sibling explosion that broke a key-value store in production.

System Design

Feb 28, 2026

Backpressure and Overload Control: Scale on Leading Signals, Not Lagging Ones

Retries and DLQs handle correctness. Backpressure handles survival. Without a signal from downstream, autoscaling on queue depth makes overload worse, not better.

Distributed Systems

Feb 27, 2026

Why Clocks Lie in Distributed Systems

NTP jumps, VM drift, and leap seconds make wall-clock ordering unsafe. Why Spanner waits out clock uncertainty, and how an 80 ms rewind erased 1200 financial orders.

System Design

Feb 27, 2026

SLOs and Error Budgets in Practice: Short Windows Beat Monthly Resets

An SLO is a target, an error budget is the runway. Monthly windows hide near-miss outages and create deploy freezes that ship even riskier code later.

Kafka & Streaming

Feb 27, 2026

DLQ and Retry Backoff: The Pattern That Saves Your Queue at 2am

A single poison message can stall a partition and stop every other consumer behind it. Tiered retries plus a DLQ keep the rest of the stream flowing.

Distributed Systems

Feb 27, 2026

Infra Cron vs Application Scheduler: Who Should Own the Job Lifecycle

Kubernetes CronJob and EventBridge are not interchangeable with Celery beat or TaskIQ. Picking wrong means either fighting your platform or losing visibility into your own jobs.

System Design

Feb 27, 2026

Circuit Breaker Pattern: Stop Cascading Failures Cold

When a downstream service starts failing, the circuit breaker pattern stops you from taking everything else down with it. Closed, Open, Half-Open explained.

Networking & Load Balancing

Feb 26, 2026

The Other Half of the Outbox: Consumer-Side Deduplication

The outbox gives you at-least-once delivery, which means duplicates are not a bug, they are the contract. The consumer side needs an inbox table or a Redis dedup store keyed on event_id.

Kafka & Streaming

Feb 25, 2026

Exactly-Once Delivery Is a Myth. Aim for Exactly-Once Effects Instead.

Exactly-once network delivery is impossible. What you actually want is exactly-once effects: at-least-once transport plus idempotent consumers with a dedupe key.

Distributed Systems

Feb 25, 2026

Read-Your-Writes Consistency: Why Your Profile Update Disappears on Refresh

You save a profile change, refresh, and see the old value. The bug is not the form. It is replication lag plus a follower read. Here are the four real fixes.

Databases

Feb 24, 2026

Relational, Document, Graph: Pick the Database That Matches Your Relationships

Database choice is really a relationship-shape choice. Joins want relational. Aggregates want document. Traversal wants graph. Forcing the wrong one shows up as recursive CTE hell.

System Design

Feb 24, 2026

Retries, Timeouts, and Bulkheads: The Resilience Patterns That Only Work Together

Retries without timeouts cause storms. Timeouts without bulkheads let one slow dependency drain your pool. These patterns are a system, not a menu.

Networking & Load Balancing

Feb 24, 2026

Polling vs SSE vs WebSocket: Pick the Simplest Transport That Meets the Need

Most teams reach for WebSockets when they hear real-time. Polling and SSE are usually enough, cheaper to operate, and friendlier to your existing HTTP stack.

Caching & Performance

Feb 23, 2026

Negative Caching: When Caching Nothing Saves the Database

Caching the absence of a value protects the database from invalid-key floods. Here is why short TTLs alone are not enough, and how a long negative TTL can hide a new record from production for hours.

Distributed Systems

Feb 23, 2026

Kubernetes Is a Control Loop, Not a Container Runner

Kubernetes is a declarative reconciler. Every feature you think of as separate (probes, restarts, PDBs, spread) is a different specific control loop on top of the same mental model.

Caching & Performance

Feb 22, 2026

Concurrency Control Under Load: When Adding Threads Makes Things Slower

More threads and more replicas feel like the safe response to a slow service. Little's Law and queue depth tell a different story: smaller pools with fast rejection often beat larger pools that queue quietly.

Caching & Performance

Feb 22, 2026

Redis Connection Pooling Is a Correctness Problem, Not a Performance One

A connection per request kills throughput and exhausts file descriptors. A pool sized to peak request rate hides the bug. The right size tracks peak concurrency, with validation on borrow.

Databases

Feb 20, 2026

Database Indexes: What They Cost You for the Speed They Buy

B+tree indexes give O(log n) reads via sorted pages, with leftmost prefix rules for composite keys and covering indexes that skip the heap. Every index also taxes every write, so prune before you add.

Databases

Feb 19, 2026

The Three Join Algorithms Every Query Planner Picks Between

Nested loop, hash join, merge join. Each wins in a different shape of data. Reading EXPLAIN tells you which one the planner chose and whether it was right.

Databases

Feb 19, 2026

Time-Series Database Internals: Why TSDBs Look Nothing Like Postgres

How InfluxDB, TimescaleDB, and Prometheus actually store metrics. Time-chunked files, columnar layout, downsampling, retention, and why insert-only is the whole game.

Distributed Systems

Feb 17, 2026

Consistent Hashing: Why Virtual Nodes Are Not Optional

Plain modulo hashing remaps almost every key when the cluster size changes. A hash ring moves only K/N keys, but without virtual nodes the load distribution is wildly uneven and gets worse when you add capacity.

Databases

Feb 16, 2026

Database Partitioning Strategies: Range, Hash, List, and the Composite You Wish You Had Picked

Range, hash, and list partitioning each optimize for different access patterns. The trap is picking a key that distributes writes well but destroys read locality.

Caching & Performance

Feb 15, 2026

Hot Partition Mitigation: When One Key Brings the Cluster to Its Knees

Salting, pre-splitting, time-bucketing, and a cache tier. Each fixes a different kind of hot partition. Picking the wrong one trades one bottleneck for another.

Databases

Feb 14, 2026

Partition Pruning: The Optimization That Vanishes When You Touch the Key

Partition pruning lets the planner skip irrelevant partitions, but only when it can see the partition key in the predicate. Wrap the column in a function and the optimization disappears.

Databases

Feb 13, 2026

The Query Planner Is Guessing, and Stale Statistics Make It Guess Wrong

The cost-based optimizer picks scan type, join order, and join method from row count estimates. When the estimates are off by five orders of magnitude, it picks nested loops on a million rows and your query runs for 25 minutes.

Databases

Feb 12, 2026

Write-Ahead Logging: The Real Boundary Between Committed and Lost

WAL appends every change to a sequential log before touching data pages. fsync on that log is the durability line. Group commit, synchronous_commit, and battery-backed caches all live on this line.

Databases

Feb 11, 2026

Transaction Isolation Levels: The Anomaly Each One Actually Stops

Read Uncommitted, Read Committed, Repeatable Read, and Serializable each ban a specific anomaly. Postgres Repeatable Read is snapshot isolation and still allows write skew. Knowing which level stops what is the whole game.

Databases

Feb 11, 2026

B-Trees vs LSM-Trees: How the Write Path Differs

B-trees do in-place page updates with random I/O. LSM-trees buffer in a memtable and flush sorted SSTables. The write path is where the two structures diverge.

Databases

Feb 9, 2026

Postgres VACUUM and Bloat: Why Your Hot Table Quietly Gets Slow

MVCC keeps old row versions until no transaction can see them. VACUUM reclaims that space. Long transactions, idle-in-transaction sessions, and replication slots pin dead tuples and let tables bloat 10x or more.

Caching & Performance

Feb 6, 2026

Write Amplification: The Hidden Multiplier That Kills SSDs Early

Write amplification is the ratio of bytes the disk writes to bytes your app writes. B-trees sit around 4 to 6. LSM trees commonly hit 10 to 30. SSDs add another 2 to 3x on top. If you do not measure it, your endurance budget evaporates.

Databases

Feb 4, 2026

Bloom Filters: A Probabilistic Permission Slip to Skip Work

K hash functions, one bit array, zero false negatives. Bloom filters answer maybe or definitely not, which is exactly what LSM reads, CDN caches, and crawlers need to skip expensive checks.

Databases

Feb 1, 2026

LSM Trees: Why Writes Are Almost Free and Reads Pay the Bill

Memtable plus WAL plus SSTable levels. The LSM write path is O(1) sequential I/O, while reads climb levels guarded by bloom filters and sparse indexes. Compaction is the price.

Databases

Feb 1, 2026

MVCC: Why Readers Never Block Writers

MVCC gives every transaction its own snapshot by keeping multiple row versions. Postgres tuples plus VACUUM, MySQL undo log. Readers and writers stop fighting.

Caching & Performance

Jan 31, 2026

Hot Partitions and Hot Keys: The Tax You Pay for Going Viral

One celebrity user can melt a single shard while the rest of the cluster sits idle. Here is how to detect hot keys early and the four fixes that actually work in production.

Databases

Jan 29, 2026

Partitioning vs Indexing: Two Layers, Two Problems, One Common Confusion

Partitioning bounds the working set, indexing accelerates lookups within it. You usually need both, and local vs global indexes is the question that decides whether your partitioned table is actually fast.

Databases

Jan 28, 2026

ACID Letter by Letter: Which Failure Each One Actually Prevents

Atomicity stops partial commits. Consistency is an app invariant, not a DB guarantee. Isolation stops concurrent transactions from corrupting each other. Durability survives the crash. Most apps lose money at the I, not the A.

Caching & Performance

Jan 27, 2026

The Cache Hierarchy: Performance Is Which Layer You Actually Hit

L1 is 1ns, L2 is 4ns, L3 is 10ns, RAM is 100ns, local Redis is 1ms, remote Redis is 5ms, a database call is 50ms, cross-region is over 100ms. Every layer is 10 to 100x slower than the one above. Optimize at the layer that moves the metric.

Databases

Jan 24, 2026

LSM Compaction Strategies: Size-Tiered, Leveled, and the Workload That Picks for You

Size-tiered compaction trades space for write throughput. Leveled compaction trades write amplification for predictable reads. Universal sits in the middle. Pick by workload, not by default.

Databases

Jan 23, 2026

LSM Reads: Bloom Filters and Sparse Indexes Are What Make GETs Survivable

An LSM GET could scan every SSTable on every level. It does not, because two structures filter the work: a bloom filter per file and a sparse index inside each file. Both are non-negotiable.

Databases

Jan 21, 2026

B-Tree vs LSM: Stop Comparing Features, Start Comparing Disk Shapes

B-trees mutate pages in place. LSM trees append immutable files and merge later. Once you see the disk shape, write amplification and read latency stop being mysterious.

Caching & Performance

Jan 21, 2026

Why B-Trees and LSM Trees Both Exist: The Disk Physics Behind the Split

B-trees and LSM trees solve different problems forced on us by disk physics. Random seek cost, sequential bandwidth, and why spinning platters and SSDs picked different winners.

Databases

Jan 20, 2026

Write Amplification in B-Trees and LSM Trees: The Ratio Math

B-trees amplify writes through page rewrites and the WAL. LSM trees amplify through compaction across levels. The math behind each, and why neither is free.

Kafka & Streaming

Jan 18, 2026

Redis Streams Trimming: The Tilde in MAXLEN Is Not Free

XADD MAXLEN ~ N is approximate and can keep nearly twice as many entries as you asked for. Under bursty writes that lie becomes silent data loss. Use exact bounds or move streams to their own instance.

Kafka & Streaming

Jan 17, 2026

Redis Streams Make Backpressure Visible, and That Is the Whole Point

The Pending Entries List is the load-bearing data structure. A slow consumer does not silently lose work, it accumulates entries you can measure with XPENDING and reassign with XAUTOCLAIM.

Kafka & Streaming

Jan 16, 2026

Redis Streams Are At-Least-Once. You Build Exactly-Once Effects on Top.

Redis Streams give you durable delivery and a pending entries list. They do not give you exactly-once. The trick is pairing them with idempotent sinks so duplicates do not cause damage.

Kafka & Streaming

Jan 16, 2026

Redis Pub/Sub vs Streams: Two Tools That Look Similar and Solve Opposite Problems

Pub/Sub is fire and forget with no memory. Streams are a durable log with consumer groups and replay. Picking the wrong one quietly loses messages until the day it does not.

System Design

Jan 15, 2026

Redis Is Single-Threaded and Still Has Race Conditions

Single-threaded does not mean race-free. Here is why GET-then-SET breaks under concurrency, and how Lua scripts, WATCH/MULTI/EXEC, and SETNX restore correctness.

Databases

Jan 14, 2026

Redis Cluster Slot Migration: How Traffic Keeps Flowing While Slots Move

Redis Cluster shards by 16384 hash slots, not by keys. Here is how MOVED and ASK redirects keep clients on the right node during a live resharding, and how one bad client setting can loop traffic forever.

Databases

Jan 12, 2026

Redis Durability: What You Actually Lose on Crash

Redis is honest about durability. RDB, AOF, and replication solve different problems and lose different amounts of data. Here is exactly what is gone after a crash under each mode.

Databases

Jan 12, 2026

Redis Durability: RDB, AOF, and the Lie of 'Just a Cache'

RDB snapshots are fast but lossy. AOF with everysec costs a second of writes on crash. Replication protects availability, not data. The mode you choose decides what you lose.

Caching & Performance

Jan 11, 2026

Redis Eviction Policies: Eight Choices, and Only One Is Right for Your Workload

noeviction, allkeys-lru, allkeys-lfu, volatile-lru, volatile-lfu, volatile-ttl, allkeys-random, volatile-random. Each picks a different victim. Mixing cache and session data in one instance with the wrong policy logs your users out.

Caching & Performance

Jan 10, 2026

Cache Stampede: When TTL Expiry Becomes a Self-Inflicted DDoS

A single key's TTL ticks to zero and a thousand concurrent requests slam the database at the same instant. Here is how singleflight, stale-while-revalidate, and jitter keep that from becoming an outage.

Caching & Performance

Jan 9, 2026

Synchronized TTL Is a Time Bomb You Lit at Deploy Time

When a thousand keys all expire on the same second, every dependent request misses at once. The origin sees a synchronized cliff, not a smooth load curve. Jitter is the cheap fix.

Kafka & Streaming

Jan 6, 2026

Redis Is Three Messaging Systems Pretending to Be One

Pub/Sub is at-most-once and forgets offline subscribers. LIST plus BLPOP is at-least-once with manual ACK. Streams add consumer groups and replay. Picking wrong costs you messages.

Caching & Performance

Jan 5, 2026

Redis Is a Toolbox, Not a String Store: Pick the Structure That Fits the Access Pattern

Sorted sets, hashes, bitmaps, HyperLogLog, GEO, and lists each map to a different system design pattern. Choosing wrong is not a style preference, it is a CPU and memory bill you pay every request.

Kafka & Streaming

Jan 4, 2026

Kafka Producer Performance Tuning: The Four Knobs That Actually Matter

Most Kafka producer throughput problems are a config problem, not a Kafka problem. Here are the four settings that turn a 50K msg/s producer into a 400K msg/s producer.

Kafka & Streaming

Jan 3, 2026

Kafka Partitioning and Ordering: Picking the Right Key

Kafka partitioning is three things at once: parallelism, ordering, and hot keys. The wrong key makes one consumer carry the whole load.

Kafka & Streaming

Jan 2, 2026

Kafka Consumer Offsets Deep Dive: Position vs Committed

Kafka consumers track two offsets per partition: an in-memory position and a durable committed offset. Auto-commit hides a five-second window where crashes cause duplicate processing.

Kafka & Streaming

Dec 31, 2025

Kafka Idempotent Producer: What `enable.idempotence=true` Actually Buys You

The idempotent producer dedupes retries by (producer_id, sequence_number) per partition. It is not exactly-once across services, and confusing the two causes real outages.

Kafka & Streaming

Dec 31, 2025

How Kafka Transactions Work: Atomicity by Visibility, Not Rollback

Kafka transactions never roll back. The coordinator writes COMMIT or ABORT markers and read_committed consumers skip the rest. Here is how that adds up to exactly-once.

Kafka & Streaming

Dec 30, 2025

Kafka Log Truncation and Unclean Leader Election: When Failover Rewrites History

Failover in Kafka is a log truncation event. Clean election drops only uncommitted data. Unclean election can erase records consumers already read.

Kafka & Streaming

Dec 28, 2025

Kafka Log Compaction vs Time Retention: Two Storage Models in One Broker

Time/size retention is for event streams. Compaction keeps the latest value per key for state topics. Pick the wrong one and your consumers drift silently.

Distributed Systems

Dec 27, 2025

Idempotency and Retries: The Pattern Every State-Changing API Owes You

Retries are a feature, but without idempotency they are a bug. Client-supplied idempotency keys plus a dedup table with TTL is the pattern Stripe and every payments API converged on for a reason.

Kafka & Streaming

Dec 27, 2025

The Transactional Outbox: Publishing Events Without Losing Them

Writing to a database and publishing an event are two operations. The transactional outbox makes them one, so events survive crashes and the broker can be down without anyone noticing.

Distributed Systems

Dec 26, 2025

Sagas: Distributed Transactions Without the Global Lock

Two-phase commit blocks. Three-phase commit assumes a network you do not have. Sagas accept eventual consistency and replace rollback with explicit compensating actions.

Kafka & Streaming

Dec 25, 2025

Kafka acks, min.insync.replicas, and Producer Retries: The Durability Dial

Kafka durability is a dial, not a switch. acks, min.insync.replicas, and producer retries together decide whether a write survives a broker failure or vanishes.

Distributed Systems

Dec 24, 2025

Three-Phase Commit: Why It Exists and Why Nobody Uses It

3PC was invented to fix the blocking problem in 2PC by adding a PreCommit phase. It works in theory and fails the moment you allow network partitions.

Distributed Systems

Dec 23, 2025

Three-Phase Commit vs Two-Phase Commit: Why Nobody Actually Runs 3PC

3PC adds a PreCommit phase to fix the blocking problem in 2PC, but the proof assumes a synchronous network. Real networks reorder and delay, and the protocol breaks in subtle ways.

Databases

Dec 23, 2025

Elasticsearch Indexing vs Search: The Two Paths That Define the System

Elasticsearch has two completely different code paths. The index path builds inverted indexes through analyzer chains. The search path fans out, scores, and merges. The refresh interval is the seam between them.

Caching & Performance

Dec 23, 2025

Traffic Spikes Are Not a Capacity Problem

More servers does not save you from a flash sale or a viral post. Spikes break the shape of the load, not its size. Front-door limits, async paths, and load leveling are what actually hold the line.

Kafka & Streaming

Dec 22, 2025

Kafka Durability and acks: Three Settings, Three Different Guarantees

Kafka's acks setting is the difference between fire-and-forget, leader-only, and full ISR durability. Pair it with min.insync.replicas or it does not mean what you think.

Distributed Systems

Dec 22, 2025

Two-Phase Commit Walkthrough: The Blocking Problem Nobody Wants to Own

2PC gives you atomicity across services at the cost of availability. Participants that voted yes will hold their locks forever if the coordinator dies at the wrong moment.

Kafka & Streaming

Dec 21, 2025

Kafka Scaling Math: Partitions, Consumers, and Why More Is Not Free

Kafka throughput is partitions times per-partition ceiling. Consumer parallelism caps at partition count. Doubling partitions costs you controller load, rebalance time, and broker boot time.

System Design

Dec 20, 2025

Why Microservices Fail: The Distributed Monolith Trap

Microservices do not fail because the pattern is wrong. They fail because teams split the code without splitting the ownership, and end up with a distributed monolith that ships slower than the original.

System Design

Dec 19, 2025

Serverless Architecture Explained: It Is Not No Architecture

Serverless does not mean architecture-free. It means your architecture moves into IAM, API Gateway, Lambda, Step Functions, and EventBridge config, where cold starts and concurrency limits are first-class constraints.

System Design

Dec 7, 2025

Access Tokens and Refresh Tokens: Why You Need Both

Access tokens are short-lived bearer credentials sent on every API call. Refresh tokens are long-lived and only ever leave storage to exchange for new access tokens. Mixing up where each one lives is how accounts get drained.

Caching & Performance

Dec 3, 2025

Anatomy of a Trading Order: What Place Order Actually Does

From API call to confirmed fill, an order touches risk checks, an OMS, a matching engine, market data fan-out, and settlement. Every hop has a sub-millisecond budget and a way to ruin it.

System Design

Nov 26, 2025

Web Crawler System Design: Politeness Is a Sharding Problem

A web crawler is a URL frontier, a fetcher pool, a parser, a dedup layer, and storage. The hard part is not any one component. It is enforcing per-host politeness when the crawler is sharded across many nodes.

Caching & Performance

Dec 12, 2023

API Gateway: One Chokepoint, One Policy Plane, One Set of Tradeoffs

An API Gateway is not a router with extra features. It is the one place where every request meets the same policies. That is both the value and the cost.