Every durable engine in use today is built on one rule about ordering two writes, and almost everything else about durability follows from it. The rule is worth deriving rather than memorising, because the derivation explains the whole design.

The two promises that conflict

A commit must survive a crash, which argues for writing to the device immediately. A page is the unit of modification and holds many rows, which argues for writing it rarely. Satisfying both at once is impossible with pages alone.

Stating the conflict first is what makes the log look inevitable rather than clever. If a commit required writing every page it touched, a transaction modifying ten rows in ten different pages would perform ten random writes before it could return, and the system would be limited by random write latency on every commit.

Why a page write cannot simply be made atomic

A page is larger than the unit the device writes atomically, so a crash partway through a page write leaves a page that is half old and half new, which is neither of the two states the engine believes possible.

This failure is worse than losing the write, because a torn page is corrupt rather than stale and the engine cannot detect it from the page alone. The problem is addressed in a later lesson, and it matters here because it rules out the obvious design of writing pages carefully and keeping no log.

The write-ahead rule

Before a modified page may be written to the device, the log records describing that modification must already be on the device. Before a commit may be acknowledged, the log records of that transaction must be on the device.

Those two sentences are the whole protocol and every durability mechanism in the engine is an implementation of one or the other. The first makes it safe to write pages whenever convenient, and the second makes it safe to acknowledge a commit without writing any page at all.

Why the ordering is what buys the freedom

Once the log is ahead of the pages, the engine may write pages in any order and at any time, because the log already describes what every page should contain. The pages become a cache of state the log defines.

Seeing the pages as a cache is the shift that makes the design comprehensible. The authoritative record of what the database contains is the log plus whatever pages happen to be on the device, and the pages alone are never authoritative, which is why recovery is a required step rather than an unusual one.

Why the log is cheap to write

It is appended sequentially to one place, so a commit costs one sequential write regardless of how many pages the transaction touched, and many transactions' records can be in the same write.

The asymmetry is the economic heart of the design. A transaction touching ten scattered pages becomes one sequential append rather than ten random writes, which is an improvement of one or two orders of magnitude, and it improves further as concurrency rises because more transactions share each write.

Why the log is also small

A log record describes a change rather than the state after it, so modifying one hundred bytes of an eight-kilobyte page logs something closer to one hundred bytes than to eight kilobytes.

The size difference compounds with the ordering benefit and explains why logging does not double the write volume in practice. It also explains why the configuration that logs whole pages, which exists for a specific reason covered later, is such a large change to a system's write profile.

What the log makes possible beyond durability

The same records support rolling back an incomplete transaction, replicating to another node, restoring a backup to a chosen point in time, and feeding a stream of changes to other systems.

Noticing the reuse matters because it explains why the log is central to the engine's architecture rather than peripheral. Four mechanisms that look unrelated from the outside are the same records read in different ways, which is also why a change to the log format is one of the hardest changes to make.

Why recovery is the design's price

Because pages on the device may be behind the log, starting up requires reading the log and reapplying whatever the pages are missing. The system cannot be available until that is finished.

Accepting a recovery step in exchange for fast commits is the trade, and it is the right trade because crashes are rare and commits are constant. The remaining design question is how long recovery takes, which is the subject of checkpoints in the next lesson.

What happens with no log

The engine would have to write every modified page before acknowledging a commit, making commit latency a function of how scattered the modification was, and it would still be exposed to torn pages with no way to detect or repair them.

Spelling out the alternative is useful because systems are still built this way by accident. An application that writes files directly and relies on the file system has exactly this design, which is why it is both slow to commit and unable to recover reliably.

Why the log must be on durable storage before the acknowledgement

Writing to the operating system is not writing to the device, because the write sits in a cache that a power failure discards. The engine must ask for it to be flushed and wait for the answer.

This is the step that costs the latency, and it is also the step most often weakened for throughput. The distinction between written and durable is the single most important distinction in this lesson, because every configuration that makes commits faster weakens it somewhere.

Why the rule is stated as two separate orderings

The page ordering protects data that was never committed from being made permanent. The commit ordering protects data that was committed from being lost. They guard opposite failures and both are required.

Keeping them distinct prevents a common confusion. An engine could satisfy the commit ordering alone and still corrupt itself, because a page written from an uncommitted transaction with no log record behind it leaves a change that recovery can neither complete nor reverse.

Why the log is the same idea as a journal in a file system

A file system faces the identical problem, that a metadata change touches several blocks and a crash between them leaves a structure that is internally impossible, and it reaches the identical answer.

Noticing the convergence is useful for two reasons. It suggests the design is forced rather than chosen, since two communities arrived at it independently from different starting points, and it explains why running a database on a journalling file system can mean logging the same change twice, once in each layer, which is a real source of write amplification.

Why the rule is older than the hardware it is tuned for

It was formulated when storage was mechanical and a random write cost a seek, and it survived the move to flash unchanged, because the asymmetry it exploits is about the number of operations rather than the cost of each.

Understanding why it survived prevents a common error. Flash made random writes far cheaper in absolute terms but did not make a scattered commit as cheap as one sequential append, and it introduced a new reason to prefer fewer writes, which is that flash endurance is consumed by writing rather than by seeking.

What to take from the rule

Three things. A commit must be durable immediately while pages are expensive to write, and the only way to satisfy both is to make a sequential log the authority and the pages a cache of what it defines. The rule is two orderings, log before page and log before acknowledgement, which guard opposite failures and are both required. And the price of the design is a recovery step at startup, which is the right price because crashes are rare and commits are constant.