Everything in the previous two lessons assumed that asking for bytes to be made durable works. This lesson is about that assumption, which is the weakest one in the whole design and the one most often violated in practice.

The gap the whole lesson is about

Handing bytes to the operating system is not making them durable. The call returns as soon as the data is accepted into a cache, and a power failure at that moment loses it with no error ever reported.

Stating the gap first is necessary because every mechanism in this lesson exists to close some part of it. The distinction between accepted and durable is the single most important one in storage, and almost every durability bug in practice is a place where the two were confused.

The layers between the engine and the media

The engine's own buffer, the operating system's page cache, the device's volatile cache, possibly a controller's cache, and finally the media. Each is a place data can sit and be lost.

Counting the layers explains why the problem is hard to reason about. A durability guarantee has to hold at every layer, and each layer is configured independently by different people, so a system can be correct in five places and wrong in the sixth.

What each layer may do with a write

Accept it and report success, reorder it relative to other writes, hold it indefinitely, and discard it on power loss. None of those behaviours is a malfunction, and all of them are permitted unless a flush is requested.

Understanding that reordering is permitted matters as much as the loss. The write-ahead rule depends on ordering between the log and the pages, and a layer that reorders freely breaks that ordering unless the engine asks for the ordering explicitly.

What a flush actually requests

That everything accepted so far be placed on the media and that the call not return until it has. It is both a durability request and an ordering barrier, which is why it is the only primitive the design needs.

Recognising it as two things at once explains its cost. A flush cannot be made asynchronous without losing the barrier property, so the commit path's wait is not an implementation shortcoming but the semantics being used.

Why the device cache exists at all

It absorbs bursts, it lets the device reorder writes into an efficient order, and on flash it accumulates enough data to fill an erase block. Removing it would cost a great deal of performance.

Knowing why it is there prevents the wrong conclusion. The cache is not a defect to be disabled but a mechanism to be flushed correctly, and a device with its cache disabled is often several times slower for no gain over flushing properly.

Why the cache can be made safe

A device with a battery or capacitor can report a write as durable immediately, because it can finish writing from its own cache after power is lost. Such a device makes flushes nearly free.

This is the hardware answer to the latency problem and it is worth asking about. Enterprise storage usually has it, consumer storage usually does not, and the difference in commit latency between the two is often an order of magnitude.

Why the question is about the whole path

A device that honours flushes correctly behind a controller that ignores them provides nothing. The guarantee is the weakest link, and every layer must be verified rather than assumed.

This is why the verification at the end of the lesson is about the path rather than the device. A correct device, a correct file system and an incorrect virtualisation layer produce a system that fails exactly as if the device were lying.

Why virtualisation deserves specific suspicion

A virtual disk is implemented by software that may cache writes in the host, and whether flushes are passed through to the host's storage depends on a configuration most people never see.

Naming it specifically is justified by how often it is the answer. A database that was durable on hardware and is not durable in a virtual machine has usually lost a flush somewhere in the virtual disk configuration, and nothing about the symptom points there.

Why network storage changes the question rather than removing it

A remote volume has its own durability model, usually replication to several devices, and a flush becomes a network round trip to a service that acknowledges when enough copies hold the data.

The model is often stronger than a local device's, which is worth saying because the instinct is to distrust it. The cost moves from device latency to network latency, and the failure modes move from power loss to partition and service degradation.

Why the operating system's cache is the easiest layer to get wrong

It is the only layer that can be bypassed entirely, and whether the engine bypasses it or flushes it changes both the correctness argument and the performance profile substantially.

This is where engine configuration actually bites. An engine using the page cache must flush it correctly, an engine bypassing it takes responsibility for its own buffering, and a system configured as though one were the other performs badly or incorrectly.

What bypassing the page cache buys and costs

It removes a copy and a second cache that duplicates the engine's own, which is why engines that manage their own buffer pool prefer it. It costs the kernel's readahead and its absorption of small writes.

The trade is usually right for a database and usually wrong for an application. An engine that already has a buffer pool is caching twice when it uses the page cache, which wastes memory that its own pool would use better.

Why double caching is worse than it sounds

The same page occupies memory twice, so a machine with sixty-four gigabytes gives the engine far less effective cache than its configuration suggests, and the operating system's copy is managed by a policy that knows nothing about the workload.

Quantifying it is what makes the configuration decision obvious. Half the memory spent on a cache with a worse replacement policy than the engine's own is a straightforward loss, which is why the bypass exists.

Why none of this is visible in normal operation

Every one of these misconfigurations produces a system that is fast and correct until it loses power. There is no error, no warning and no degradation to notice.

This is the property that makes the area dangerous rather than merely intricate. A durability defect is indistinguishable from a durability guarantee until the moment it matters, which is why verification has to be deliberate rather than incidental.

What to take from the storage stack

Three things. Handing bytes to the operating system is not making them durable, and a layer may accept, reorder, hold or discard a write without any of that being a malfunction. A flush is both a durability request and an ordering barrier, which is why it cannot be made asynchronous and why the commit path's wait is the semantics rather than an implementation flaw. And the guarantee is the weakest link in a path of five layers configured by different people, which is why every misconfiguration here produces a system that is fast and correct right up until it loses power.