0%
Database Internals: Build the Engine
Storage Structures
Durability
Concurrency Control
Query Processing
Access Paths
Distribution
Pages, Slots and Tuples
Every structure in a database engine is built from fixed-size pages, and the internal layout of a page is one of those designs that looks arbitrary until you try to do without it, at which point every element turns out to be load-bearing.
Why the size is fixed and why it is small
A fixed size makes allocation trivial, since any free page can hold any page, and makes addressing trivial, since a page number multiplied by the size gives an offset. A small size keeps the unit of transfer and the unit of locking fine-grained.
The common choices, four, eight and sixteen kilobytes, are not arbitrary either. They are multiples of the operating system page size and of the storage device's internal block size, which means a page read is one device operation rather than a straddle across two. An engine whose pages were misaligned with the device beneath would double its read cost for no benefit.
What choosing a larger page buys and costs
Larger pages reduce the number of pointers needed to index the same data, flatten trees, and make sequential access cheaper. They also waste more space on a half-empty page, lock more rows at once, and make every write larger.
The trade sharpens in specific places. A B-tree over large pages is shallower, which means fewer page reads per lookup, but each read moves more bytes. Analytical workloads that scan favour large pages strongly; transactional workloads that touch single rows favour small ones; and this is one of the clearest cases where the right answer depends entirely on what the system is for.
The page header
At the start of a page sits identity and bookkeeping: the page number or a checksum covering the contents, a page type, the log sequence number of the last modification, counts of used and free space, and pointers to where the usable regions begin and end.
The log sequence number deserves special notice because it is the link between a page and the write-ahead log. Recovery compares the number on the page with the number in a log record to decide whether a change is already present, which is what makes replay safe to run more than once. That one field in the header is what makes crash recovery idempotent.
The slotted page design
The canonical layout puts a directory of slots growing from one end of the page and the row data growing from the other, with free space in the middle. A slot holds the offset and length of a row, and a row is addressed by page number plus slot number.
This arrangement solves several problems at once. Rows of different sizes coexist without external fragmentation inside the page. A row can be moved within its page by updating only its slot. And free space is a single contiguous region rather than a list of holes, so allocation is a pointer bump. The elegance is why nearly every engine uses it.
Why the indirection through slots is worth a level of lookup
Addressing a row by its byte offset would be faster to resolve and would make the row immovable, because every index entry and every pointer elsewhere would encode that offset. Any reorganisation would invalidate all of them.
Since reorganisation is routine, an update that lengthens a row has to move it, and compaction has to shuffle rows to reclaim space, immovability is unacceptable. The slot is the price of being able to rearrange the inside of a page without telling anybody, and that ability is used constantly.
Compaction inside a page
Deleting rows leaves holes. Rather than maintain a free list within the page, the engine periodically slides the surviving rows together, updating their slots as it goes, which turns scattered holes back into one contiguous free region.
This is cheap because the page is already in memory and small, and because the slot indirection means no external reference needs updating. It is also why a page can report less free space than the arithmetic suggests: the space exists but is fragmented, and the engine will compact when it next needs to fit something that does not currently fit.
Slot reuse and the tombstone problem
When a row is deleted its slot cannot always be reused immediately, because index entries may still point at it and concurrent transactions may still need to see it. The slot is often marked dead and kept until it is safe to reclaim.
This is a small detail with large consequences, and it is the point at which page layout and concurrency control stop being separable. How long a dead slot must survive is a visibility question, which is why vacuum, cleanup and garbage collection appear in a discussion that began as a question about bytes in a page.
Pages for different purposes
Not every page is a data page. There are index pages with a different internal structure, free space map pages, visibility map pages, large object pages, and metadata pages, and the page type field in the header is what distinguishes them.
The uniformity is the point. Because every one of them is the same size and goes through the same buffer pool with the same locking and the same logging, the engine has one mechanism for caching and one for durability rather than a special case per structure. That is a large simplification bought by a single early decision.
How a page is identified
A page identifier is usually a file or relation identifier plus a block number within it, and that pair is the key under which the buffer pool caches the page. Every structure in the engine ultimately refers to data by such a pair.
Keeping the identifier small matters more than it sounds, because it appears in every index entry, every pointer and every log record. Eight bytes versus sixteen, multiplied across a large index, is a real difference in how much of that index fits in memory, which is a performance difference rather than a storage one.
Free space inside pages, aggregated
An insert needs a page with room. Scanning pages to find one would be unacceptable, so engines maintain a compact summary structure recording approximately how much free space each page has, consulted before every insert.
Approximate is deliberate. An exact structure would need updating on every modification and would become a contention point, whereas an approximate one can be updated lazily and occasionally be wrong, costing one wasted page visit. Trading exactness for reduced contention is a pattern that recurs throughout engine design and this is one of its clearest instances.
Why page layout decides the cost of everything above
How many rows fit in a page determines how many pages a table occupies, which determines how many reads a scan performs. Narrow rows mean more rows per page and a cheaper scan; wide rows mean the opposite.
This is why schema decisions that look purely logical have direct physical consequences. Adding a column to a wide table can push rows per page from eleven to ten, making every full scan nine percent more expensive forever, and nothing in the logical model hints at it. Thinking about rows per page is the simplest tool for predicting such effects.
What to take from page layout
Three things. The slotted design, with a directory from one end and data from the other, solves variable-size rows, in-page movement and free space allocation together, which is why it is near-universal. The slot indirection exists so that pages can be reorganised without invalidating external references, and reorganisation is routine rather than exceptional. And the log sequence number in the header is what makes recovery replay idempotent, linking the lowest-level layout decision directly to durability.
The log sequence number in a page header is the quietest important field in a database. Recovery compares the number on the page against the number in a log record to decide whether a change is already applied, which is the entire reason replay can be run repeatedly without double-applying anything. A detail of page layout is what makes crash recovery safe.