0%
Database Internals: Build the Engine
Storage Structures
Durability
Concurrency Control
Query Processing
Access Paths
InnoDB Internals Tour
InnoDB makes almost the opposite storage decision from the engine in the previous lesson, and the comparison is the fastest way to understand both. Here the table is not a bag of pages, it is a tree, and nearly every distinctive behaviour follows from that.
What the clustered index is
The rows themselves are stored in the leaves of a balanced tree ordered by the primary key, so there is no separate heap holding the data.
Defining the structure first is the right opening, because everything else in the lesson refers back to it. The table and its primary index are one object rather than two.
Why that makes primary key lookups fast
Finding the key finds the row, with no second access needed, because the leaf containing the key is the leaf containing all of its columns.
Giving the immediate benefit is necessary. The arrangement eliminates an access that other layouts require.
What a secondary index holds instead of a location
It holds the indexed columns and the primary key, rather than a physical address, so following it means searching the clustered index for that key.
Explaining the pointer choice is essential to the section. The indirection is logical rather than physical.
Why that means two traversals
A query filtered on a secondary index must walk that tree to find primary keys and then walk the clustered tree for each one, which is two searches rather than one lookup and one fetch.
Quantifying the cost is important. The indirection is paid on every secondary access.
Why the arrangement is still a good trade
A logical pointer never needs updating when a row moves within the tree, which happens constantly as pages split and merge, so the alternative would require rewriting index entries continuously.
Being fair to the design is necessary. The cost buys the absence of a much larger one.
Why a covering secondary index removes the second traversal entirely
If the index holds every column the query needs, including the primary key it already carries, the clustered index is never consulted.
Giving the optimisation is valuable, and it is the single most effective index technique in this engine. The second traversal is avoidable by design.
Why the primary key appears in every secondary index
Each secondary entry must carry it to serve as the pointer, so the primary key's width is added to every entry of every secondary index on the table.
Stating the multiplication explicitly is the section's most practical content. One choice propagates everywhere.
Why a wide primary key is therefore expensive
A key of many columns or a long string inflates every secondary index, which means more pages for the same number of entries and more reading on every secondary lookup.
Drawing the conclusion is necessary. The cost is paid in space and in reads across the whole table.
Why a short synthetic key is the usual recommendation
A compact integer keeps every secondary index small while still identifying the row, which is why this engine's conventional advice differs from engines with physical pointers.
Giving the practical guidance is right. The recommendation has a mechanism behind it.
Why the key's ordering matters as well as its width
Rows are inserted into the tree in key order, so an increasing key appends to the rightmost leaf while a random key inserts into arbitrary leaves across the whole tree.
Introducing the ordering question is essential. The second property is as consequential as the first.
Why an increasing key is cheap
Only the rightmost leaf is being modified, so that page stays in memory, it fills completely before a new one is needed, and no existing page is disturbed.
Explaining the good case is necessary. The locality is near-perfect.
Why a random key is expensive
Each insertion lands in a different leaf, so pages must be read before being written, they split when full, and the splits leave both halves partly empty.
Giving the bad case is the section's most important operational content. The cost appears in three ways at once.
Why the resulting half-empty pages matter
A tree whose pages are half full occupies twice the space and requires twice the reading for the same rows, which affects every query against the table and not just the inserts.
Explaining the lasting cost is valuable. The damage outlives the insertion.
Why this makes random identifiers a storage decision
A randomly distributed identifier used as the primary key produces exactly this pattern, which is why it is discouraged here even though it is harmless in engines that do not cluster.
Drawing the practical conclusion is right. The advice is engine-specific and it has a reason.
What the usual remedy is
Either a monotonically increasing primary key with the random identifier kept in a unique secondary index, or an identifier scheme whose leading portion increases with time.
Giving both remedies is practical. The problem has two standard answers and both are commonly used.
Why the second remedy works
Insertions cluster into a narrow range of leaves rather than spreading across the whole tree, which recovers most of the locality while keeping the identifier's other properties.
Explaining the mechanism is worthwhile. The partial fix is genuinely effective.
Why a full table scan is an index scan here
Reading every row means walking the clustered index's leaves in order, since that is where the rows live, so there is no cheaper unordered path.
Covering the scan path is necessary. The arrangement has no heap to scan instead.
Why that gives scans a useful property
The rows arrive in primary key order at no additional cost, so a query wanting that ordering gets it for free.
Giving the benefit is worthwhile. One consequence is favourable.
Why it also means a scan is affected by tree health
A clustered index whose pages are poorly filled makes every scan read more pages, so insertion order affects sequential performance and not just insertion performance.
Connecting back is valuable. The two consequences compound.
What happens when no primary key is declared
The engine uses a unique index if one exists and otherwise creates a hidden increasing key, so the table is clustered regardless of what was declared.
Covering the default is necessary. The structure is not optional.
Why relying on that hidden key is unwise
It cannot be used by secondary indexes as a useful pointer for application purposes, and its existence is invisible in the schema, so the arrangement is harder to reason about.
Giving the recommendation is practical. The default works and should still be overridden.
Why large values are stored away from the row
A value too large for a page is kept in overflow pages with a pointer in the row, which keeps the leaves compact so more rows fit per page.
Covering overflow storage is necessary. The mechanism protects the tree's density.
Why that makes selecting large columns costly
Each such column read requires following the pointer, so selecting one unnecessarily turns a single page access into several.
Giving the practical consequence is valuable. The cost is controlled by the query's column list.
What to take from the clustered arrangement
Three things. The table is the primary key index, so a primary key lookup fetches the row with no second access while a secondary index lookup traverses two trees, because secondary entries point at the primary key rather than a location, which is what lets rows move within the tree without rewriting index entries. The primary key's width is added to every entry of every secondary index, so a wide key inflates all of them and a compact one is worth real money. And the key's ordering decides insertion locality: an increasing key appends to one leaf and fills pages, while a random key lands everywhere, splits pages and leaves them half full, which permanently inflates the tree and slows every read against it.
The single most consequential schema decision in InnoDB is the primary key, and it is consequential twice over. Its width is added to every entry of every secondary index, so a long key inflates all of them and makes every secondary lookup read more pages. Its ordering decides insertion locality, so an increasing key appends to the rightmost leaf and fills pages completely while a random one scatters across the tree, splits pages and leaves them half full. The second effect is permanent: those half-empty pages slow every scan and every lookup afterwards, long after the insertions have finished.