0%
Database Internals: Build the Engine
Storage Structures
Durability
Concurrency Control
Query Processing
Distribution
Hash, Bitmap and Specialized Indexes
The ordered tree is so dominant that the alternatives are often assumed to be exotic, and the first of them is the structure most programmers would reach for first outside a database, which makes its narrow usefulness inside one worth understanding properly.
What a hash index does
It applies a function to the key to compute a bucket number, stores the entry in that bucket, and finds it later by computing the same function and reading that bucket directly.
Starting from the mechanism is right because the appeal is immediate. There is no descent, no comparison of keys along a path, and no tree to maintain.
Why the lookup looks unbeatable
One computation and one page read reach the entry regardless of how many entries exist, which is a cost that does not grow with the size of the data at all.
Stating the theoretical appeal plainly is necessary because it is genuine. The structure delivers exactly what its reputation promises.
Why it nonetheless rarely wins
The tree it competes with is only three or four levels deep and its upper levels are permanently resident, so a tree lookup is also one page read plus a few in-memory comparisons.
Delivering the comparison early is the honest structure for this section. The theoretical advantage turns out to be an advantage over a cost that was already negligible.
Why tree height is so low
Each internal page holds hundreds of entries, so three levels address tens of millions of rows and four levels address billions, which is why the depth barely grows.
Giving the arithmetic matters because it is the whole reason. Fan-out makes tree height nearly constant across every practical data size.
What the hash index gives up
Any question about order, because the function deliberately scatters nearby keys, so entries for adjacent values sit in unrelated buckets.
Naming the loss is the central limitation. Scattering is the property that makes lookups uniform and it destroys every ordered operation.
Why that removes range queries entirely
Finding every key between two values would mean visiting every bucket, which is reading the whole index, so a range condition cannot use the structure at all.
Spelling out the consequence is necessary. This is not a weaker range capability, it is the complete absence of one.
Why it also removes free ordering
A query wanting sorted results gets nothing from the index, so it must sort, which reintroduces a pipeline breaker the tree would have avoided.
Covering the second loss is worthwhile because it compounds. The tree's ordering serves purposes beyond range conditions.
Why it removes prefix matching too
Hashing the whole key means a partial key hashes to an unrelated bucket, so no condition on part of a compound key can be used.
Completing the list of losses is useful. Every capability the prefix rule describes depends on ordering and disappears with it.
What buckets do when they fill
Entries beyond a bucket's capacity go into chained overflow pages, which must be read as well, so the lookup cost stops being one page read.
Introducing overflow is necessary because it undermines the main claim. The constant cost holds only while the buckets are not overfull.
Why overflow chains are the real failure mode
A skewed key distribution or simply growth sends many entries to the same bucket, and the lookup then walks a chain whose length grows with the data.
Marking this as the failure mode is right because it is the practical problem. The structure degrades towards a linear scan of a chain.
Why resizing is awkward
Changing the number of buckets changes where every key belongs, so growing the index means relocating entries, which is expensive and must happen while the index is in use.
Covering the growth problem matters because trees do not have it. A tree grows by splitting one page at a time and a hash table grows by rearranging everything.
How incremental resizing avoids the worst of it
Splitting one bucket at a time in a fixed order, with a marker recording how far the split has progressed, spreads the relocation across many operations.
Giving the standard technique is useful. The problem is solvable and the solution adds complexity the tree never needed.
Why the structure is harder to make crash safe
A bucket split touches several pages that must change together, and a half-completed split leaves entries unreachable, so the operation needs careful logging.
Covering durability is necessary because it is a historical reason for the structure's absence. Early implementations were not crash safe and were therefore unusable.
Why concurrency is harder as well
A split moves entries between buckets while readers are looking for them, so readers must be prevented from missing an entry that is in transit.
Including concurrency completes the engineering picture. Every operation that was simple in the tree has an awkward analogue here.
Why uniqueness is still available
Enforcing that no two rows share a key needs only equality, so the structure can carry a uniqueness constraint even though it cannot answer any ordered question.
Noting the one ordered-sounding capability that survives is useful, because it is easy to assume all of them go together. Uniqueness is an equality property rather than an ordering one.
Where the structure genuinely wins
When the key is long, because the index stores a short hash rather than the whole key, which makes the index far smaller and each page far denser.
Giving the real use case is important for balance. A long text key as a tree key inflates every internal node and a hash of it does not.
Why that case is more common than it sounds
Keys that are long strings, concatenations or document identifiers appear constantly, and an equality lookup on one is exactly the query the structure serves.
Expanding on the case is worthwhile. The win is narrow in capability and not rare in practice.
Why in-memory engines favour it more
With no page reads to amortise, the comparison becomes instruction counts, where computing one hash beats walking three levels of comparisons.
Covering the memory-resident case matters because the conclusion reverses. The tree's advantage was partly an artefact of storage costs.
Why it also appears inside other operators
The same structure is used transiently by hash joins and hash aggregates, where only equality matters and the structure is discarded afterwards.
Connecting to execution is useful. Hashing is central to query processing and marginal as a persistent index.
Why the usual advice is to leave it alone
The tree serves equality lookups almost as well and everything else far better, so the default choice costs little and preserves every option.
Giving the practical conclusion is right. The structure is correct in specific circumstances and wrong as a default.
What to take from hash indexes
Three things. The constant-cost lookup is real and competes against a tree lookup that is already one page read plus a few resident comparisons, because fan-out keeps tree height at three or four levels for any practical size. In exchange the structure gives up every ordered capability: ranges, free ordering and prefix matching all disappear, and overflow chains, resizing, crash safety and concurrency are each harder than in a tree. It earns its place on long keys, where storing a short hash instead of the whole key shrinks the index substantially, and in memory-resident engines where the comparison becomes instruction counts.
The hash index's constant-cost lookup is an advantage over a cost that was already negligible. A tree's internal pages hold hundreds of entries each, so three levels address tens of millions of rows and four address billions, and those upper levels are read so constantly that they are permanently resident. A tree lookup is therefore also one page read plus a few in-memory comparisons. The hash index wins a race that was already close and loses ranges, ordering and prefix matching to do it.