The previous lesson's engine is a database without a server. This one is a storage engine without a database: the layer underneath everything the query processing lessons described, offered on its own as a component for other systems to build on.

What this kind of library provides

An ordered map from byte strings to byte strings, with reads, writes, deletions, ordered iteration, atomic groups of writes and point-in-time snapshots.

Starting with the interface is right, because the interface is unusually small. That list is close to the whole of it.

What it deliberately does not provide

No query language, no schema, no data types, no joins, no planner, no network listener and no notion of users.

Listing the absences is necessary. Everything the middle of this course covered is missing by design.

Why that makes it a component rather than a product

It is the bottom third of a database, so a system that wants the rest must supply the rest itself.

Framing the position is valuable, and it is the lesson's central idea. The library is a layer, not an application.

Who owns the meaning of the data

The caller, entirely. Serialisation, key layout, secondary indexes and every constraint are the embedding system's responsibility.

Giving the division of labour is essential. The library stores bytes and knows nothing about them.

Why the interface being this small is a feature

A library with six operations can be understood completely, embedded without surprises and kept stable across many years of releases.

Explaining the value of the narrowness is worthwhile, because it is easy to read the absences as poverty. The small surface is what makes the component dependable.

Why that stability matters to an embedding system

A system that has built its schema, indexes and transaction policy on this layer cannot absorb changes to it, so a storage layer that keeps its promises is worth more than one that grows features.

Giving the dependency is valuable. The contract is load bearing.

Why keys being bytes is consequential

Iteration proceeds in bytewise order of the key, so the ordering the application wants must be produced by how it encodes keys.

Explaining the encoding problem is necessary. Key design becomes a real design task.

Why integer keys need care

An integer written in the machine's native byte order does not sort numerically as bytes, so a big-endian encoding is required for range scans to work as expected.

Giving the concrete trap is valuable, and it is a classic first mistake. The ordering is wrong in a way that looks fine on small data.

How composite keys are built

By concatenating fixed-width or length-prefixed parts so that a prefix of the key is a meaningful range, which is how one keyspace holds many logical tables.

Covering the technique is necessary. Prefixes do the work schemas would.

Why that prefix trick is the usual pattern

A short identifier at the front of every key lets a scan over that prefix retrieve one logical collection, which is how embedding systems partition a single map.

Explaining its ubiquity is worthwhile. One mechanism covers most needs.

Why the deletion interface deserves attention

A deletion is itself a written entry marking the key as removed, rather than an in-place erasure, so deleting is a write with the same costs as any other.

Giving the asymmetry is necessary, since it surprises people who expect deletion to free space. Removal is an addition.

What a range deletion offers instead

A single entry covering a whole key range, which removes a large span at the cost of one write rather than one write per key.

Covering the specialised form is practical. The operation is the right tool for bulk removal.

What separate keyspaces within the database offer

Independent namespaces that can be configured, flushed and compacted separately while still sharing one write log and one atomic write group.

Covering the feature is necessary. The separation is partial and deliberate.

Why sharing the log matters

An atomic write can span those namespaces, which is what makes them useful for an index kept beside its data.

Giving the reason is valuable. The sharing is the point rather than a limitation.

Why systems embed a storage engine instead of writing one

The storage layer is the hardest part of a database to get right and the least differentiated, so reusing it concentrates effort where a system is actually distinctive.

Giving the motivation is the most useful content here. The reasoning explains the whole category.

Why the versioned key pattern is so common above this layer

Systems that need their own multi-version concurrency control append a version to each key, which turns version lookup into a range scan backwards from a bound.

Giving the pattern is valuable, because it is how most distributed stores built on this layer work. The encoding carries the versioning.

Why that pattern depends on the ordering guarantee

It only works because iteration is bytewise ordered and can start from an arbitrary key, which is the one structural property the whole arrangement rests on.

Explaining the dependency is worthwhile. Ordered iteration is the feature that makes the layer general.

What kinds of systems do this

Relational engines that replace their own storage layer, distributed key-value stores that need a local engine per node, stream processors that keep operator state, and ledger systems that want ordered durable storage.

Listing the uses is practical. The pattern is widespread rather than exotic.

Why the distributed case is the clearest

A distributed store solves replication, placement and consensus, and none of those require inventing a local storage format, so the local layer is exactly the part to borrow.

Explaining the clearest instance is valuable. The division of concerns is clean.

Why the stream processing case is interesting

Operator state must survive restarts and be queryable by key, which is a storage problem rather than a streaming one, so the same library serves it unchanged.

Giving the second instance is worthwhile. The requirement is more familiar than it looks.

What the embedding system still has to build

Transactions above single write groups, a schema and type system, indexes and their maintenance, a query interface, and anything to do with the network.

Being explicit about the remaining work is necessary. The borrowed layer is a start rather than a shortcut.

Why the transaction boundary is the subtle part

The library offers atomic write groups and snapshots, which are the ingredients of transactions, but the concurrency control the course described must be assembled from them.

Giving the subtlety is valuable. The pieces are present and the policy is not.

What the library does offer towards that

An optional transaction layer providing both pessimistic and optimistic variants, built on the same primitives the embedding system could use directly.

Being accurate about what exists is right. Some assembly is provided.

Why it is organised as an append-structured engine

Writes accumulate in memory and are flushed to immutable sorted files that are later merged, which is the structure the storage lessons described.

Connecting to earlier material is necessary. The mechanism is already known.

Why that choice fits a library

Immutable files and a memory buffer make the engine easy to embed, since there is no in-place page writing to coordinate with anything else in the process.

Explaining the fit is valuable. The structure suits the packaging.

What this lesson adds to the earlier treatment

Not the mechanism, which the storage lessons covered, but what the structure means operationally once it is inside somebody else's system.

Setting the scope is right. The ground is deliberately different.

What to take from the component model

Three things. The library provides an ordered map over byte strings with atomic write groups and snapshots and nothing above that, so the embedding system owns serialisation, key encoding, indexes, schema, types, queries and the network. Keys are compared bytewise, which makes key encoding a design task: integers need a big-endian encoding to sort numerically, and concatenated prefixes are how one map holds many logical collections. And systems embed it because the storage layer is the hardest and least differentiated part of a database, which is why relational engines, distributed stores and stream processors all build on the same layer.