Physical and Logical Replication

Topics Covered

What Replication Means Inside the Engine

What every form of replication has in common

Why the log is already that stream

Why that is such a convenient accident

What the two readings of the log are

Why the verbatim reading is the simpler one

Why a standby is a perpetually recovering engine

Why the decoded reading is harder

Why anyone bothers with the harder reading

Why both forms are asynchronous by default

Why that gap is a design choice rather than a limitation

What the receiver must guarantee about ordering

Why that makes the receiver a bottleneck

How receivers claw back some parallelism

Why the stream must be positionable

Why the sender must then retain the stream

Why retention is the feature's sharpest hazard

What replication is for, beyond the obvious

Why the purpose determines the form

Why many systems need both

Why neither form makes the primary faster

Why offloading reads is still conditional

What to take from the mechanism

Replication is usually described from the outside, as a second server with the same data, which hides the one fact that explains almost everything about how it behaves: the engine already produces a complete, ordered record of every change it makes, and replication is that record being read somewhere else.

What every form of replication has in common

A stream of changes produced by one engine and applied by another, in the order the first engine made them, so that the second converges on the same contents.

Starting from the common core is right because the two forms differ only in what the stream contains. Ordering and completeness are shared requirements.

Why the log is already that stream

Durability requires every change to be written to the log before it is considered committed, which means the log is by construction a complete ordered record of everything the engine did.

Connecting replication to the log immediately is the most useful move available, because it explains why the feature exists at all. The stream was already being produced for another reason.

Why that is such a convenient accident

No separate mechanism has to capture changes, no triggers are needed, nothing has to be instrumented, and the stream cannot miss a change without durability also being broken.

Explaining the convenience matters because alternatives were tried historically. Trigger-based capture existed and was worse in every respect.

The log is produced for durability and read for replication. Applying its bytes verbatim gives a physical replica; decoding its intent gives a logical one.

What the two readings of the log are

The records can be applied verbatim as changes to specific bytes in specific pages, or they can be decoded into statements about rows, which are the physical and logical forms.

Introducing the distinction as two readings of one artifact is the clearest framing. Nothing differs on the sending side until the reading is chosen.

Why the verbatim reading is the simpler one

The records already describe exactly what to change, so a receiver with an identical starting state and the same code can reproduce the result by doing what the records say.

Explaining the simplicity is useful because it is why this form came first. The work is already done by the recovery machinery.

Why a standby is a perpetually recovering engine

Crash recovery replays log records against pages to bring them up to date, and a physical standby does exactly that without ever stopping, so the mechanism is reused rather than invented.

This framing is worth stating explicitly because it explains a great deal of standby behaviour. The standby is in recovery permanently.

Why the decoded reading is harder

The records describe byte changes rather than rows, so recovering the row-level meaning requires knowing the table definitions and reconstructing which row each change applied to.

Previewing the difficulty is useful because it motivates the requirements that follow. The information is present and not directly available.

Why anyone bothers with the harder reading

A stream of row-level changes can be applied by something other than an identical engine, filtered to particular tables, and transformed on the way, none of which the verbatim form permits.

Giving the motivation immediately is right. The extra difficulty buys capabilities that matter.

Why both forms are asynchronous by default

The sending engine commits when its own log is durable, and the receiver applies the stream afterwards, so the receiver is always some distance behind.

Establishing the default is necessary because almost all confusion about replication is about this gap. Lag is the normal state rather than a fault.

Why that gap is a design choice rather than a limitation

Waiting for a receiver before committing would make every commit depend on another machine, so the default trades a window of possible loss for latency that does not cross the network.

Explaining the trade now prepares the later section. The gap is deliberate and adjustable.

What the receiver must guarantee about ordering

Changes must be applied in the order the sender made them, because a later change may depend on an earlier one, and applying them out of order can produce a state the sender never had.

Making ordering explicit is important because it constrains parallelism. The requirement limits how fast a receiver can go.

Why that makes the receiver a bottleneck

The sender can commit transactions concurrently across many connections while the receiver must respect their order, so a write-heavy sender can outpace a receiver that is not slower in any hardware sense.

Explaining the asymmetry is valuable because it is counterintuitive. Identical hardware does not mean identical throughput.

How receivers claw back some parallelism

Changes that cannot affect each other can be applied concurrently, which engines detect by examining what each transaction touched, recovering much of the lost throughput.

Covering the mitigation is necessary because it is widely implemented. The ordering requirement is on dependent changes rather than all changes.

Why the stream must be positionable

A receiver that disconnects needs to resume from where it stopped, so every point in the stream carries an identifier the receiver can report and the sender can seek to.

Introducing positions is necessary because retention depends on them. Resumption is what makes the arrangement operable.

Why the sender must then retain the stream

A receiver can only resume from a position the sender still has, so the sender must keep log records until every receiver has confirmed consuming them.

Marking retention here is deliberate because it causes the feature's worst outage. Retention is an obligation the sender takes on.

Why retention is the feature's sharpest hazard

A receiver that stops consuming while remaining registered makes the sender keep log records indefinitely, which fills the disk and stops the sender entirely.

Naming the hazard early is worthwhile because it is so severe. A disconnected replica can take down the primary.

What replication is for, beyond the obvious

Surviving the loss of a machine, serving reads elsewhere, moving between versions or engines, feeding other systems, and taking backups without loading the primary.

Listing the purposes is useful because the right form depends on which one applies. The purposes have different requirements.

Why the purpose determines the form

Surviving machine loss wants an identical copy, while feeding another system wants interpretable changes, and the two forms serve those two needs respectively.

Giving the mapping is the practical core of the section. The choice follows from the purpose rather than from preference.

Why many systems need both

A standby for failover and a logical stream for an analytics or search system are different jobs, and running both from one primary is normal rather than unusual.

Noting this is useful because the forms are often presented as alternatives. They are complementary in most real deployments.

Why neither form makes the primary faster

Both add work to the primary, in retention, in network transmission and sometimes in decoding, so replication is about availability and reach rather than throughput.

Being clear about this is important because the expectation is common. Reads can be offloaded and writes cannot.

Why offloading reads is still conditional

A receiver is behind the sender, so a read served there may not reflect a write just acknowledged, which breaks any workflow expecting to see its own change.

Previewing the consistency problem is right. The most common disappointment with read replicas is this one.

What to take from the mechanism

Three things. Replication is the write-ahead log, produced for durability, being read somewhere else, which is why no separate capture mechanism is needed and why the stream cannot silently miss a change. The log can be read two ways: verbatim as byte changes to identical pages, which makes a physical standby a perpetually recovering engine, or decoded into row-level changes, which is harder and buys filtering, transformation and non-identical receivers. And the sender must retain log records until every registered receiver has consumed them, which is the obligation behind the feature's worst outage.

Key Insight

Replication needs no mechanism of its own. Durability already requires the engine to write a complete, ordered record of every change before committing it, which means the stream replication needs was being produced anyway. That is why nothing has to be instrumented, why no triggers are required, and why the stream cannot miss a change without durability also being broken. Historical trigger-based capture schemes existed and were worse in every respect, which is worth remembering whenever a change-capture scheme is proposed that does not read the log.