0%
Database Internals: Build the Engine
Storage Structures
Durability
Concurrency Control
Query Processing
Access Paths
Two-Phase Commit and Its Failure Modes
A single engine can make a transaction atomic because one component decides the outcome and one log records it. The moment two engines each hold part of a transaction, neither of those conditions holds, and everything difficult about distributed commit follows from trying to restore them.
What the problem actually is
Two engines each holding part of one transaction must either both commit or both abort, and neither can see the other's state at the instant it has to decide.
Stating the problem before any protocol is the right opening. The difficulty is in the simultaneity rather than in the mechanics.
Why committing locally is not enough
An engine that commits its own part has made that part permanent and visible, and nothing can take it back if the other engine then fails to commit.
Making this explicit matters because it is the naive approach. Local commit is an irreversible act.
Why aborting locally is not enough either
An engine that aborts its own part has discarded work the other engine may already have made permanent, which leaves the two disagreeing in the other direction.
Covering the symmetric case is worthwhile. Neither unilateral choice is safe.
Why the order of the two commits does not help
Whichever engine commits first is exposed for the interval before the second one does, so sequencing moves the window rather than closing it.
Addressing the obvious fix is necessary because it is the first thing people try. Ordering changes who is exposed.
What the requirement reduces to
Every participant must reach the same decision, and no participant may act on a decision that could still change.
Reducing the requirement to two clauses is the most useful framing available. Agreement and irreversibility are the whole of it.
Why that forces a separate decision step
If a participant decides for itself, the decisions can differ, so the decision has to be made once and communicated, which is a step that did not exist before.
Deriving the need for a coordinator rather than asserting it is better. The role follows from the requirement.
What a participant must be able to promise
That it can commit its part later, on request, without any possibility of refusing, which is a stronger statement than being able to commit now.
Introducing the promise is the conceptual core of the section. The ability to commit later is what makes agreement possible.
Why that promise is unusual
An ordinary transaction may fail at commit time for any number of reasons, so a participant offering this guarantee must have already resolved all of them.
Explaining the unusualness is valuable because it is where the cost lives. The promise is expensive to make.
What a participant must do to make it
Write everything needed to commit durably, acquire and hold every conflicting lock, and verify every constraint, so that nothing remains that could fail.
Listing the obligations makes the promise concrete. All the work of committing happens before the decision.
Why the locks must be held through the promise
Releasing them would let another transaction change the same rows, which could make the promised commit impossible, so the promise requires continued exclusion.
Explaining the lock requirement is essential because it causes the worst consequences later. Holding is not optional.
Why that makes the promise a liability
A participant holding a promise is blocking other work while waiting for an answer, so the duration of the wait is the duration of the blockage.
Previewing the cost is right. The protocol's worst behaviour is already visible here.
What the decision maker must guarantee
That its decision survives its own failure, so that a participant asking later gets the same answer it would have got before.
Introducing durability of the decision is necessary. The answer must outlive the machine that produced it.
Why the decision must be recorded before being sent
A decision sent and not recorded would be forgotten on failure while participants had already acted on it, which is exactly the disagreement being prevented.
Explaining the ordering constraint is important because it is subtle. Recording precedes telling.
Why this adds a durable write that was not there before
The decision is a new piece of state that must be made permanent, so the protocol costs at least one additional synchronous write beyond what the participants were doing anyway.
Quantifying the cost is useful. The latency is structural rather than implementational.
Why the number of round trips matters
Each participant must be asked, must answer, must be told, and should confirm, so commit latency includes several network crossings instead of none.
Giving the latency shape is practical. A distributed commit is not comparable to a local one.
Why the slowest participant sets the pace
The decision cannot be made until every participant has answered, so the whole transaction runs at the speed of the least responsive one.
Explaining the tail dependence is valuable. Adding participants worsens the expected latency more than linearly.
Why read-only participants can be skipped
A participant that changed nothing has nothing to commit, so once it has answered it can be released and need not be told the outcome.
Covering the optimisation is worthwhile because it is real and standard. Not every participant costs the same.
Why one-participant transactions collapse to a local commit
With a single participant there is nothing to agree with, so the decision can be delegated to it and the protocol disappears entirely.
Noting the degenerate case is useful. The cost is a function of genuine distribution.
Why this problem is unavoidable once state is split
Any arrangement where one logical change touches two independently recoverable stores faces it, whether the stores are databases, a database and a queue, or two services.
Generalising is important because the problem is usually met outside a database. The shape recurs everywhere.
Why it is nonetheless worth avoiding where possible
The protocol is correct and expensive and fragile under failure, so an arrangement that does not need it is usually better than one that implements it well.
Giving the honest judgement early is right. Avoidance is a legitimate strategy.
Why the problem is older than distributed systems
The same requirement arose between a database and a message queue on one machine, so the protocol predates the networks usually blamed for it.
Noting the history is useful because it corrects a common assumption. The difficulty is in the separation rather than the distance.
Why learning it is still necessary
Systems that avoid it are avoiding something specific, and the alternatives make sense only against an understanding of what they are declining to provide.
Justifying the lesson is worthwhile. The alternatives are defined by this protocol.
Why the database's own version matters operationally
Engines expose this protocol directly, and a transaction left in the promised state inside a real database causes outages of a particular and recognisable kind.
Previewing the operational content is right. The abstraction has a concrete manifestation.
What to take from the requirement
Three things. Atomicity across participants requires that every participant reach the same decision and that none act on a decision that could still change, which forces a separate decision step because self-decision permits disagreement. A participant must be able to promise an unconditional later commit, which means doing all the work of committing first and holding every conflicting lock while it waits, making the promise a liability for as long as the wait lasts. And the decision must be recorded durably before it is communicated, which adds a synchronous write and several network crossings that a local commit does not have.
The expensive part of distributed commit is not the messages, it is the promise. A participant that says it can commit later is asserting that nothing remains which could make the commit fail, so it has already written everything durably, verified every constraint and acquired every conflicting lock. It must then keep those locks until it is told the outcome, because releasing them would let another transaction make the promised commit impossible. That is why the protocol's cost scales with how long the answer takes rather than with how much data is involved.