A single engine's optimiser chooses access paths, join orders and operators. Once the data lives on many machines, every one of those choices acquires a second dimension, which is where the work happens and how much data has to move to make it possible.

What stays the same

The relational operators, the join algorithms and the execution model are all unchanged, because a node running a fragment of a plan is still an ordinary engine doing ordinary work.

Establishing the continuity first is the right opening. Most of what was learned still applies.

What is genuinely new

Data movement between nodes becomes an operator in its own right, and it is usually the most expensive one in the plan.

Naming the new element immediately is important. One addition explains almost everything that follows.

Why movement dominates the cost

Reading a page from local storage is fast and sending a row to another machine involves serialisation, a network crossing and deserialisation, so moving data costs far more per row than processing it.

Quantifying the difference is necessary. The asymmetry drives every decision.

What a plan looks like once it is distributed

It is divided into fragments, each of which runs in parallel on many nodes, with data movement operators marking the boundaries between them.

Introducing fragments is the structural vocabulary of the lesson. The plan becomes a graph of stages.

What the coordinating node does

It plans the query, distributes the fragments, tracks their progress and assembles the final result, which makes it a different role rather than just another worker.

Describing the coordinator is necessary. One node has a distinct job.

Why the coordinator must not become the bottleneck

Any plan that routes large volumes of rows through a single node loses the parallelism the system exists to provide, so the final fragment should receive little.

Giving the design rule is valuable. The funnel at the top must stay narrow.

Why that makes aggregation placement so important

A query reducing a billion rows to ten should do almost all of that reduction on the nodes holding the rows, so only the small result travels.

Previewing the key technique is right. Reduction belongs near the data.

What partition pruning does

It eliminates whole nodes or partitions from the plan when the query's predicates prove they can hold no matching rows.

Introducing pruning is necessary. The cheapest work is work not sent.

Why pruning is the single largest lever

A query touching one partition out of a hundred does a hundredth of the work, which no amount of operator tuning can match.

Giving the magnitude is valuable. The effect is in a different class from other optimisations.

Why pruning requires the predicate to be usable

The engine must be able to prove the partition cannot match, so a predicate wrapping the partition key in a function or comparing it to a value it cannot evaluate early defeats it.

Explaining the requirement is practical. The failure is a silent loss.

Why that failure is so costly here

Losing pruning on a single engine means reading more pages, while losing it in a distributed system means waking every node in the cluster.

Drawing the contrast is important. The same mistake is far more expensive.

What predicate pushdown does

Filters are evaluated on the nodes holding the data rather than after the rows have been moved, so the movement carries only rows that can matter.

Covering pushdown is necessary. Filtering before moving is the obvious discipline.

Why both pushdowns must happen before the movement

The whole value lies in the ordering, because a filter or a projection applied after the rows have crossed the network has already paid the cost it was meant to avoid.

Making the ordering explicit is worth a line of its own. The rewrite is about position in the plan rather than about the operation itself, and a plan that filters above a movement rather than below it is doing the same work for a far higher price.

Why projection pushdown matters as much

Sending only the columns the query needs can reduce the volume moved by an order of magnitude on a wide table, for no loss of anything.

Including projection is worthwhile because it is often forgotten. Narrowing is as valuable as filtering.

Why columnar storage compounds that

A column-oriented layout makes reading a subset of columns cheap locally as well as cheap to move, which is why analytical systems almost all use it.

Connecting to storage is useful. The two decisions reinforce each other.

What the optimiser now needs to know

Where each table's partitions live, how the data is distributed across them, and what it costs to move rows between nodes.

Listing the new inputs is necessary. The cost model gains a dimension.

Why statistics matter more here than locally

A misestimate that causes a bad local plan wastes one machine's effort, while one that causes an unnecessary movement of a large relation wastes the whole cluster's.

Explaining the amplification is valuable. The same error has a larger consequence.

Why the optimiser's hardest choice is placement

Deciding which node evaluates which part of the plan determines how much data moves, and that decision interacts with join order rather than following it.

Identifying the hard problem is right. Placement and ordering are coupled.

Why that coupling makes the search space large

Each join order admits several placements and each placement changes the cost of the remaining joins, so the number of candidate plans grows faster than in a single engine.

Explaining the difficulty is worthwhile. The optimisation problem is genuinely harder.

How systems cope with that

They restrict the search with heuristics, they prefer plans that avoid movement even when other plans look cheaper, and they increasingly adjust plans while running.

Giving the practical responses is useful. The approaches are pragmatic.

Why avoiding movement is a good heuristic

Movement costs are both large and poorly estimated, so a plan that avoids a shuffle is usually safer than one whose advantage depends on an accurate estimate.

Explaining the heuristic's value is important. Robustness beats optimality here.

Why a query touching one node should look like a local query

If the plan can be proved to need only one node's data, the whole thing can be sent there and run as an ordinary query with no movement at all.

Covering the degenerate case is valuable. The best distributed plan is often not distributed.

Why that case is worth designing for

A workload whose queries each touch one partition scales linearly with nodes, which is the behaviour operational systems want and analytical ones cannot have.

Drawing the practical conclusion is right. The access pattern decides the ceiling.

Why analytical queries are different in kind

They deliberately touch everything, so their plans always move data, and the goal becomes moving as little as possible rather than none.

Distinguishing the two workloads is necessary. The techniques differ accordingly.

What to take from the new dimension

Three things. The operators and algorithms are unchanged, and what is new is that data movement becomes an operator in its own right, costing far more per row than processing because it involves serialisation, a network crossing and deserialisation. The optimiser therefore gains a placement dimension coupled to join order, which enlarges the search space and makes statistics errors more expensive, so avoiding movement is a better heuristic than chasing the cheapest estimated plan. And pruning is the largest single lever available, because a predicate that eliminates partitions avoids waking nodes at all, where the same lost opportunity in a single engine merely reads more pages.