Every deployment system in existence does two separate jobs, and most of the confusion about GitOps comes from treating them as one. The first job is deciding what should be running: which image, which replica count, which configuration values, which environment. The second job is making that true in a live system that is already running something else.

Imperative delivery mixes the two. A pipeline step runs a command that says "replace the running thing with this", and the record of what should be running exists only as the history of commands that were issued. If you want to know the intended state of production, you read the log of the last successful pipeline run and hope nobody ran anything by hand afterwards.

Declarative delivery separates them. The intended state lives in files, the files live somewhere versioned, and a separate process continuously compares the live system to those files and closes the gap. Nothing ever runs a deployment command; a file changes, and a reconciler notices. That is the whole idea, and the interesting consequences all follow from the separation rather than from any particular tool.

The two halves, and which one you are automating

When a team says they have automated deployment, they almost always mean the second half: the mechanics of getting bytes onto machines. That half is genuinely easy now. Any orchestrator will do a rolling update correctly, and a pipeline step that calls it is four lines of configuration.

The first half is the one that causes incidents. Nobody can say with certainty what is running in production, because the answer is distributed across pipeline logs, hotfixes applied by hand during an incident, a scaling change somebody made from a console, and a configuration value that was edited live and never written down. Declarative delivery is a way of making the first half have a single answer, and it is worth adopting for that reason alone even if the mechanics of the second half do not change at all.

What declarative means here, precisely

Declarative does not mean written in YAML. It means the artefact describes the desired end state rather than the steps to reach it, and it means applying the same artefact twice has the same result as applying it once. That second property, idempotence, is what makes continuous reconciliation safe. A reconciler that runs every two minutes against a description of steps would run those steps every two minutes. A reconciler running against a description of state does nothing when the state already matches.

The practical test is whether you can take the artefact, delete the running system entirely, apply the artefact to an empty cluster, and arrive at the same place. If the answer is no, something important is only recorded in the sequence of past actions, and that something is the part that will be missing when you need to rebuild.

The reconcile loop, step by step

The loop is short enough to state completely. A controller running inside the target environment holds a reference to a source of truth, usually a repository and a path at a revision. On an interval, and ideally also on a webhook, it fetches that source. It renders the source into a set of concrete resources, since the files on disk are usually templates or overlays rather than final manifests. It reads the current state of the corresponding live resources. It computes a difference. If the difference is empty, it records that the system is synced and sleeps. If it is not empty, it applies the difference, and then it reports what it did.

Two things in that list deserve attention. The render step is where most of the complexity in real setups lives, because it is where templating, environment overlays and generated values interact. And the report step is what turns the loop into something you can operate: a system that silently reconciles is indistinguishable from a system that has stopped, so the health of the reconciler has to be a first class signal.

Push and pull, and why the direction matters

In the push model the pipeline holds credentials for the target environment and reaches into it. In the pull model a controller inside the environment reaches out to the repository and no external system holds write access to the cluster.

The pull direction has three consequences that are easy to undervalue. Credentials for production stop existing outside production, which removes an entire category of incident in which a compromised build runner becomes a compromised cluster. Environments that are not reachable from the network where your pipeline runs become deployable, which matters for anything behind a customer firewall. And the deployment stops depending on the pipeline being up, since the controller keeps reconciling whether or not anything else is working.

The cost is that the pipeline no longer knows whether the deployment succeeded. It knows the commit landed. Closing that gap, so that a human or a pipeline can learn the outcome, is work that the pull model creates and that teams routinely forget to do.

Drift, and what the loop does about it

Drift is any difference between the repository and the live system that did not come from the repository. Somebody scaled a deployment during an incident. An admission controller mutated a field. A second controller owns the same field and keeps writing its own value. A human edited a config map at two in the morning and told nobody.

A reconciler can respond to drift in exactly three ways, and choosing between them per resource is one of the real design decisions in this model. It can report the drift and do nothing, which keeps a human in the loop and lets emergency changes survive. It can overwrite the drift on its next pass, which guarantees the repository is the truth and will cheerfully undo a fix somebody applied during an outage. Or it can treat the drift as a signal to stop reconciling that resource until a person decides, which avoids both failure modes at the cost of needing somebody to decide.

Everything else that belongs in the repository

Once the mechanism exists, the interesting question becomes what else should be described this way. Workloads are the obvious answer and the least valuable one, because they were already being deployed by something. The components that benefit most are the ones that nobody owns and everybody assumes are there.

Cluster add ons are the first category: the ingress controller, the certificate issuer, the metrics agent, the log shipper, the policy engine. These are installed once by whoever set up the environment and then live for years with nobody able to say which version is running or why a flag was set. Putting them under the same loop means their upgrades go through review and their configuration is visible, and it means a rebuilt cluster comes back with them.

Policy is the second. Network policies, admission rules, quotas and role bindings are exactly the kind of thing that gets loosened during an incident and never tightened again, and they are the kind of thing where a reconciler that overwrites drift is unambiguously the right setting.

Operational configuration is the third and the one teams reach last. Alert rules, dashboards, service level objectives and on call routing can all be described as resources, which means a change to an alert threshold gets reviewed like a code change and can be reverted when it turns out to be noisy. The argument against is that it slows down a quick dashboard edit, and the argument for is that nobody can explain why the threshold is four hundred.

A reasonable sequence is to start with workloads because the value is immediate, add add ons next because the rebuild case justifies it, then policy, then operational configuration once the team is comfortable. Trying to do all four in the first month produces a large repository that nobody trusts and a reconciler that is permanently unsynced.

What this does not fix

It is worth being blunt about the limits, because the model is often sold as more than it is. Declarative delivery does not make your rollouts safer; the progressive delivery mechanisms from the previous lesson are what do that, and they work the same way under either model. It does not remove the need to test, and a bad manifest reconciles just as reliably as a good one. It does not give you a rollback for anything that is not described in the repository, which notably includes your data.

What it does give you is an answer to "what should be running" that does not depend on anyone remembering, a diff for every change, and a cluster that repairs itself when something knocks a resource out of shape. Those are worth a great deal, and they are not the same as safety.