Continuous integration is a feedback mechanism, and almost every problem teams have with it comes from forgetting that and treating it as a gate. The original argument is narrow and strong: when many people change one codebase, the cost of discovering an incompatibility grows with the time between the change and the discovery, so integrate often and check immediately.

Everything that follows, the build servers, the pipeline definitions, the test suites and the merge policies, exists to shorten that interval. A pipeline that takes forty minutes has not failed at being a gate, it has failed at being feedback, and the consequences show up in behaviour rather than in any dashboard.

The interval is the product

The number that matters most in a CI setup is how long a developer waits between pushing and knowing. Everything else is secondary to it, including coverage, including the number of checks, and including how thorough the suite is.

This is because of what people do while waiting. Under about five minutes, they wait and stay in context. Between five and fifteen, they switch to something else and come back, which costs a context switch in each direction but keeps the change in working memory. Beyond about twenty minutes, they move on properly, and the feedback arrives after the mental model that produced the change has been swapped out.

The cost of that last case is not the twenty minutes. It is that fixing a failure in a change you no longer remember is several times more expensive than fixing one you still have in your head, and that people start batching changes to avoid paying the wait more often than necessary, which makes each failure harder to attribute.

Why slow pipelines produce worse code

The behavioural effects of a slow pipeline are predictable and they all point the same way.

People batch. Instead of pushing a small change, they accumulate three and push once, which means a failure now has three candidate causes. They stop running things locally, reasoning that the pipeline will tell them, which moves the cheapest feedback loop onto the most expensive infrastructure. They start merging on a green that is twenty minutes stale. And when a failure looks unrelated, they retry rather than investigate, because investigating costs another twenty minutes.

None of these is laziness. Each is a rational response to a loop that is too slow, and the only durable fix is to make the loop faster rather than to ask people to behave as though it were.

What the pipeline is actually checking

A useful pipeline answers a small number of questions, and it helps to name them because teams accumulate checks without ever asking which question each one answers.

Does it build. Do the unit tests pass. Does it integrate with the things it talks to. Does it meet the standards we have agreed, such as formatting, linting and licence rules. Is it safe to deploy, which covers dependency vulnerabilities and secrets. And does it still work once deployed somewhere realistic.

These have very different costs and very different failure rates, which is the basis of every optimisation in this lesson. Building and unit testing are fast and catch most mistakes. Full integration testing is slow and catches fewer. Deploying to a realistic environment is slowest and catches the fewest, while catching the ones that matter most.

The ordering follows from the costs

Once the checks are listed with their cost and their catch rate, the ordering is close to mechanical: run the cheap, high yield checks first and fail fast, then the expensive ones.

Formatting and linting take seconds and catch a surprising fraction of pull request comments. Compilation takes a minute and catches everything that cannot possibly work. Unit tests take a few minutes and catch most logic errors. Integration tests take longer and catch wiring. Deployment verification takes longest.

Running them in that order means the common failure is reported in ninety seconds rather than twenty minutes. It also means the expensive stages run only against changes that have already passed the cheap ones, which is where most of the infrastructure saving comes from.

Trunk based development and the thing it depends on

The integration argument has a direct consequence for branching, and it is worth stating because the two decisions are usually made separately.

If the cost of an incompatibility grows with the age of the divergence, then long lived branches are expensive by construction. A branch that lives for three weeks accumulates three weeks of divergence from everybody else's work, and the merge at the end is where all of that cost arrives at once, usually on the person least able to pay it.

Trunk based development, where changes land on the main branch within a day or two, is the branching strategy that follows from taking integration seriously. It is also the strategy that depends most heavily on a fast pipeline, because a slow loop makes frequent small merges painful and pushes people back toward batching. This is the loop that closes: the pipeline speed determines the branching strategy that is comfortable, and the branching strategy determines how much the pipeline speed matters.

Why the main branch must stay green

A broken main branch is not a local problem, which is what makes it worth more attention than it usually gets.

When the main branch is red, nobody can tell whether their own change is broken. Every subsequent failure has at least two candidate causes, and the signal that the pipeline exists to provide is gone for everybody at once. Worse, people start merging on top of the breakage, so the fix has to be made against a moving target.

The practice that follows is that fixing or reverting a broken main branch outranks everything else, and that reverting is usually the right first move rather than the last resort. A revert restores the signal for the whole team in minutes, and the original author can then fix the change properly without an audience.

What CI is not

Two things commonly get attached to CI that do not belong to it, and both make it worse.

It is not a quality gate in the sense of guaranteeing that what passes is good. A pipeline checks what somebody wrote a check for. Treating green as a statement about quality rather than about a specific set of assertions leads to the belief that adding more checks adds more quality, which is how pipelines become forty minutes long.

It is also not a substitute for running things locally. A loop that only exists on shared infrastructure is slow, contended and hard to debug. The healthiest setups make it possible to run most of the pipeline on a laptop, so the shared system is confirming rather than discovering.

One more thing a pipeline buys

There is a second return on continuous integration that is easy to miss because it is not about catching bugs at all.

A pipeline is the only place where the build is fully described. On a laptop, a build depends on whatever is installed, whatever environment variables happen to be set and whatever state previous builds left behind. In a pipeline, the build is described in a file, runs from a known starting state and either works or does not. That description is what makes the build reproducible, and reproducibility is what makes it possible to onboard somebody in an afternoon, to ship a release that somebody else can rebuild in a year, and to debug a failure by rerunning exactly what failed.

Teams that have this treat the pipeline definition as the authoritative description of how the software is built, and the laptop setup as a convenience that is expected to drift. Teams that do not have it discover, usually during an incident, that only one person can produce a release.

The honest baseline

Before optimising anything, it is worth measuring four numbers, because most teams do not have them and the answers are usually surprising.

The median and the ninety fifth percentile of pipeline duration. The pass rate on the main branch. The fraction of failures that are not caused by the change, which is the flakiness number. And the number of retries per merged change.

These four describe the loop from the developer's point of view rather than from the infrastructure's. A pipeline with a seven minute median and a thirty minute tail feels like a thirty minute pipeline, because the tail is what people plan around.