The Container Network Interface and Network Policy

Topics Covered

The Pod Network Model and Its Requirements

The four requirements

Why no translation matters

Address planning and what it caps

Address reuse and its hazard

The node's view of the model

What the model does not say

Host networking and when it is justified

The address family question

Where the model is extended

How to describe the model

What the node agent contributes

Before any plugin, there is a model. The orchestrator specifies what the network must provide and deliberately says nothing about how, which is why there are dozens of implementations with very different properties and identical observable behaviour for the common case.

The four requirements

One short list is the entire contract.

Every pod gets its own address. Pods on a node can communicate with all pods on all nodes without address translation. Agents on a node, such as the node agent and daemons, can reach all pods on that node. And a pod sees the same address for itself that others use to reach it.

That last requirement is the one with the most consequences and the one people overlook. It means a pod can take its own address from its interface and advertise it to peers, which is how clustered software discovers itself, and it rules out translation in the pod path.

Why no translation matters

One prohibition removes a long list of problems.

Address translation between pods would break self-identification, break protocols that carry addresses in their payload, make connection tracking a cluster-wide dependency rather than a per-node one, and make logs and audit trails ambiguous.

So the model is a flat address space in which every pod is directly addressable from every other. This is a stronger guarantee than most virtual machine networks provide, and it is the reason a cluster's address planning is a serious exercise rather than an afterthought: a flat space must be big enough and must not collide with anything it needs to reach.

Address planning and what it caps

One set of numbers decides the cluster's maximum size.

A pod address range is configured for the cluster, and each node is allocated a block from it, commonly a block of two hundred and fifty six addresses. The node's own block caps the pods that can run on it regardless of resource capacity, and the range divided by the block size caps the node count.

So a modest range with generous per-node blocks produces a cluster that cannot grow past a surprisingly small node count, and discovering that requires renumbering, which is disruptive. The arithmetic is worth doing explicitly at cluster creation, with a multiple of the expected maximum, because this is among the hardest things to change later.

Address reuse and its hazard

One reuse behaviour causes confusing incidents.

Pod addresses are recycled, often quickly, because a node's block is small and pods churn. A client holding a stale address, in a resolver cache or a connection pool, may find that address now belongs to a completely different pod, possibly in a different namespace.

That turns a stale reference from a connection failure, which is loud, into a successful connection to the wrong workload, which is quiet. It is the strongest argument for referring to workloads by service name rather than by address, and for bounding how long any component caches an address.

The self-identification requirement is the consequential one: it rules out translation in the pod path, which is why the address space is flat and must be planned.

The node's view of the model

One arrangement is common to nearly every implementation.

Each pod has a network namespace with an interface and a route sending traffic to the node. The node has a route for every other node's block, pointing either at that node directly or at an encapsulation interface. Traffic to a local pod is routed within the node; traffic to a remote pod leaves via the node's route for that block.

The result is that the node is a router for its own pods, and the interesting differences between implementations are entirely in how the node learns and reaches other nodes' blocks.

What the model does not say

One set of omissions explains the variety of plugins.

Nothing is specified about encapsulation, routing protocol, address allocation strategy, policy enforcement mechanism, encryption, multiple interfaces per pod, address family support or observability. All are implementation choices.

That is deliberate: specifying the mechanism would have frozen the design at the state of the art of the time. The cost is that choosing a plugin is a real decision with real consequences, and that two clusters satisfying the model identically can differ by a factor in latency, in maximum size and in what policy they can express.

Host networking and when it is justified

One escape hatch bypasses the model entirely.

A pod may use the node's network namespace, in which case it has the node's addresses and ports and no isolation. It is appropriate for node-level agents that must observe or configure the node's network, and for a small number of latency-critical workloads where the extra hop is measurable.

The costs are substantial: port conflicts between pods on a node, no policy enforcement since there is no pod address to select, the pod reachable at the node's address from anywhere the node is, and a compromised pod with a view of the node's network. It should be a deliberate, reviewed exception rather than a performance habit.

The address family question

One decision affects everything downstream.

A cluster may run a single family, or both. Dual-stack means every pod has an address of each family, each from its own range, with services and policies in both.

The operational points are that both ranges must be planned and either can exhaust alone, that a workload listening on one family only will be unreachable on the other even though it has an address there, that policies must be written for both or one family is unprotected, and that resolution returns both kinds so query volume roughly doubles. Enabling the second family after the fact is possible but intrusive, which again argues for deciding at creation.

Where the model is extended

One mechanism attaches additional interfaces.

Some workloads need a second interface on a separate network: a dedicated storage network, a high-throughput path, or a provider network. A meta-plugin that invokes several plugins per pod provides this, giving the pod its ordinary cluster interface plus others.

This is genuinely useful and should be used sparingly, because policy enforcement, service routing and observability generally apply to the primary interface only. A pod with a second interface can have traffic that no policy governs and no tooling sees, which is a legitimate choice for a telecommunications workload and an accident waiting to happen in an ordinary application cluster.

How to describe the model

One summary is enough for most conversations.

The orchestrator requires a flat address space where every pod has a unique address, pods reach each other across nodes without translation, node agents reach local pods, and a pod sees its own address as others do. Everything about how that is achieved is a plugin's choice, the node acts as a router for its own pods, and the per-node block size and cluster range together cap how large the cluster can ever be.

What the node agent contributes

One division of labour is worth knowing because it decides who to blame.

The network plugin creates the interface and assigns the address. The node agent writes the pod's resolver configuration, from the cluster's resolver service address and the pod's resolver policy, and it is the node agent that reports the pod's address back into the pod object's status.

So a pod with an address but no working resolution is a node agent or resolver problem, not a plugin problem, and a pod whose status shows no address means the plugin's result was never recorded even if the interface exists. Those two observations separate the two owners immediately, which is why reading the pod's status alongside its interface is the right first move rather than either alone.

Level Expectations

A strong answer states the four requirements and then names the consequence of the fourth: because a pod must see the same address others use to reach it, translation in the pod path is ruled out, which is what forces a flat address space. From there the practical point is that the cluster range divided by the per-node block caps the node count, and renumbering later is disruptive.