Starting a container with no networking options creates half a dozen kernel objects and a set of firewall rules, and every confusing networking symptom later is explained by one of them.

The objects created per container

Four things are created, and they can all be inspected from the host.

A network namespace, which gives the container its own interfaces, routing table, firewall rules and socket table. A virtual ethernet pair, which is two interfaces wired to each other, one placed in the namespace and named as the container's primary interface, the other left on the host. The host end is attached to a bridge, which is a software switch. And an address from the bridge's subnet is assigned to the container end, with a default route pointing at the bridge's own address.

That is the whole topology. A packet leaving the container goes down one end of the pair, appears on the other, is switched by the bridge, and is then subject to the host's routing and firewall.

How outbound traffic reaches the internet

The container's address is private to the host, so something must translate it, and that something is one firewall rule.

A rule in the post-routing stage matches traffic sourced from the bridge subnet and destined outside it, and rewrites the source address to the host's. Replies come back to the host and the connection tracking subsystem reverses the translation, returning them to the container.

Two consequences follow immediately. A remote service sees the host's address, not the container's, which is why per-container source address rules do not work. And outbound connections depend on connection tracking state, so a table exhausted by a high connection rate drops new connections with no application-level explanation.

How inbound traffic reaches a container

Publishing a port installs a second translation, in the opposite direction.

A rule in the pre-routing stage matches traffic to the host's chosen port and rewrites the destination to the container's address and port. A companion rule handles traffic originating on the host itself, which takes a different path through the firewall and would otherwise miss the first rule.

There is also a small userland process in some configurations that accepts connections on the host port and copies bytes to the container. It exists to handle cases the translation rules cannot, including loopback traffic and certain address family combinations, and it costs a copy in both directions. Disabling it is a throughput improvement and occasionally breaks loopback access, which is the tradeoff.

Container to container, on the same bridge

Traffic between containers on one bridge is switched rather than routed, which has one notable subtlety.

Both containers' host-side interfaces are on the same bridge, so a packet from one to the other is forwarded at layer two without touching the routing table or the translation rules. Latency is a few microseconds and throughput is near memory bandwidth.

The subtlety is reaching a container through the host's published port from another container on the same bridge. The packet goes out to the host's address, hits the pre-routing rule, and is sent back into the bridge it came from, which some configurations drop unless hairpin forwarding is enabled. The fix is almost always to use the container's own address or name instead of going out through the host.

What the namespace does and does not isolate

Being precise about the boundary prevents a class of wrong assumption.

Isolated per namespace: interfaces, addresses, routing tables, firewall rules, the socket table and the abstract socket namespace. Shared with the host: the connection tracking table, which is global and exhaustible; the kernel's networking parameters unless the specific one is namespaced; and the physical interface's queues and offload capabilities.

So two containers cannot see each other's sockets but can exhaust each other's connection tracking entries, and a kernel parameter tuned on the host may or may not apply inside a container depending on whether that parameter is namespaced. That asymmetry is behind a good share of mysterious networking behaviour.

Looking at it from the host

All of this is inspectable without any container-specific tooling, which is the fastest way to settle a question.

1docker inspect -f '{{.State.Pid}}' <ctr>              # the namespace entry point
2nsenter -t <pid> -n ip addr                           # interfaces inside the namespace
3nsenter -t <pid> -n ss -tnp                           # listening sockets inside
4ip -d link show type bridge && bridge link show       # the bridge and its ports
5iptables -t nat -S | grep -E 'MASQUERADE|DNAT'        # the two translations

The pattern worth internalising is entering the namespace with a host tool rather than needing tools inside the container, which works identically for a minimal image with no shell. The fourth and fifth commands answer most questions about why traffic is not arriving.

The resolver inside the container

Name resolution in a container is configured by a file the runtime writes, and knowing what goes in it explains several odd behaviours.

The runtime generates the resolver configuration inside the namespace. On a user-defined network it points at an address the runtime itself serves, which answers queries for container names and forwards everything else to the host's configured servers. On the default bridge it is a copy of the host's configuration, with loopback addresses filtered out because loopback inside the namespace is not the host.

That filtering is why a host using a local resolver on loopback produces containers with no usable server, which presents as every lookup failing while the host resolves fine. The fix is configuring the runtime with explicit servers, and recognising the symptom saves an hour.

Interface naming and the pair's other end

Matching a host interface to a container is a routine need during an incident and the obvious approach does not work.

The host side of the pair has a generated name with no relation to the container's. The link between them is an index: the container's interface records the index of its peer, and the host's records the same in reverse. Reading the peer index inside the namespace and finding the host interface with that index gives the mapping.

pid=$(docker inspect -f '{{.State.Pid}}' <ctr>)
nsenter -t $pid -n ip -o link show eth0 | sed -n 's/.*eth0@if\([0-9]*\).*/\1/p'
ip -o link | awk -F': ' '$1==<index>{print $2}'

This matters because per-interface statistics, packet captures and traffic shaping all happen on the host side, so the mapping is the first step for any of them.

Where to capture packets

Three capture points answer different questions, and choosing wrongly wastes the capture.

Capturing inside the namespace shows what the application sent and received, before any translation. Capturing on the host end of the pair shows the same traffic from the other side, which is the right place for a container the host tooling can reach without entering a namespace. Capturing on the physical interface shows the translated traffic as the network sees it.

The useful technique is capturing at two points simultaneously and comparing, because the difference localises a drop precisely. A packet present inside the namespace and absent on the physical interface was dropped by the host's firewall or routing, which is a much smaller search space than the whole path.

Throughput, and what the bridge path costs

Putting a number on the overhead settles the recurring argument about whether to use host networking for speed.

A packet crossing a virtual ethernet pair and a bridge is copied within the kernel and traverses the firewall hooks, which costs a few microseconds and some processor time per packet. For a service moving moderate traffic the difference against host networking is not measurable in any end-to-end metric. For a service moving millions of small packets per second, the per-packet cost becomes the bottleneck and the difference is large.

So the decision is driven by packet rate rather than by bandwidth. A workload shifting large payloads reaches line rate through a bridge without difficulty, because the per-packet cost is amortised over a large payload and offload features do the heavy lifting. A workload with tiny messages at a high rate does not, and that is the case where host networking or a driver bypassing the bridge is worth the isolation it gives up.