Namespaces and cgroups

Topics Covered

What a Container Actually Is

A process with three things done to it

Where the isolation comes from

Namespaces are referenced by file

What containers do not isolate

Why this is cheap

Doing it by hand, to see the pieces

What the image contributes

Three layers of software above the kernel

What the tools inside report

There is no container object in the Linux kernel. A container is a normal process that was started with several isolation features switched on, and every one of those features can be used independently, which is both why containers are cheap and why their isolation has the specific gaps it has.

A process with three things done to it

What people call a container is a process running with its own namespaces, constrained by a control group, and with a root filesystem pointing at an image's contents.

Namespaces decide what the process can see: which other processes, which filesystem, which network interfaces, which hostname. Control groups decide how much it can use: processor time, memory, input and output bandwidth, process count. The root filesystem decides what binaries and libraries it finds.

Those three mechanisms predate containers and are each independently useful. A container runtime is largely a program that arranges them in the right order and then executes the entrypoint, which is why a minimal runtime is a few thousand lines rather than a hypervisor.

Where the isolation comes from

The kernel call that creates a process takes flags, and those flags are what create namespaces.

Passing a namespace flag means the new process gets a fresh instance of that subsystem rather than sharing the parent's. Omitting it means the new process shares. So a process can have its own network stack while sharing the filesystem, or its own process table while sharing the network, and both arrangements are used in practice.

Two more calls complete the picture: one detaches the calling process from a namespace it currently shares, and another joins a process to an existing namespace given a file descriptor. That second one is what makes it possible to attach a debugging process to a running container's network namespace, which is the most useful operational consequence of the whole design.

There is no container object in the kernel, only a process with namespaces, a control group and a root filesystem, each of which can be switched on independently.

Namespaces are referenced by file

Each namespace a process belongs to appears as a symbolic link under its entry in the process filesystem, and that representation is what makes namespaces manageable.

Reading those links shows the namespace identifiers, so two processes sharing a namespace show the same identifier. Comparing them is the quickest way to answer whether two containers share a network stack, which is exactly what a sidecar arrangement does deliberately.

A namespace exists as long as something references it, which can be a process or an open file descriptor. That is how a namespace is kept alive with no process in it, which is the mechanism behind a pod's network namespace surviving the restart of every container in it.

What containers do not isolate

The list of things shared with the host is short and important, and every item on it is a source of real surprise.

The kernel itself is shared, so there is one scheduler, one memory manager and one set of kernel modules. Kernel version and kernel parameters are the host's, so a container cannot load a module or set most tunables. The system clock is shared except where a specific namespace is used. Device access is mediated but the devices are the host's.

Most consequentially, a kernel vulnerability is a host vulnerability, because the attack surface is the same kernel for every container on the machine. This is the reason container isolation is weaker than virtualisation, and the reason untrusted workloads are usually given a virtual machine boundary instead.

Why this is cheap

A container starts in milliseconds because nothing is being emulated and nothing is being booted.

Creating namespaces is a few kernel data structures. Creating a control group is writing to a filesystem. Setting up the root filesystem is mounting a union of existing layers, which is metadata rather than copying. Then the process is executed normally, scheduled by the same scheduler as everything else, with no additional layer between it and the hardware.

So the overhead of running a process in a container is close to zero for processor work, small and measurable for network and filesystem work where an extra layer is involved, and the memory cost is the process plus a few kilobytes of kernel bookkeeping rather than an operating system image.

Doing it by hand, to see the pieces

The clearest way to understand a container is to build one with shell commands, which takes four of them.

Create a process in new namespaces, change into a directory holding an extracted image, make that directory the root, create a control group and put the process in it, and set a limit. At that point the process is a container in every meaningful sense, and no container software was involved.

bash
1unshare --pid --net --mount --uts --ipc --fork --mount-proc /bin/sh
2# then inside: hostname isolated; ps aux; ip addr
3mkdir /sys/fs/cgroup/demo && echo 200000 100000 > /sys/fs/cgroup/demo/cpu.max
4echo $$ > /sys/fs/cgroup/demo/cgroup.procs

Doing this once removes most of the mystery, and it is also how you reason about what a runtime did when something is configured in a way you did not expect.

What the image contributes

The image is the least interesting of the three pieces and the one people think about most, which is worth correcting.

An image is a stack of filesystem layers plus a small configuration document naming the entrypoint, the default environment, the declared ports and the user to run as. None of it is privileged and none of it changes what the kernel will allow; it is packaging. The runtime reads the configuration, assembles the layers into a single view with a union filesystem, and uses that as the root.

So the same image runs with wildly different isolation depending on how it was started, which is why a security review of a workload has to look at the runtime configuration rather than the image. An image built carefully and run with the privileged flag is not a careful deployment, and an ordinary image run with everything dropped can be a very good one. The image decides what code is present; the runtime decides what that code is allowed to do.

Three layers of software above the kernel

The word runtime covers three different things, and the distinction shows up whenever something fails.

At the bottom is the component that actually makes the kernel calls, creating namespaces, writing the control group files and executing the entrypoint. Above it sits a supervisor that pulls images, manages their storage, tracks container state and restarts things, exposing an interface for the layer above. At the top sits whatever orchestrates across machines, deciding what runs where.

The practical value of keeping them separate is diagnostic. A container that fails to start with a kernel-level error is a problem at the bottom layer and the message usually names the syscall. A container that never starts because its image cannot be fetched is a problem at the middle layer. A container that was never scheduled at all is a problem at the top layer and nothing on the node will show it. Knowing which log to read first comes directly from knowing which layer owns which job.

What the tools inside report

One consequence of the shared kernel deserves stating on its own, because it causes more wasted time than anything else here.

The process filesystem a container reads is the host's, so the processor count, the memory total, the load average and the uptime reported inside are all the machine's rather than the container's. A tool showing sixty-four cores and five hundred gigabytes inside a container limited to two cores and four gigabytes is not broken; it is reading the only source it has.

That makes almost all capacity reasoning from inside the container wrong, and it is why older language runtimes sized themselves catastrophically. The correct source is the control group files, which report the real limits, the real usage and the throttling counters. Knowing to read those instead of the familiar tools is the most immediately useful practical skill in this lesson, and it is the first thing to do when a containerised workload's resource behaviour does not match its configuration.

Level Expectations

A strong answer states that there is no container object in the kernel, only a process with namespaces, a control group and a root filesystem, and that each of the three is independently switchable. It then names what is shared: one kernel, one scheduler, one memory manager, the host's kernel version and parameters. That last point is the one that matters in an interview, because it is why a kernel vulnerability is a host vulnerability and why untrusted workloads get a virtual machine boundary instead of a container.