The Resource Model, Storage and the Datastore

Topics Covered

Groups, Versions, Kinds and Subresources

The two triples, and why both exist

Subresources

Why the scale subresource matters

Namespaced and cluster-scoped

Conversion and the stored version

Reading the model from a cluster

Labels, annotations and selectors

Owner references and cascading deletion

Validation, defaulting and the structural schema

Every interaction with a cluster is a request against a typed resource, and the three-part naming plus the subresource mechanism explain a long list of behaviours that otherwise look arbitrary.

The two triples, and why both exist

Objects are named one way in the wire format and another way in the request path.

A kind triple is a group, a version and a kind, written as it appears inside an object: an empty group for the core resources, a named group for everything else, a version such as the first stable one or a beta, and a capitalised singular kind. A resource triple is a group, a version and a lowercase plural resource, which is what appears in the request path.

The mapping between them is not mechanical, which is why a discovery step exists. A client asks the server which resources exist, under which groups and versions, with which plural names, whether each is namespaced, and which verbs each supports. Everything a command line client does with short names and abbreviations is driven by that discovery response, which is also why a cluster with a broken aggregated interface makes unrelated commands slow.

Subresources

A subresource is a separate request path on the same object with its own permissions and semantics, and the set in use is small and important.

The status subresource separates observation from intent, so a controller may write status while a user writes specification, enforced by authorisation on different paths. The scale subresource exposes a replica count in a uniform shape, which is what lets one autoscaler drive deployments, stateful sets and custom resources without knowing their types. The binding subresource is how the scheduler assigns a node. The eviction subresource is how a drain requests a pod's removal subject to disruption budgets.

Others are not object fields at all but tunnelled operations: logs, port forwarding, attaching and executing. Those are served by the interface server proxying to the node agent, which is why they depend on node reachability while ordinary object reads do not.

The path names a resource and the object names a kind, and subresources are separate paths on one object with their own permissions.

Why the scale subresource matters

One example shows how much work a uniform subresource does.

An autoscaler needs to read a current replica count and write a new one. Without a uniform shape it would need to know that a deployment keeps replicas in one place, a stateful set in another, and a custom resource wherever its author chose. With the subresource, it issues the same request against any resource that declares one, and the resource's own server-side logic maps it onto the real fields.

This is why a custom resource becomes autoscalable by declaring a scale subresource with two field paths, with no change to the autoscaler. It is the clearest example in the system of an interface designed so that extensions get existing machinery for free.

Namespaced and cluster-scoped

The scoping decision is per resource and has consequences beyond naming.

A namespaced resource's name is unique within its namespace, it is listed by namespace or across all of them, and authorisation rules can be granted per namespace. A cluster-scoped resource has one name across the cluster, and granting access to it is necessarily cluster-wide.

That asymmetry decides design questions for extensions. A resource holding per-team configuration should be namespaced so that permission can be delegated. A resource holding cluster-wide policy must be cluster-scoped, which means only cluster administrators can change it, which is usually the intent. Getting this wrong is not fixable later without a migration, because the scope is part of the resource's definition.

Conversion and the stored version

A resource may be served at several versions while being stored at one, which is the mechanism behind interface evolution.

One version is marked as the storage version. A write at any served version is converted to it before storage, and a read at any served version is converted from it. For built-in resources the conversion is compiled in; for custom resources it is either a trivial mapping or a webhook the author provides.

The practical consequences are two. An object written at an old version and read at a new one comes back with the new version's shape, which is why a cluster can serve both during a migration. And changing the storage version does not rewrite stored objects, so they remain at the old version until something writes them, which is why a version cannot be removed from the served set until every stored object has been migrated.

Reading the model from a cluster

All of this is queryable, and the commands are worth having.

bash
1kubectl api-resources -o wide                  # plural, short names, scope, verbs
2kubectl api-versions                           # every served group and version
3kubectl get --raw='/apis/apps/v1' | jq '.resources[] | {name, namespaced, verbs}'
4kubectl explain deployment.spec.strategy --recursive
5kubectl get crd <name> -o jsonpath='{.spec.versions[*].storage}{"\n"}'

The verbs column is the most useful and least read. A resource supporting only create, such as the review resources used for authorisation checks, is not storage at all but a function call shaped like a resource, and noticing that saves a lot of confusion about why it cannot be listed.

Labels, annotations and selectors

Two metadata maps exist with different purposes, and using the wrong one has real consequences.

Labels are short identifying key-value pairs, validated for length and character set, and indexed in the sense that selectors match on them. Annotations are arbitrary key-value pairs with a much larger size allowance, not selectable, intended for information tools need rather than for identity.

The rules that follow are practical. Anything a selector must match is a label, which means services, deployments and policies all select on labels. Anything large, structured or purely informational is an annotation. And because a label change may move an object in or out of a selector's scope, changing a label on a running workload can reroute traffic or orphan a replica set, which is why label changes on live objects deserve more care than they usually get.

Owner references and cascading deletion

Objects form a graph, and the edge type that matters is ownership.

An object may list owner references naming the objects it belongs to. The garbage collector watches for owners that no longer exist and deletes their dependents, which is how deleting a deployment removes its replica sets and then their pods without the deployment controller doing anything.

Deletion has two propagation policies worth distinguishing. Foreground deletion keeps the owner present until dependents are gone, which is what makes a delete command appear to wait. Background deletion removes the owner immediately and cleans up afterwards, which returns faster and leaves a brief window where dependents exist with no owner. An orphan policy removes the owner and deliberately leaves the dependents, which is occasionally the right tool when adopting resources into new management.

Validation, defaulting and the structural schema

Three transformations happen to an object before it is stored, in a fixed order.

Defaulting fills unset fields with their declared defaults, which is why reading back a created object shows far more fields than were sent and why those values are then permanent, since the stored object has them explicitly. Validation rejects objects violating the schema or the type's rules. And for custom resources, a structural schema is required, meaning every field is declared with a type, which is what allows pruning: fields not in the schema are silently removed rather than stored.

The pruning behaviour surprises people. A field misspelled in a custom resource vanishes without an error, so the object is accepted and the setting never takes effect. Validation rules can be added to reject unknown fields instead, and doing so converts a silent failure into a clear one, which is almost always worth the strictness.

Level Expectations

A strong answer distinguishes the kind triple inside an object from the resource triple in the request path and explains that a discovery step maps between them. It then names the subresources that carry real semantics, status for separating observation from intent, scale for uniform replica control, binding for assignment, eviction for disruption-aware removal, and notes that logs and exec are tunnelled to the node agent rather than being object fields at all.