Time-Series Databases: Why a Generic Row Store Falls Over at Metric Scale

Metrics have a shape. They arrive every second from every machine, they are appended in timestamp order, they are almost never updated, they are queried by time window, and they get less valuable with age. A general-purpose row store is wrong for almost every line in that description.

Run a million metrics through Postgres and each data point becomes a tiny row. The B-tree on the primary key keeps re-splitting. Index maintenance burns IO that should be serving queries. Compression is mediocre because the storage layout still interleaves columns. You can make it work for a quarter, then you are buying disks instead of fixing the architecture.

Time-series engines are built around the access pattern. The on-disk layout is columnar per metric, so a query that asks for one metric over an hour reads one contiguous block instead of skipping across rows. Timestamps are delta-encoded: store the first timestamp and a sequence of small deltas, then bit-pack them. A metric scraped every ten seconds compresses 10 to 100 times better than the same data in a generic row store. InfluxDB, Prometheus's TSDB, and TimescaleDB's hypertables all rely on this layout, with hypertables sitting on top of Postgres so you keep SQL.

Retention is part of the design, not an afterthought. Keep raw resolution for a week, one-minute rollups for ninety days, one-hour rollups for a year. The query layer picks the right resolution based on the window. Tiered storage moves cold blocks to object storage so hot RAM stays small.

The production failure I see most often is cardinality, not volume. A Prometheus instance scrapes a Kubernetes cluster where every deploy rotates pod IPs and container IDs. Each (pod_ip, container_id) pair becomes a new series. The in-memory head block tracks every active series. After three days the series count crosses ten million, the head block balloons, the process OOM-kills, and your dashboards go dark right when you need them.

The fix is upstream. Drop high-cardinality labels at the relabel step before ingest. Replace pod_ip with a stable identifier like deployment or service. If you genuinely need per-pod granularity, move to a backend with native cardinality limits and horizontal sharding: Mimir, Cortex, or VictoriaMetrics. Treat label sets as a schema, not a free-form bag.

Time-series engines are fast because they say no to the access patterns they were not built for. The discipline is on the writer. Pick the engine for the workload, hold the line on label hygiene, and the system stays cheap for years instead of months.