Most PromQL confusion is not about syntax. It is about types. The language has four of them, every function accepts and returns specific ones, and almost every error message that begins "expected type" is really telling you that you have forgotten which one you are holding.

Four types, and only two that matter daily

An instant vector is a set of series, each contributing exactly one sample, all at the same evaluation timestamp. http_requests_total is an instant vector: at the moment of evaluation it is a list of label sets each with one number. This is the type you can graph, the type most functions return, and the type every binary operator expects.

A range vector is a set of series, each contributing many samples across a window. http_requests_total[5m] is a range vector, and it cannot be graphed. There is no single value to plot. Range vectors exist to be consumed by functions that reduce a window to a number, which is why rate, increase, max_over_time and their relatives are the only places a bracket should appear.

A scalar is a single number with no labels, such as the literal 2 or the result of scalar(...). A string is almost never seen outside of a label manipulation.

The single most common beginner error follows directly: writing http_requests_total[5m] into a dashboard panel and being told that a range vector cannot be used. The fix is not syntax, it is deciding what the window was for. If you wanted a rate, wrap it in rate. If you wanted the maximum observed in the window, wrap it in max_over_time. If you wanted all the raw points, you wanted a table rather than a graph.

Selectors, matchers and the metric name

A selector is a metric name plus zero or more label matchers, and the metric name is itself a matcher on the reserved __name__ label. That equivalence is not trivia: it explains why {__name__=~"http_.*"} is legal, why it is slow, and why a selector with no matcher at all is rejected rather than returning everything.

Four matchers exist. Equality, inequality, regex match and regex non match. The regex forms are fully anchored, so status=~"5.." matches 500 but not x500, and a leading .* is therefore redundant and costly. Inequality matchers have a subtlety worth knowing: {status!="500"} matches series where the label is present with another value and also series where the label is absent entirely, because an absent label is treated as the empty string.

That last rule is responsible for a specific class of silent bug. A query like sum(rate(http_requests_total{status!="200"}[5m])) quietly includes any series that has no status label at all, which on a mixed fleet is usually the subset instrumented by a different library. The defensive form is to match positively on what you want.

Offset, the at modifier, and the lookback

Two modifiers move the evaluation point. offset 1h shifts the selector back in time, so rate(x[5m] offset 1w) compares against last week, which is the idiomatic way to write a week over week panel. The @ modifier pins a selector to an absolute timestamp regardless of where the query is being evaluated, which makes x / x @ start() a way to normalise a graph against its own first value.

Underneath both sits the lookback delta, five minutes by default. An instant query for a series looks back up to five minutes for the most recent sample before giving up and returning nothing. This is why a metric scraped every fifteen seconds survives a brief scrape failure without a gap, and why a metric scraped every ten minutes appears to vanish between scrapes unless the lookback is raised. It is also why staleness markers matter: without one, a departed target keeps answering for five minutes.

What an evaluation actually does

It helps to picture one evaluation of a complete expression. Take sum by (service) (rate(http_requests_total{job="api"}[5m])) evaluated at 14:30:00.

First the selector runs against the index, intersecting the postings lists for __name__="http_requests_total" and job="api", producing a set of series references, say 800 of them. Then for each one the engine loads every sample with a timestamp in the half open interval from 14:25:00 to 14:30:00, which at a fifteen second interval is twenty samples each, sixteen thousand in total. Then rate runs per series, producing 800 single values. Then sum by (service) groups those 800 values by their service label and adds them, producing perhaps six values. The result is an instant vector of six series.

For a range query that whole process repeats once per step. A six hour range at a fifteen second step runs it 1,440 times. Holding this picture is what makes the cost arguments later in the lesson feel obvious rather than arbitrary.

Where people get stuck

Three habits separate people who are fluent from people who are guessing. The first is reading an expression inside out, starting at the selector and asking what type comes out of each layer. The second is checking the cardinality of the selector before anything else, because an expression that is correct over six series and ruinous over sixty thousand is the same expression. The third is using the expression browser's table view rather than the graph view while building, because the table shows labels, and labels are what every subsequent operator is actually working on.

Aggregation operators, and the labels they keep

Every aggregation operator takes a vector and returns a smaller one, and the only question that matters is which labels survive. sum(x) with no clause returns a single series with no labels at all, which is almost never what a dashboard wants. sum by (service) (x) keeps only service. sum without (instance) (x) keeps everything except instance, which is the right choice when a metric carries labels you have not thought about yet, because adding a new label to the metric does not silently change the grouping.

The choice between by and without is a judgment about what you expect to change. For an expression that will live in a recording rule for years, by states the output shape explicitly and is the safer default. For an ad hoc query collapsing a single dimension, without says what you meant more directly.

Eleven operators exist and five cover almost everything: sum, min, max, avg and count. topk and bottomk return the highest or lowest series rather than a reduced value, and they keep all labels, which makes them the right tool for "which ten pods are using the most memory" and the wrong tool for anything that feeds another operator. count_values is the oddity, counting how many series share each value, and it is the way to answer questions like how many instances are running each version.

quantile deserves a warning of its own. quantile(0.99, x) computes the ninety ninth percentile across the series in the vector, which is a percentile over instances and not over requests. It answers "which instance is at the ninety ninth percentile of this measurement", and it is unrelated to the request latency percentile people usually mean.

Reading an expression you did not write

A method that works on any expression: find the selectors first, because they tell you what data is involved and how much of it. Then find the brackets, because they tell you which functions are consuming windows. Then read outwards, naming the type at each layer. Then, before trusting it, run just the innermost selector in the table view and look at the label sets and the series count, since an expression is only as correct as its assumptions about what labels exist.