Introduction

Kafka record headers are key-value metadata attached to a record alongside its key and value. In Kafka Streams, headers travel with the record, but the high-level KStream DSL does not have a simple .to(..., headers=...) style API for creating them at the sink. If you want to add or modify headers inside a topology, you usually do it through the Processor API or a transformer that has access to the processing context.

The main design choice is where you want the header to be created. If the producer already knows the header values, the simplest option is often to set them before the record ever enters Kafka Streams. If the headers are derived from stream processing, then you modify them inside the topology.

Kafka Headers Are Part of the Record

A header is a pair of:

• header key as a String
• header value as a byte array

That means you are responsible for converting application data into bytes consistently.

In Streams, the current record's headers are exposed through the processor context. Mutating them changes the metadata forwarded with that record.

Add Headers Inside a Transformer

One common way is to use a transformer that gets a ProcessorContext and modifies context.headers().

1import org.apache.kafka.common.header.Headers;
2import org.apache.kafka.streams.KeyValue;
3import org.apache.kafka.streams.kstream.KStream;
4import org.apache.kafka.streams.kstream.Transformer;
5import org.apache.kafka.streams.kstream.TransformerSupplier;
6import org.apache.kafka.streams.processor.ProcessorContext;
7
8import java.nio.charset.StandardCharsets;
9
10public class HeaderAdder implements Transformer<String, String, KeyValue<String, String>> {
11    private ProcessorContext context;
12
13    @Override
14    public void init(ProcessorContext context) {
15        this.context = context;
16    }
17
18    @Override
19    public KeyValue<String, String> transform(String key, String value) {
20        Headers headers = context.headers();
21        headers.add("source", "streams-app".getBytes(StandardCharsets.UTF_8));
22        headers.add("processed", "true".getBytes(StandardCharsets.UTF_8));
23        return KeyValue.pair(key, value);
24    }
25
26    @Override
27    public void close() {
28    }
29}

Then wire it into the stream:

1KStream<String, String> input = builder.stream("input-topic");
2
3input.transform(() -> new HeaderAdder())
4     .to("output-topic");

The output records keep the modified headers.

When the Producer Should Set the Header Instead

If the header is already known before the data reaches Kafka Streams, set it in the producer rather than inside the Streams application. That is simpler and avoids embedding metadata-generation logic in the topology unnecessarily.

Use in-topology header mutation when the header depends on the stream computation itself, such as:

• routing decisions
• enrichment results
• validation status
• processing timestamps

That keeps the header close to the business logic that defines it.

Byte Encoding Matters

Headers are bytes, not typed Java objects. If one service writes UTF-8 strings and another service expects JSON or a number encoded some other way, the system becomes hard to debug.

A simple, explicit encoding pattern is usually enough:

byte[] encoded = Long.toString(System.currentTimeMillis())
    .getBytes(StandardCharsets.UTF_8);

The important part is consistency across producers and consumers.

Reading the Header Later

A downstream processor or consumer can inspect the headers and act on them.

1Headers headers = context.headers();
2if (headers.lastHeader("processed") != null) {
3    // react to header
4}

This is one reason headers are useful: they let you add metadata without changing the payload schema.

Be Careful with High-Level DSL Expectations

A common beginner mistake is expecting the pure KStream DSL to expose headers as ordinary record fields in every operator. Headers exist with the record, but not every DSL method makes them directly visible.

That is why the Processor API and transformer context are the normal tools when you need header access.

Common Pitfalls

The biggest mistake is expecting a sink call to accept headers directly the way a plain Kafka producer does. Another is forgetting that header values are raw bytes and need explicit encoding rules. Developers also often put header-generation logic in the wrong place, even when the producer already knows the metadata before Kafka Streams begins processing. Finally, if several services use the same header key with different byte encodings, the topology becomes hard to troubleshoot quickly.

Summary

• Kafka headers are record metadata stored as key-byte-array pairs.
• In Kafka Streams, add or modify headers through the Processor API or a transformer with context access.
• If the producer already knows the metadata, set headers before the record enters the Streams topology.
• Keep header encoding explicit and consistent across services.
• Use headers for metadata, not as an excuse to smuggle arbitrary payload structure into record metadata.