Introduction

kafkacat, now commonly referred to as kcat, is excellent for quick Kafka debugging and ad hoc message production. Producing records from a file is easy, and attaching headers is easy too, but only when you are clear about whether the same headers should apply to every record or each line needs different metadata.

That distinction matters because the normal -H flag is invocation-wide. It is a good fit for shared headers, but not a full structured file format for per-message header variation.

Producing One Message Per Line

The basic file-driven pattern is line-based production with -P and -l:

kcat -P -b localhost:9092 -t my_topic -l messages.txt

If messages.txt contains:

{"event":"login","user":"u1"}
{"event":"logout","user":"u2"}

then each line becomes one produced message value.

The older executable name kafkacat still appears in many examples, but current packages and documentation usually use kcat.

Adding the Same Headers to Every Produced Record

If all records from that command should share the same headers, pass repeated -H flags:

1kcat -P \
2  -b localhost:9092 \
3  -t my_topic \
4  -l messages.txt \
5  -H source=test-run \
6  -H format=json

This attaches source=test-run and format=json to every record produced during that invocation. For testing, replaying a fixture, or sending a batch with shared metadata, this is usually all you need.

What -l Does and Does Not Mean

The -l option is only about line-delimited message values. It does not automatically interpret each line as a complex header-plus-body structure.

That is the source of a lot of confusion. Developers often assume a file such as this will somehow drive both headers and payload:

header1=value1 {"event":"login"}
header1=value2 {"event":"logout"}

Out of the box, kcat does not infer that format for you. Unless you add your own preprocessing, the file is just message data.

When Each Message Needs Different Headers

If every record needs its own headers, the simplest reliable pattern is to define your own input format and call kcat once per record.

For example, suppose the input file is tab-delimited:

source-a	{"event":"login"}
source-b	{"event":"logout"}

Then a shell loop can split the file and attach the correct header for each line:

1while IFS=$'\t' read -r source body; do
2  printf '%s\n' "$body" | \
3    kcat -P -b localhost:9092 -t my_topic -H "source=$source"
4done < input.tsv

This is less compact than a single -l invocation, but it is explicit and easy to reason about. Each line produces one record, and the header value is derived from that same line.

Verifying the Result

Whenever headers matter, consume the records back and inspect them. Producing blindly is not enough, because a command can succeed while still sending the wrong metadata.

A simple verification loop might look like this:

kcat -C -b localhost:9092 -t my_topic -f 'Headers: %h\nValue: %s\n'

The exact output format can be adjusted, but the important part is checking that both the payload and headers match what you intended to send.

When a Script Is Better Than a One-Liner

If the file format becomes complicated, kcat may stop being the right abstraction. It is still a great tool for quick operational work, but once each record has its own keys, headers, timestamps, or validation rules, a small producer script in Python, Go, or Java is often easier to maintain.

That does not mean kcat is weak. It just means shell pipelines have a complexity limit.

Common Pitfalls

The most common mistake is assuming -l implies a built-in per-line header format. In normal use, it only means one message value per line.

Another mistake is using -H and expecting each header to apply only to one record in a multi-line batch. Repeated -H flags apply to the produced records of that command invocation.

Developers also forget to verify the produced output. A successful command tells you only that records were sent, not that the headers were correct.

Finally, be aware of naming drift. Many older posts say kafkacat, while current installations often expose kcat. The behavior is conceptually the same, but the command name may differ.

Summary

• Use -P with -l file to produce one message per line from a file.
• Use repeated -H key=value flags when all messages should share the same headers.
• If each message needs different headers, preprocess the file and invoke kcat per record.
• Consume the records back to verify that both payloads and headers are correct.
• Treat kafkacat as the historical name and kcat as the modern command name.