Kafka Consumer
Spring Integration
Message Headers
Apache Kafka
Software Debugging

Kafka Spring Integration Headers not coming for kafka consumer

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

When Kafka headers do not appear on the Spring consumer side, the problem is usually not that Kafka "lost" them. The more common causes are: the producer never sent them, the consumer is reading the payload but not the headers API, or the Spring message conversion and header mapping setup is not the one you assumed.

First Distinguish Kafka Headers from Spring Message Headers

Kafka records can carry native record headers. Spring can map those into Spring Messaging headers, but they are not the same thing conceptually.

That means you should ask:

  • did the producer send Kafka record headers at all
  • is the consumer reading native Kafka headers or mapped Spring headers
  • is there a message converter or header mapper involved

If any one of those assumptions is wrong, the headers may appear to be missing even though the record itself is fine.

Verify Headers on the Producer Side

Start by proving the producer actually sent headers.

java
1ProducerRecord<String, String> record =
2    new ProducerRecord<>("orders", "key1", "{\"id\":123}");
3
4record.headers().add("traceId", "abc-123".getBytes());
5kafkaTemplate.send(record);

If the producer never added headers, nothing on the consumer side can recover them later.

Read Headers Explicitly in @KafkaListener

In a Spring Kafka listener, you can access mapped headers directly with @Header.

java
1import org.springframework.kafka.annotation.KafkaListener;
2import org.springframework.kafka.support.KafkaHeaders;
3import org.springframework.messaging.handler.annotation.Header;
4
5@KafkaListener(topics = "orders", groupId = "billing")
6public void listen(
7        String payload,
8        @Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
9        @Header(name = "traceId", required = false) byte[] traceId) {
10
11    System.out.println(payload);
12    System.out.println(topic);
13    System.out.println(traceId == null ? null : new String(traceId));
14}

If you are only consuming the payload parameter, you are not automatically printing or inspecting headers.

Inspect the Native ConsumerRecord When in Doubt

When debugging, the clearest view is often the raw Kafka record.

java
1import org.apache.kafka.clients.consumer.ConsumerRecord;
2import org.springframework.kafka.annotation.KafkaListener;
3
4@KafkaListener(topics = "orders", groupId = "billing")
5public void listen(ConsumerRecord<String, String> record) {
6    record.headers().forEach(header ->
7        System.out.println(header.key() + "=" + new String(header.value()))
8    );
9}

If the headers are visible here but not in your mapped Spring method parameters, the problem is in message conversion or header mapping, not in Kafka transport.

Spring Mapping and Converters Matter

When Spring turns a Kafka record into a Spring Message<?>, header mapping rules decide what gets exposed and how. Custom converters, custom container factories, or integration adapters can all affect that mapping.

So if native headers exist on ConsumerRecord but do not appear as expected in your higher-level message handling, inspect:

  • custom RecordMessageConverter
  • custom header mapper
  • integration flow adapters
  • any filtering or transformation layer between Kafka and your handler

This is especially important in Spring Integration flows, where the incoming Kafka record may be adapted into another message abstraction before your code sees it.

Batch and Type Details Can Change Access Patterns

Header handling also differs depending on whether you use:

  • record listeners versus batch listeners
  • payload parameters versus Message<?>
  • typed headers versus raw byte arrays

A header value sent as bytes should usually be read as bytes unless you have explicit conversion logic. Assuming it will automatically arrive as a String is a frequent source of confusion.

Common Pitfalls

The most common mistake is checking only the payload and then concluding headers are missing. Another is assuming Spring-specific headers and Kafka record headers are identical without understanding the mapping layer. Developers also forget to verify the producer side first, which can waste time debugging a consumer that is behaving correctly. Finally, custom converters or integration adapters can unintentionally hide or remap headers if their configuration is not aligned with how the listener reads them.

Summary

  • First verify that the producer actually sent Kafka record headers.
  • In Spring consumers, access headers explicitly with @Header or inspect the raw ConsumerRecord.
  • Distinguish native Kafka headers from mapped Spring Messaging headers.
  • If native headers exist but mapped ones do not, inspect message conversion and header mapping.
  • Debug the transport layer first, then the Spring abstraction layer on top of it.

Course illustration
Course illustration

All Rights Reserved.