How to catch deserialization error in Kafka-Spring?

Kafka-Spring

Deserialization Error

Error Handling

Spring Boot

Java Programming

How to catch deserialization error in Kafka-Spring?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

In Spring Kafka, a deserialization failure usually happens before your @KafkaListener method is invoked. That is why a try/catch block inside the listener does not handle the problem: the listener argument never existed because the record could not be deserialized in the first place.

Why Listener-Level `try/catch` Is Too Late

Consider a listener like this:

java

1@KafkaListener(topics = "orders")
2public void listen(OrderCreated event) {
3    try {
4        System.out.println(event.getOrderId());
5    } catch (Exception ex) {
6        // too late for deserialization failures
7    }
8}

If the record value cannot be converted into OrderCreated, Spring cannot even call listen with a valid argument. The failure occurs in the consumer pipeline before your business method runs.

That means the fix belongs in consumer configuration and container error handling, not only in application logic.

Wrap the Real Deserializer With `ErrorHandlingDeserializer`

Spring Kafka provides ErrorHandlingDeserializer so low-level deserialization failures can be surfaced into the framework's error-handling flow.

java

1import java.util.HashMap;
2import java.util.Map;
3import org.apache.kafka.clients.consumer.ConsumerConfig;
4import org.apache.kafka.common.serialization.StringDeserializer;
5import org.springframework.context.annotation.Bean;
6import org.springframework.context.annotation.Configuration;
7import org.springframework.kafka.core.ConsumerFactory;
8import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
9import org.springframework.kafka.support.serializer.ErrorHandlingDeserializer;
10import org.springframework.kafka.support.serializer.JsonDeserializer;
11
12@Configuration
13public class KafkaConsumerConfig {
14
15    @Bean
16    public ConsumerFactory<String, OrderCreated> consumerFactory() {
17        Map<String, Object> props = new HashMap<>();
18        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
19        props.put(ConsumerConfig.GROUP_ID_CONFIG, "orders-group");
20        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
21        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ErrorHandlingDeserializer.class);
22        props.put(ErrorHandlingDeserializer.KEY_DESERIALIZER_CLASS, StringDeserializer.class);
23        props.put(ErrorHandlingDeserializer.VALUE_DESERIALIZER_CLASS, JsonDeserializer.class);
24        props.put(JsonDeserializer.VALUE_DEFAULT_TYPE, OrderCreated.class.getName());
25        props.put(JsonDeserializer.TRUSTED_PACKAGES, "com.example.events");
26
27        return new DefaultKafkaConsumerFactory<>(props);
28    }
29}

Without that wrapper, deserialization failures are harder to route through Spring's higher-level error strategy.

Add a Container-Level Error Handler

Once Spring can surface the failure properly, configure a DefaultErrorHandler for the listener container.

java

1import org.springframework.context.annotation.Bean;
2import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
3import org.springframework.kafka.core.ConsumerFactory;
4import org.springframework.kafka.listener.DefaultErrorHandler;
5import org.springframework.util.backoff.FixedBackOff;
6
7@Bean
8public ConcurrentKafkaListenerContainerFactory<String, OrderCreated> kafkaListenerContainerFactory(
9        ConsumerFactory<String, OrderCreated> consumerFactory) {
10
11    ConcurrentKafkaListenerContainerFactory<String, OrderCreated> factory =
12            new ConcurrentKafkaListenerContainerFactory<>();
13
14    factory.setConsumerFactory(consumerFactory);
15    factory.setCommonErrorHandler(new DefaultErrorHandler(new FixedBackOff(0L, 0L)));
16    return factory;
17}

This example does not retry. That is often a sensible default for malformed payloads, because a broken message rarely becomes valid on the next poll.

Dead-Letter Topics Are Often the Best Production Answer

For production systems, preserving the failed record is usually better than simply discarding it. A dead-letter topic gives you an audit trail and a place to inspect bad messages later.

java

1import org.springframework.context.annotation.Bean;
2import org.springframework.kafka.core.KafkaTemplate;
3import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
4import org.springframework.kafka.listener.DefaultErrorHandler;
5import org.springframework.util.backoff.FixedBackOff;
6
7@Bean
8public DefaultErrorHandler errorHandler(KafkaTemplate<Object, Object> kafkaTemplate) {
9    DeadLetterPublishingRecoverer recoverer = new DeadLetterPublishingRecoverer(kafkaTemplate);
10    return new DefaultErrorHandler(recoverer, new FixedBackOff(0L, 0L));
11}

That gives you a clean policy:

malformed record arrives
deserialization fails
container error handler takes over
record is published to a dead-letter topic for investigation

Separate Payload Problems From Configuration Problems

Not every deserialization error means the producer sent bad JSON. The issue could also be:

the wrong target class
missing trusted packages
incompatible schema versions
producer and consumer using different serialization formats

That is why simply saying "catch the exception" is not enough. You need to know whether the record itself is corrupt or the consumer is configured incorrectly.

Common Pitfalls

The biggest mistake is trying to catch deserialization failures inside the listener method. By then the framework has already failed earlier in the pipeline.

Another mistake is configuring DefaultErrorHandler without ErrorHandlingDeserializer. The error handler can only help after the failure is surfaced in a way the container understands.

People also forget trusted-package and target-type settings on JsonDeserializer, which can make a valid message look like a broken one.

Finally, be careful with retries. If a payload is malformed, retrying the same message over and over often just blocks the partition and creates noise.

Summary

Spring Kafka deserialization errors usually happen before the listener method starts.
Listener-level try/catch is therefore not enough.
Use ErrorHandlingDeserializer to expose failures to the container.
Pair it with DefaultErrorHandler to skip, retry, or dead-letter bad records.
Dead-letter topics are often the safest production strategy for malformed messages.

How to catch deserialization error in Kafka-Spring?

Master System Design with Codemia

Introduction

Why Listener-Level try/catch Is Too Late

Wrap the Real Deserializer With ErrorHandlingDeserializer

Add a Container-Level Error Handler

Dead-Letter Topics Are Often the Best Production Answer

Separate Payload Problems From Configuration Problems

Common Pitfalls

Summary

Why Listener-Level `try/catch` Is Too Late

Wrap the Real Deserializer With `ErrorHandlingDeserializer`