Introduction

In Kafka Streams, deserialization happens before your topology logic sees a record. If deserialization fails, the application must decide whether to stop or skip the bad record. That decision is controlled by the deserialization exception handler configuration.

Why This Handler Matters

Kafka Streams consumes bytes from Kafka and converts them into keys and values using configured serdes. If the incoming bytes do not match the expected format, deserialization throws an error before the record enters your stream logic.

Without a clear policy, a single malformed record can halt an otherwise healthy application. In other workloads, silently skipping bad data would be worse than stopping. The right handler depends on your error budget and data-integrity requirements.

Built-In Handler Options

Kafka Streams provides two common built-in policies:

• 'LogAndFailExceptionHandler'
• 'LogAndContinueExceptionHandler'

LogAndFailExceptionHandler is conservative. It logs the problem and stops processing. This is usually the right choice when data loss is unacceptable.

LogAndContinueExceptionHandler logs the problem and skips the record. This is useful when the stream should stay alive even if some events are malformed.

Configure a Built-In Handler

You configure the handler in the Streams application properties.

1import java.util.Properties;
2import org.apache.kafka.streams.KafkaStreams;
3import org.apache.kafka.streams.StreamsBuilder;
4import org.apache.kafka.streams.StreamsConfig;
5import org.apache.kafka.streams.errors.LogAndContinueExceptionHandler;
6
7Properties props = new Properties();
8props.put(StreamsConfig.APPLICATION_ID_CONFIG, "orders-app");
9props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
10props.put(
11    StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
12    LogAndContinueExceptionHandler.class.getName()
13);
14
15StreamsBuilder builder = new StreamsBuilder();
16builder.stream("orders-input");
17
18KafkaStreams streams = new KafkaStreams(builder.build(), props);
19streams.start();

That one property changes what the application does when a record cannot be deserialized.

Implement a Custom Handler

If the built-in choices are too blunt, implement DeserializationExceptionHandler yourself.

1import java.util.Map;
2import org.apache.kafka.clients.consumer.ConsumerRecord;
3import org.apache.kafka.streams.errors.DeserializationExceptionHandler;
4import org.apache.kafka.streams.errors.DeserializationExceptionHandler.DeserializationHandlerResponse;
5import org.apache.kafka.streams.processor.ProcessorContext;
6
7public class CustomDeserializationHandler implements DeserializationExceptionHandler {
8    @Override
9    public DeserializationHandlerResponse handle(
10        ProcessorContext context,
11        ConsumerRecord<byte[], byte[]> record,
12        Exception exception
13    ) {
14        System.err.println("Failed to deserialize record at topic " +
15            record.topic() + ", partition " + record.partition() +
16            ", offset " + record.offset());
17
18        return DeserializationHandlerResponse.CONTINUE;
19    }
20
21    @Override
22    public void configure(Map<String, ?> configs) {
23    }
24}

Then register it just like a built-in handler:

1props.put(
2    StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
3    CustomDeserializationHandler.class.getName()
4);

What a Custom Handler Should Usually Do

A custom handler is a good place to add structured logging, metrics, or alert-friendly context. For example, you might record the topic, partition, offset, and exception type so operators can find the exact bad event.

What a custom handler should not do is try to repair arbitrary broken payloads blindly. If records are malformed often enough to need complex recovery, the schema and producer contracts usually need attention upstream.

Choosing Between Fail and Continue

Use fail-fast behavior when:

• every record matters
• bad input indicates a serious upstream bug
• silent loss would be unacceptable

Use continue behavior when:

• occasional bad records are expected
• downstream consumers can tolerate missing malformed events
• availability matters more than strict completeness

The key is to choose intentionally. "Continue" is not free. It means dropping data.

Do Not Confuse This With Processing Exceptions

Deserialization handlers deal only with failures while converting bytes into objects. They do not handle exceptions thrown later by your topology code, processors, or downstream services. Those failure modes need separate handling strategies.

That distinction matters operationally because a stream can deserialize successfully and still fail later for completely different reasons.

Common Pitfalls

The biggest mistake is using LogAndContinueExceptionHandler without good metrics or logging. The application keeps running, but bad data disappears unless you can measure it.

Another issue is choosing fail-fast behavior in a noisy environment without understanding the restart impact. A single recurring malformed record can repeatedly crash the app.

Developers also sometimes assume the deserialization handler covers all stream exceptions. It only covers serde failures before records enter the topology.

Finally, if many records are failing deserialization, do not hide the issue forever with a continue policy. That is usually a producer, schema, or compatibility problem that deserves a real fix.

Summary

• Kafka Streams deserialization handlers decide what happens when incoming bytes cannot be converted into objects.
• Built-in options include fail-fast and log-and-continue behavior.
• A custom handler can add logging, metrics, and application-specific policy.
• Skipping bad records improves availability but can introduce silent data loss.
• Deserialization handling is only one part of stream error handling, not the whole story.