Kafka Streams
SerializationException
LongDeserializer
Data Size Error
Programming Bugs

Kafka streams error SerializationException Size of data received by LongDeserializer is not 8

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

LongDeserializer expects exactly eight bytes because a Kafka Long is encoded as an eight-byte signed integer. When Kafka Streams throws SerializationException: Size of data received by LongDeserializer is not 8, it means the bytes on the topic do not match the serde configuration your stream application is using.

Core Sections

Why the error happens

Kafka stores raw bytes. Serializers and deserializers are only agreements between producers and consumers. If the producer writes a string, JSON payload, Avro record, or null-ish unexpected value and the stream reads it with LongDeserializer, the consumer tries to interpret those bytes as an eight-byte integer and fails.

Common causes include:

  • producer and consumer use different value serdes
  • the key uses LongDeserializer even though keys are strings
  • old topic data was written with a previous format
  • the topic contains tombstones or malformed records you did not account for

Check the configured serdes first

Kafka Streams can inherit default serdes from StreamsConfig, and those defaults often cause confusion when one topic differs from the rest of the topology.

java
1Properties props = new Properties();
2props.put(StreamsConfig.APPLICATION_ID_CONFIG, "my-app");
3props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
4props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
5props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.Long().getClass());

If a topic actually contains JSON values or string values, that default is wrong for that branch of the topology.

Use explicit serdes when reading the topic.

java
1KStream<String, Long> stream = builder.stream(
2    "events",
3    Consumed.with(Serdes.String(), Serdes.Long())
4);

That makes the type assumption visible in code instead of hiding it in configuration.

Verify what is really on the topic

Do not debug only from the Streams code. Inspect the topic itself. If the topic has historical data written by a different producer or serializer, your application may be correct for new writes but still crash on old records.

For example, this producer writes a proper long value:

java
ProducerRecord<String, Long> record =
    new ProducerRecord<>("events", "k1", 42L);

But this producer writes a string that looks numeric while still using string serialization:

java
ProducerRecord<String, String> record =
    new ProducerRecord<>("events", "k1", "42");

The second record is only two bytes for the characters 4 and 2, not eight bytes for a binary long.

Handle migrations and mixed data carefully

A common real-world case is schema evolution or a topic reused for a new format. If old records were strings and new records are longs, a fresh consumer group reading from the beginning will hit old incompatible data.

Options include:

  • create a new topic for the new format
  • reset offsets to skip incompatible historical data if that is acceptable
  • add a preprocessing layer that reads bytes and converts safely

Trying to force one deserializer across mixed historical formats usually creates brittle code.

Consider custom or schema-based serialization

If you use Avro, Protobuf, JSON Schema, or custom bytes, then LongDeserializer is simply the wrong choice. The fix is not to manipulate the bytes until they happen to be length eight. The fix is to use the serde that matches the actual record format.

java
1KStream<String, MyEvent> stream = builder.stream(
2    "events",
3    Consumed.with(Serdes.String(), myEventSerde)
4);

Choose the serde based on how the producer serialized the data, not based on what Java type you wish the bytes represented.

Common Pitfalls

  • Assuming numeric-looking data such as the string "42" can be read by LongDeserializer, even though it is not stored as an eight-byte binary long.
  • Relying on default stream serdes and forgetting that a specific topic branch uses a different format.
  • Reusing a topic after changing serialization format and then being surprised when older records break new consumers.
  • Fixing the consumer only, without checking whether the producer is serializing keys or values differently from what the stream expects.
  • Treating the problem as random data corruption when it is more often a straightforward serde mismatch.

Summary

  • 'LongDeserializer requires exactly eight bytes because that is the binary representation of a Kafka long.'
  • The error almost always indicates a mismatch between the topic’s actual bytes and the serde configuration.
  • Check the producer format, the topic history, and the explicit serdes used in Kafka Streams.
  • Numeric text is not the same thing as a serialized long.
  • If the topic format changed, consider a new topic or a migration path instead of forcing one deserializer onto mixed data.

Course illustration
Course illustration

All Rights Reserved.