KafkaProducer Difference between `callback` and returned `Future`?
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
KafkaProducer.send() gives you two ways to observe the result of a send: the returned Future<RecordMetadata> and an optional Callback. They are not competing mechanisms so much as two different ways to react to the same asynchronous completion event.
What the Returned Future Represents
Every call to send() returns a Future<RecordMetadata>. That future is a handle you can keep and inspect later. If you call get(), the calling thread blocks until the broker acknowledges the record or the send fails.
This approach is useful when a later step really depends on the send result. It is also convenient at application boundaries, such as a command-line tool or a simple batch job, where blocking is acceptable.
What the Callback Adds
The callback lets you keep the send path non-blocking. Kafka invokes the callback once the record has been acknowledged or the send has failed.
This is usually the better choice for high-throughput producer code because the sender thread does not stop to wait for every record.
They Can Be Used Together
The important detail is that send(record, callback) still returns a Future<RecordMetadata>. The callback is an extra completion hook, not a replacement for the future.
That means you can attach lightweight side effects in the callback, such as logging or metrics, and still keep the future if you need to wait later at a controlled boundary. In practice, though, most code chooses one primary style to stay readable.
Choose Based on Control Flow
Use the future when the caller wants to decide when to wait, combine results, or propagate the send failure in a synchronous-looking flow. Use the callback when the application is naturally event-driven and should continue doing useful work while Kafka handles delivery in the background.
Another practical difference comes from the producer internals. Kafka's documentation notes that callbacks generally execute on the producer I/O thread, so callback bodies should stay fast. If you put heavy blocking work into the callback, you can delay delivery of other messages.
Kafka also guarantees callback order for records sent to the same partition. That matters when downstream code relies on ordered completion side effects, such as updating partition-specific metrics.
Common Pitfalls
- Thinking the callback and future describe different send operations. They describe the same one.
- Calling
future.get()immediately after everysend()and accidentally turning an asynchronous producer into a synchronous one. - Doing expensive work in the callback even though it generally runs on the producer I/O thread.
- Forgetting to handle exceptions in both styles. Failed sends surface through the callback argument or through
Future.get(). - Assuming callback completion order is global across all partitions when the ordering guarantee is partition-specific.
Summary
- '
send()always returns aFuture<RecordMetadata>.' - A callback is an optional non-blocking completion hook for the same send operation.
- '
Future.get()blocks, so it is best when the caller intentionally wants to wait.' - Callbacks are better for asynchronous, high-throughput flows, but they should stay lightweight.
- Pick the style that matches your control flow, and remember you can technically use both on one send.
Related reading
- KafkaSpout working example
- KafkaStream createTopic not respecting Kafka server's auto.create.topics.enable settings
- KafkaStreams - InconsistentGroupProtocolException
- KafkaStreams Getting Window Final Results
- Keras Tensorflow - Exception while predicting from multiple threads
- Keras utilises less CPU when number of workers grows and numpy generates a large array
- KafkaStreams serde exception
- KafkaTimeoutError Failed to update metadata after 60.0 secs

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.