KafkaProducer
Callback vs Future
Apache Kafka
Programming
Asynchronous Communication

KafkaProducer Difference between `callback` and returned `Future`?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

KafkaProducer.send() gives you two ways to observe the result of a send: the returned Future<RecordMetadata> and an optional Callback. They are not competing mechanisms so much as two different ways to react to the same asynchronous completion event.

What the Returned Future Represents

Every call to send() returns a Future<RecordMetadata>. That future is a handle you can keep and inspect later. If you call get(), the calling thread blocks until the broker acknowledges the record or the send fails.

java
1import java.util.Properties;
2import java.util.concurrent.Future;
3import org.apache.kafka.clients.producer.KafkaProducer;
4import org.apache.kafka.clients.producer.ProducerRecord;
5import org.apache.kafka.clients.producer.RecordMetadata;
6
7public class FutureSendDemo {
8    public static void main(String[] args) throws Exception {
9        Properties props = new Properties();
10        props.put("bootstrap.servers", "localhost:9092");
11        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
12        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
13
14        try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
15            Future<RecordMetadata> future =
16                producer.send(new ProducerRecord<>("orders", "k1", "created"));
17
18            RecordMetadata metadata = future.get();
19            System.out.println(metadata.topic() + ":" + metadata.partition() + ":" + metadata.offset());
20        }
21    }
22}

This approach is useful when a later step really depends on the send result. It is also convenient at application boundaries, such as a command-line tool or a simple batch job, where blocking is acceptable.

What the Callback Adds

The callback lets you keep the send path non-blocking. Kafka invokes the callback once the record has been acknowledged or the send has failed.

java
1import java.util.Properties;
2import org.apache.kafka.clients.producer.Callback;
3import org.apache.kafka.clients.producer.KafkaProducer;
4import org.apache.kafka.clients.producer.ProducerRecord;
5import org.apache.kafka.clients.producer.RecordMetadata;
6
7public class CallbackSendDemo {
8    public static void main(String[] args) {
9        Properties props = new Properties();
10        props.put("bootstrap.servers", "localhost:9092");
11        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
12        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
13
14        try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
15            producer.send(new ProducerRecord<>("orders", "k1", "created"),
16                new Callback() {
17                    @Override
18                    public void onCompletion(RecordMetadata metadata, Exception exception) {
19                        if (exception != null) {
20                            exception.printStackTrace();
21                            return;
22                        }
23                        System.out.println(metadata.offset());
24                    }
25                });
26
27            producer.flush();
28        }
29    }
30}

This is usually the better choice for high-throughput producer code because the sender thread does not stop to wait for every record.

They Can Be Used Together

The important detail is that send(record, callback) still returns a Future<RecordMetadata>. The callback is an extra completion hook, not a replacement for the future.

That means you can attach lightweight side effects in the callback, such as logging or metrics, and still keep the future if you need to wait later at a controlled boundary. In practice, though, most code chooses one primary style to stay readable.

Choose Based on Control Flow

Use the future when the caller wants to decide when to wait, combine results, or propagate the send failure in a synchronous-looking flow. Use the callback when the application is naturally event-driven and should continue doing useful work while Kafka handles delivery in the background.

Another practical difference comes from the producer internals. Kafka's documentation notes that callbacks generally execute on the producer I/O thread, so callback bodies should stay fast. If you put heavy blocking work into the callback, you can delay delivery of other messages.

Kafka also guarantees callback order for records sent to the same partition. That matters when downstream code relies on ordered completion side effects, such as updating partition-specific metrics.

Common Pitfalls

  • Thinking the callback and future describe different send operations. They describe the same one.
  • Calling future.get() immediately after every send() and accidentally turning an asynchronous producer into a synchronous one.
  • Doing expensive work in the callback even though it generally runs on the producer I/O thread.
  • Forgetting to handle exceptions in both styles. Failed sends surface through the callback argument or through Future.get().
  • Assuming callback completion order is global across all partitions when the ordering guarantee is partition-specific.

Summary

  • 'send() always returns a Future<RecordMetadata>.'
  • A callback is an optional non-blocking completion hook for the same send operation.
  • 'Future.get() blocks, so it is best when the caller intentionally wants to wait.'
  • Callbacks are better for asynchronous, high-throughput flows, but they should stay lightweight.
  • Pick the style that matches your control flow, and remember you can technically use both on one send.

Course illustration
Course illustration

All Rights Reserved.