Java
Kafka 8.2 API
Message Production
Coding
Software Development

How can I produce messages with Kafka 8.2 API in Java?

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

If by "Kafka 8.2 API" you mean the Java producer API introduced in Kafka 0.8.2, the core workflow is still familiar today: create a KafkaProducer, configure serializers and brokers, build ProducerRecord instances, then send them asynchronously or synchronously.

What matters most is choosing the right producer settings for reliability. A producer that merely sends bytes is easy to write; a producer that behaves well under retries, broker failures, and batching needs a little more care.

Create a Minimal Producer

A producer needs bootstrap brokers plus key and value serializers.

java
1import org.apache.kafka.clients.producer.KafkaProducer;
2import org.apache.kafka.clients.producer.ProducerRecord;
3import java.util.Properties;
4
5public class SimpleProducer {
6    public static void main(String[] args) {
7        Properties props = new Properties();
8        props.put("bootstrap.servers", "localhost:9092");
9        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
10        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
11
12        KafkaProducer<String, String> producer = new KafkaProducer<>(props);
13        producer.send(new ProducerRecord<>("events", "user-1", "hello kafka"));
14        producer.flush();
15        producer.close();
16    }
17}

That is enough to publish a message to the events topic, assuming the broker is reachable and the topic exists or auto-creation is enabled.

Use the Modern Producer Pattern Correctly

Even in older Kafka client generations, the new producer API was designed around asynchronous send. send() queues the record and returns immediately with a Future.

If you want to know whether the broker acknowledged the write, add a callback:

java
1import org.apache.kafka.clients.producer.Callback;
2import org.apache.kafka.clients.producer.RecordMetadata;
3
4producer.send(new ProducerRecord<>("events", "user-1", "payload"),
5    new Callback() {
6        @Override
7        public void onCompletion(RecordMetadata metadata, Exception exception) {
8            if (exception != null) {
9                exception.printStackTrace();
10                return;
11            }
12            System.out.println("topic=" + metadata.topic()
13                + " partition=" + metadata.partition()
14                + " offset=" + metadata.offset());
15        }
16    }
17);

That is usually better than assuming the send succeeded just because no exception was thrown immediately.

Make Reliability Settings Explicit

For real applications, set the producer properties intentionally instead of relying on defaults you have not reviewed.

java
1Properties props = new Properties();
2props.put("bootstrap.servers", "localhost:9092");
3props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
4props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
5props.put("acks", "all");
6props.put("retries", 3);
7props.put("linger.ms", 5);
8props.put("batch.size", 16384);

Why these matter:

  • 'acks=all asks the broker side for stronger durability semantics'
  • 'retries helps with transient failures'
  • 'linger.ms allows small batching windows for better throughput'
  • 'batch.size controls how much data can accumulate per partition batch'

If you need strict ordering guarantees, review how retries interact with in-flight requests in the client version you are using.

Sending Synchronously When You Must

Kafka producers are optimized for async use, but you can block for the broker result by calling get() on the returned Future.

java
1try {
2    RecordMetadata metadata = producer
3        .send(new ProducerRecord<>("events", "key", "value"))
4        .get();
5
6    System.out.println(metadata.offset());
7} catch (Exception e) {
8    e.printStackTrace();
9}

This is useful for tests, scripts, or workflows where the next step depends on confirmed delivery. It reduces throughput, so do not use it blindly in hot paths.

Topic, Serialization, and Keys

Keys are optional, but they matter. Kafka uses the key to choose a partition when a custom partitioner is not involved. If you want all events for one user to stay in order, give them the same key.

java
producer.send(new ProducerRecord<>("events", "user-42", "login"));
producer.send(new ProducerRecord<>("events", "user-42", "purchase"));

Both records are likely to land in the same partition, which preserves relative ordering for that key.

Serialization matters too. Strings are a fine starting point, but production systems often move to JSON, Avro, or Protobuf once the payload structure becomes important.

Common Pitfalls

The biggest mistake is confusing the version number in the question. Kafka 0.8.2 introduced the newer Java producer API, but many modern examples target later client versions. The core pattern is similar, but configuration defaults may differ.

Another common problem is forgetting flush() or close() in short-lived programs. If the process exits immediately after send(), buffered records may never be transmitted.

People also ignore callbacks and assume a send always worked. Kafka may reject or retry a write after the send() call has already returned.

Finally, avoid producing without thinking about keys, acknowledgments, and serializers. Those choices determine ordering, durability, and interoperability.

Summary

  • Use KafkaProducer with explicit broker and serializer settings.
  • Build ProducerRecord objects and send them asynchronously by default.
  • Add callbacks or Future.get() when you need delivery feedback.
  • Set reliability-related properties such as acks and retries deliberately.
  • Use keys when ordering by entity matters.
  • Always flush() and close() the producer in short-lived programs.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.