embedded Kafka
Spring Boot
Kafka testing
Java
software development

Simple embedded Kafka test example with spring boot

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Embedded Kafka is useful when you want an integration-style test for producer and consumer logic without standing up an external broker manually. In Spring Boot, the common pattern is to use spring-kafka-test, annotate the test with @EmbeddedKafka, and wire the embedded broker address into spring.kafka.bootstrap-servers.

Dependencies and Test Setup

At minimum, include the Kafka starter and the Kafka test dependency.

xml
1<dependencies>
2    <dependency>
3        <groupId>org.springframework.boot</groupId>
4        <artifactId>spring-boot-starter</artifactId>
5    </dependency>
6
7    <dependency>
8        <groupId>org.springframework.kafka</groupId>
9        <artifactId>spring-kafka</artifactId>
10    </dependency>
11
12    <dependency>
13        <groupId>org.springframework.kafka</groupId>
14        <artifactId>spring-kafka-test</artifactId>
15        <scope>test</scope>
16    </dependency>
17</dependencies>

Spring Boot's Kafka reference documentation shows two important details for tests:

  • use @EmbeddedKafka
  • point the embedded broker addresses at spring.kafka.bootstrap-servers

A Minimal Producer and Listener

First, define a tiny component that sends messages:

java
1import org.springframework.kafka.core.KafkaTemplate;
2import org.springframework.stereotype.Service;
3
4@Service
5public class MessageProducer {
6
7    private final KafkaTemplate<String, String> kafkaTemplate;
8
9    public MessageProducer(KafkaTemplate<String, String> kafkaTemplate) {
10        this.kafkaTemplate = kafkaTemplate;
11    }
12
13    public void send(String topic, String payload) {
14        kafkaTemplate.send(topic, payload);
15    }
16}

Then define a listener that captures the received message:

java
1import org.springframework.kafka.annotation.KafkaListener;
2import org.springframework.stereotype.Component;
3
4import java.util.concurrent.CountDownLatch;
5import java.util.concurrent.TimeUnit;
6
7@Component
8public class TestListener {
9
10    private final CountDownLatch latch = new CountDownLatch(1);
11    private volatile String payload;
12
13    @KafkaListener(topics = "orders")
14    public void listen(String message) {
15        this.payload = message;
16        latch.countDown();
17    }
18
19    public boolean awaitMessage() throws InterruptedException {
20        return latch.await(5, TimeUnit.SECONDS);
21    }
22
23    public String getPayload() {
24        return payload;
25    }
26}

This is intentionally simple. It gives the test something concrete to assert.

Embedded Kafka Test Class

Now create the integration test:

java
1import org.junit.jupiter.api.Test;
2import org.springframework.beans.factory.annotation.Autowired;
3import org.springframework.boot.test.context.SpringBootTest;
4import org.springframework.kafka.test.context.EmbeddedKafka;
5
6import static org.assertj.core.api.Assertions.assertThat;
7
8@SpringBootTest
9@EmbeddedKafka(
10    topics = "orders",
11    partitions = 1,
12    bootstrapServersProperty = "spring.kafka.bootstrap-servers"
13)
14class KafkaIntegrationTest {
15
16    @Autowired
17    private MessageProducer producer;
18
19    @Autowired
20    private TestListener listener;
21
22    @Test
23    void sendsAndReceivesMessage() throws Exception {
24        producer.send("orders", "created");
25
26        assertThat(listener.awaitMessage()).isTrue();
27        assertThat(listener.getPayload()).isEqualTo("created");
28    }
29}

The key annotation attribute is bootstrapServersProperty. It lets Spring Boot auto-configuration use the embedded broker instead of a real external cluster.

Why This Works

The embedded broker runs inside the test lifecycle, so your KafkaTemplate and @KafkaListener talk to a real Kafka broker instance, just one that happens to be local to the test process.

That makes the test stronger than a pure mock-based test because you exercise:

  • topic creation
  • serialization and deserialization wiring
  • producer send path
  • consumer listener wiring

It is still not a full production-cluster test, but it is an excellent middle ground.

Useful Variations

If your test does not need the full Spring Boot context, you can build narrower tests with Spring Kafka test utilities. If you do need to read raw records directly, KafkaTestUtils can help create consumer properties and poll records from the embedded broker.

For more complex systems, consider whether your test should verify:

  • listener behavior only
  • producer behavior only
  • full producer-to-consumer flow through the application

The setup should match the question the test is supposed to answer.

Common Pitfalls

The biggest pitfall is forgetting to map the embedded broker address into spring.kafka.bootstrap-servers. Then the application context may still try to connect to a real broker or to a missing default address.

Another pitfall is making the assertion before the listener has had time to consume the message. Kafka tests are asynchronous by nature, so use a latch or polling helper instead of assuming immediate delivery.

A third pitfall is overloading one integration test with every Kafka scenario in the application. Keep embedded broker tests focused so they stay readable and reasonably fast.

Finally, remember that embedded Kafka is great for application integration tests, but it is not a perfect substitute for environment-level testing against infrastructure that resembles production.

Summary

  • Use spring-kafka-test and @EmbeddedKafka for local Kafka integration tests in Spring Boot
  • Map the embedded broker into spring.kafka.bootstrap-servers
  • A small KafkaTemplate producer plus @KafkaListener consumer is enough for a good end-to-end example
  • Use latches or similar synchronization because message delivery is asynchronous
  • Embedded Kafka is ideal for application-level integration testing, not for replacing every production-like test

Course illustration
Course illustration

All Rights Reserved.