EmbeddedKafka is a powerful tool for Kafka-based application developers, easing the testing process by providing an in-memory Kafka instance. This test harness allows you to test your Kafka integration using real broker and server implementations, thus ensuring high fidelity test results. This article will explain how to use EmbeddedKafka to check received messages in unit tests, making your Kafka-driven application more robust and fault-tolerant.

What is EmbeddedKafka?

EmbeddedKafka is part of the larger Spring Kafka project, which includes full support for Kafka consumer and producer functionality in a Spring environment. It provides a convenient way to set up an isolated Kafka environment during unit tests, behaving like an actual Kafka setup for a more effective and realistic testing scenario. EmbeddedKafka typically simulates the broker and ZooKeeper services, automatically managing their lifecycle within the test's context.

Setting up EmbeddedKafka

To begin testing with EmbeddedKafka, you must include the spring-kafka-test library, which contains the necessary dependencies. Here is an example of how to add it to your Maven project:

1<dependency>
2    <groupId>org.springframework.kafka</groupId>
3    <artifactId>spring-kafka-test</artifactId>
4    <version>Your.Spring.Kafka.Version</version>
5    <scope>test</scope>
6</dependency>

In your unit test, you would use the @EmbeddedKafka annotation to initialize the embedded server. Here’s how you might set it up:

1@RunWith(SpringRunner.class)
2@SpringBootTest
3@DirtiesContext
4@EmbeddedKafka(partitions = 1, topics = { "topic1" })
5public class KafkaProducerTest {
6
7    @Autowired
8    private KafkaTemplate<String, String> kafkaTemplate;
9
10    @Test
11    public void testReceive() throws Exception {
12        kafkaTemplate.send("topic1", "Hello Kafka");
13
14        // Here would be your verification logic
15    }
16}

Receiving and Checking Messages

To check messages received on a Kafka topic, you need first to send a message to that topic, typically within your test code itself or through some triggered business logic. Then, you need to verify whether the message has been received correctly.

Here's how to set up a simple consumer to receive messages:

1@Autowired
2private EmbeddedKafkaBroker embeddedKafkaBroker;
3
4@Test
5public void testMessageReception() {
6    Map<String, Object> consumerProps = KafkaTestUtils.consumerProps("sender", "false", embeddedKafkaBroker);
7    Consumer<String, String> consumer = new DefaultKafkaConsumerFactory<String, String>(consumerProps).createConsumer();
8    embeddedKafkaBroker.consumeFromAnEmbeddedTopic(consumer, "topic1");
9
10    ConsumerRecords<String, String> replies = KafkaTestUtils.getRecords(consumer);
11
12    assertThat(replies.count(), is(1));
13    assertThat(replies.iterator().next().value(), is("Hello Kafka"));
14}

This snippet uses the KafkaTestUtils utility class to obtain properties suitable for a consumer and checks whether the consumer receives exactly one message with the content "Hello Kafka".

Summary of Key Points

Additional Considerations

When using EmbeddedKafka, be aware of resource utilization, especially in terms of memory and processor time, as running an embedded broker can be resource-intensive. Also, while EmbeddedKafka is excellent for integration and messaging tests, you should supplement it with mock-based tests for larger-scale integration scenarios due to potential performance implications.

Conclusion

EmbeddedKafka provides a robust choice for Kafka application developers aiming to ensure their applications can handle Kafka interactions adeptly. By following the setup and example tests outlined above, developers can improve their application quality through thorough, reliable testing, thereby reducing runtime errors and ensuring smoother deployments.