Spring Framework
Kafka
MessageListenerContainer
Programming
Software Development

Spring Kafka MessageListenerContainer

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In Spring Kafka, the MessageListenerContainer is the component that owns the consumer loop. It creates and manages Kafka consumer instances, polls records from brokers, and dispatches those records to your listener code according to the container's concurrency, acknowledgment, and error-handling configuration.

What the Container Actually Does

It helps to think of the listener container as the runtime engine behind a consumer, not as the business logic itself. Its responsibilities include:

  • creating the Kafka consumer
  • subscribing or assigning topics and partitions
  • polling records
  • invoking your listener method
  • managing commits and acknowledgments
  • coordinating error handling and retries

That is why most Spring Kafka tuning discussions eventually come back to the container configuration.

Two Common Container Types

Spring Kafka exposes two closely related container types:

  • 'KafkaMessageListenerContainer for a single consumer thread'
  • 'ConcurrentMessageListenerContainer to run several child containers in parallel'

The concurrent container is what most applications use through ConcurrentKafkaListenerContainerFactory.

A Typical Factory Configuration

You usually do not instantiate containers manually for every listener. Instead, you configure a factory.

java
1import org.apache.kafka.clients.consumer.ConsumerConfig;
2import org.apache.kafka.common.serialization.StringDeserializer;
3import org.springframework.context.annotation.Bean;
4import org.springframework.context.annotation.Configuration;
5import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
6import org.springframework.kafka.core.ConsumerFactory;
7import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
8
9import java.util.HashMap;
10import java.util.Map;
11
12@Configuration
13public class KafkaConsumerConfig {
14
15    @Bean
16    public ConsumerFactory<String, String> consumerFactory() {
17        Map<String, Object> props = new HashMap<>();
18        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
19        props.put(ConsumerConfig.GROUP_ID_CONFIG, "demo-group");
20        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
21        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
22        return new DefaultKafkaConsumerFactory<>(props);
23    }
24
25    @Bean
26    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() {
27        ConcurrentKafkaListenerContainerFactory<String, String> factory =
28            new ConcurrentKafkaListenerContainerFactory<>();
29        factory.setConsumerFactory(consumerFactory());
30        factory.setConcurrency(3);
31        return factory;
32    }
33}

The factory later produces the listener containers used by @KafkaListener.

How @KafkaListener Uses It

When you define a listener, Spring wires it to a container created by the configured factory.

java
1import org.springframework.kafka.annotation.KafkaListener;
2import org.springframework.stereotype.Component;
3
4@Component
5public class OrderListener {
6
7    @KafkaListener(topics = "orders", groupId = "demo-group")
8    public void handle(String payload) {
9        System.out.println("Received: " + payload);
10    }
11}

The annotation is concise, but under the hood the container is still doing the real consumer management work.

Why Acknowledgment Mode Matters

One of the most important container behaviors is when offsets are committed. Depending on your setup, commits may happen automatically or manually.

For manual acknowledgment:

java
1import org.springframework.kafka.annotation.KafkaListener;
2import org.springframework.kafka.support.Acknowledgment;
3import org.springframework.stereotype.Component;
4
5@Component
6public class ManualAckListener {
7
8    @KafkaListener(topics = "orders")
9    public void handle(String payload, Acknowledgment acknowledgment) {
10        System.out.println(payload);
11        acknowledgment.acknowledge();
12    }
13}

That can be useful when offset management must align with successful downstream processing.

Error Handling and Retries

Containers also define how failures are handled. If a listener throws an exception, the container can route that through an error handler, retry strategy, or dead-letter publishing configuration.

That means application reliability is not just about the listener method. It is also about how the container responds when the listener fails.

Common Pitfalls

The biggest mistake is thinking the listener method alone defines consumer behavior. In reality, concurrency, commits, retries, and error handling all live in the container configuration.

Another issue is setting container concurrency higher than the available topic partitions. Extra threads do not help if Kafka cannot assign them useful work.

Developers also forget that manual acknowledgment changes the operational model. If the code never calls acknowledge(), offsets may not advance as expected.

Finally, do not treat @KafkaListener as "magic." It is convenient, but there is still a real container lifecycle underneath it that affects performance and correctness.

Summary

  • 'MessageListenerContainer is the runtime component that manages Kafka consumer polling and listener dispatch.'
  • 'ConcurrentMessageListenerContainer enables parallel consumption through multiple child containers.'
  • '@KafkaListener methods run inside containers created by a listener-container factory.'
  • Acknowledgment and error-handling behavior are container concerns, not just listener-method concerns.
  • Concurrency, partition count, and commit strategy should be configured together, not independently.

Course illustration
Course illustration

All Rights Reserved.