Kafka
ConcurrentKafkaListenerContainerFactory
Programming
Software Development
Java

When to use ConcurrentKafkaListenerContainerFactory?

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

ConcurrentKafkaListenerContainerFactory is the Spring Kafka factory most applications use to create listener containers for @KafkaListener methods. You use it when you want Spring to manage Kafka consumer containers for you, especially when you need configurable concurrency, acknowledgment behavior, deserialization setup, batch mode, or error handling.

The word “concurrent” does not mean “always turn concurrency up.” It means the factory can create a listener container that runs multiple consumer threads when your topic partition count and workload justify it.

What the Factory Actually Does

The factory creates listener containers that back @KafkaListener methods. Those containers handle:

  • creating Kafka consumer instances
  • polling records
  • dispatching them to listener methods
  • committing offsets according to your configuration
  • applying error handlers and retry rules

A minimal configuration looks like this:

java
1import org.apache.kafka.clients.consumer.ConsumerConfig;
2import org.apache.kafka.common.serialization.StringDeserializer;
3import org.springframework.context.annotation.Bean;
4import org.springframework.context.annotation.Configuration;
5import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
6import org.springframework.kafka.core.ConsumerFactory;
7import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
8
9import java.util.HashMap;
10import java.util.Map;
11
12@Configuration
13public class KafkaConfig {
14
15    @Bean
16    public ConsumerFactory<String, String> consumerFactory() {
17        Map<String, Object> props = new HashMap<>();
18        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
19        props.put(ConsumerConfig.GROUP_ID_CONFIG, "demo-group");
20        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
21        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
22        return new DefaultKafkaConsumerFactory<>(props);
23    }
24
25    @Bean
26    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
27            ConsumerFactory<String, String> consumerFactory) {
28        var factory = new ConcurrentKafkaListenerContainerFactory<String, String>();
29        factory.setConsumerFactory(consumerFactory);
30        factory.setConcurrency(3);
31        return factory;
32    }
33}

When to Use It

Use this factory when you are building Spring Kafka listeners with @KafkaListener. That is the default and normal case for most Spring Boot consumer services.

It becomes especially useful when you need one or more of these:

  • more throughput from multiple partitions
  • custom acknowledgment mode
  • batch listeners instead of single-record listeners
  • centralized error handling
  • record filtering or interception

If your application only needs a basic @KafkaListener, you are still probably using this factory, just with minimal customization.

When Concurrency Helps

Increase concurrency when:

  • the topic has multiple partitions
  • message processing is independent per record
  • one consumer thread is not enough to keep up

Example listener:

java
1import org.springframework.kafka.annotation.KafkaListener;
2import org.springframework.stereotype.Component;
3
4@Component
5public class OrderListener {
6
7    @KafkaListener(topics = "orders", groupId = "demo-group")
8    public void onMessage(String payload) {
9        System.out.println("Received: " + payload + " on thread " + Thread.currentThread().getName());
10    }
11}

If the factory concurrency is set to 3, Spring can create up to three concurrent consumer threads for that listener container.

The practical limit is partition count. Setting concurrency higher than the number of partitions does not create more parallel consumption than the topic can provide.

When Not to Rely on Concurrency Alone

Concurrency is not a magic throughput switch. If the listener does heavy blocking I/O, poor database writes, or slow remote calls, higher consumer concurrency may only move the bottleneck elsewhere.

It is also the wrong fix when strict ordering matters across all messages. Kafka preserves order within a partition, but concurrent consumers processing multiple partitions do not give you one global order.

Other Reasons to Customize the Factory

Beyond concurrency, this factory is where many Spring Kafka behaviors are configured. For example, you might enable batch listeners:

java
factory.setBatchListener(true);

Or configure container properties such as acknowledgment mode through container properties on the factory-created container.

That makes the factory the natural central point for listener behavior across the application.

Common Pitfalls

A common mistake is increasing concurrency beyond the topic’s partition count and expecting more throughput. Kafka cannot assign more active consumer threads than there are partitions to consume.

Another mistake is forgetting that each concurrent consumer still belongs to the same consumer group semantics. Concurrency changes parallelism, not the basic group model.

Developers also turn on concurrency without checking whether downstream systems can handle the extra parallel load.

Finally, do not assume the factory is only for advanced cases. In Spring Kafka, it is the standard way listener containers are built, even when your configuration is simple.

Summary

  • Use ConcurrentKafkaListenerContainerFactory for Spring Kafka @KafkaListener containers.
  • It is the right place to configure concurrency, acknowledgment, batch mode, and error handling.
  • Increase concurrency only when partition count and workload actually support parallel consumption.
  • It improves throughput for independent message processing, not for every bottleneck.
  • Treat it as the central listener-container configuration point, not just as a concurrency toggle.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design