How to implement a Kafka consumer in a Spring MVC web app (using Spring Boot)

Kafka Consumer

Spring MVC

Spring Boot

Web Application Development

Java Programming

How to implement a Kafka consumer in a Spring MVC web app (using Spring Boot)

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Kafka & Streaming Java

Introduction

A Spring MVC application can consume Kafka messages without becoming a dedicated stream-processing service. The usual design is to let Spring Boot manage the Kafka listener container, keep the consumer logic in a service layer, and expose the resulting application state through regular MVC controllers.

Add the Right Dependencies and Configuration

At minimum, the application needs Spring Boot web support and Spring for Apache Kafka.

xml

1<dependencies>
2  <dependency>
3    <groupId>org.springframework.boot</groupId>
4    <artifactId>spring-boot-starter-web</artifactId>
5  </dependency>
6  <dependency>
7    <groupId>org.springframework.kafka</groupId>
8    <artifactId>spring-kafka</artifactId>
9  </dependency>
10</dependencies>

Then configure Kafka in application.yml.

yaml

1spring:
2  kafka:
3    bootstrap-servers: localhost:9092
4    consumer:
5      group-id: order-web-app
6      auto-offset-reset: earliest
7      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
8      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

A stable consumer group id matters because offsets are stored per group. If the group id changes unexpectedly, the app may re-read old messages or skip expected behavior.

Keep the Listener Thin

The listener should hand work off to a service instead of embedding all business logic inside the @KafkaListener method.

java

1package com.example.orders;
2
3import java.util.List;
4import java.util.concurrent.CopyOnWriteArrayList;
5import org.springframework.kafka.annotation.KafkaListener;
6import org.springframework.stereotype.Service;
7
8@Service
9public class OrderEventService {
10    private final List<String> recentMessages = new CopyOnWriteArrayList<>();
11
12    @KafkaListener(topics = "orders")
13    public void onMessage(String message) {
14        recentMessages.add(0, message);
15        if (recentMessages.size() > 20) {
16            recentMessages.remove(recentMessages.size() - 1);
17        }
18    }
19
20    public List<String> getRecentMessages() {
21        return List.copyOf(recentMessages);
22    }
23}

This example keeps only recent messages in memory. A production application might validate the payload and then write it to a database, cache, or internal domain service.

Expose the Result Through MVC

The web controller should read from the application service, not from Kafka directly. Kafka ingestion is asynchronous. MVC rendering is request-response. Keeping them separate avoids a lot of coupling.

java

1package com.example.orders;
2
3import org.springframework.stereotype.Controller;
4import org.springframework.ui.Model;
5import org.springframework.web.bind.annotation.GetMapping;
6
7@Controller
8public class OrderController {
9    private final OrderEventService orderEventService;
10
11    public OrderController(OrderEventService orderEventService) {
12        this.orderEventService = orderEventService;
13    }
14
15    @GetMapping("/orders/recent")
16    public String recentOrders(Model model) {
17        model.addAttribute("messages", orderEventService.getRecentMessages());
18        return "orders";
19    }
20}

This keeps the controller synchronous and simple even though the underlying data is fed by Kafka.

Configure Concurrency and Failure Handling Intentionally

A basic consumer can run with default settings, but real applications usually need explicit decisions about concurrency, retries, and dead-letter behavior.

java

1package com.example.orders;
2
3import org.apache.kafka.clients.consumer.ConsumerConfig;
4import org.apache.kafka.common.serialization.StringDeserializer;
5import org.springframework.context.annotation.Bean;
6import org.springframework.context.annotation.Configuration;
7import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
8import org.springframework.kafka.core.ConsumerFactory;
9import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
10
11import java.util.HashMap;
12import java.util.Map;
13
14@Configuration
15public class KafkaConsumerConfig {
16    @Bean
17    public ConsumerFactory<String, String> consumerFactory() {
18        Map<String, Object> props = new HashMap<>();
19        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
20        props.put(ConsumerConfig.GROUP_ID_CONFIG, "order-web-app");
21        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
22        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
23        return new DefaultKafkaConsumerFactory<>(props);
24    }
25
26    @Bean
27    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
28            ConsumerFactory<String, String> consumerFactory) {
29        ConcurrentKafkaListenerContainerFactory<String, String> factory =
30                new ConcurrentKafkaListenerContainerFactory<>();
31        factory.setConsumerFactory(consumerFactory);
32        factory.setConcurrency(3);
33        return factory;
34    }
35}

Concurrency should match topic partitions and processing cost. Adding consumer threads without enough partitions will not improve throughput.

Test the End-to-End Flow

A consumer that compiles is not yet a working integration. Start Kafka locally, publish a test message, and verify the MVC endpoint reflects the consumed state.

bash

kafka-topics.sh --create --topic orders --bootstrap-server localhost:9092 --partitions 3 --replication-factor 1
kafka-console-producer.sh --topic orders --bootstrap-server localhost:9092

If the page does not update as expected, inspect logs first. Broker address, topic name, deserializer choice, and group id mistakes are far more common than framework bugs.

Common Pitfalls

Putting business logic directly inside @KafkaListener methods makes the consumer harder to test and evolve.
Treating MVC requests as if they should wait for Kafka consumption confuses asynchronous ingestion with synchronous page rendering.
Changing the consumer group id casually can reset offset behavior in surprising ways.
Increasing concurrency without enough topic partitions adds complexity without useful throughput.
Ignoring retry and dead-letter strategy turns transient processing failures into fragile behavior.

Summary

Use Spring Boot configuration to wire Kafka into a standard Spring MVC application.
Keep the Kafka listener thin and move domain work into a service layer.
Let MVC controllers read processed application state rather than talk to Kafka directly.
Configure concurrency and failure handling based on real workload and partition design.
Verify the full path with a real topic and an end-to-end test, not just with compilation.

System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

How to implement a Kafka consumer in a Spring MVC web app (using Spring Boot)

System Design practice on Codemia

Introduction

Add the Right Dependencies and Configuration

Keep the Listener Thin

Expose the Result Through MVC

Configure Concurrency and Failure Handling Intentionally

Test the End-to-End Flow

Common Pitfalls

Summary

Related reading

System Design Fundamentals

Track what you have practised

System Design practice on Codemia