spring-integration-kafka config consumer to receive message from specify partition
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
If you want a Spring Integration Kafka consumer to read from a specific partition, you should use explicit partition assignment instead of normal topic subscription. That tells the container exactly which partition to consume and avoids the usual group-based partition assignment process.
Subscription vs Explicit Partition Assignment
Kafka consumers normally subscribe to a topic and let the consumer group coordinator assign partitions dynamically. That is the right default for scalable consumer groups.
But some use cases need direct control:
- replaying one partition for debugging
- processing a partition with dedicated business logic
- wiring a fixed partition into a specialized integration flow
In those cases, do not rely on plain topic subscription. Create container properties that explicitly identify the partition.
Spring Integration Pattern
In Spring Integration Kafka, the usual flow is:
- create a listener container
- wrap it in a
KafkaMessageDrivenChannelAdapter - route messages into an integration channel
When you need a specific partition, build the container with TopicPartitionOffset.
This configuration pins the consumer to partition 2 of the orders topic.
Starting Offset Behavior
Partition assignment is only part of the story. You also need to think about where consumption starts.
Depending on your needs, you may want:
- the committed offset for that group
- the beginning of the partition
- a specific absolute offset
TopicPartitionOffset supports more explicit offset control when needed. For example, you can assign a particular offset instead of relying on the group’s last commit.
That is useful for replay workflows, but it changes the semantics significantly. A fixed offset is a deliberate operational choice, not something to turn on casually in normal production consumers.
What Changes When You Pin a Partition
Explicit partition assignment gives you precision, but it also gives up some of the conveniences of normal group-based subscription.
With a normal subscription:
- Kafka balances partitions across group members
- scaling out is straightforward
- partition ownership can shift automatically
With fixed partition assignment:
- this consumer targets only the partitions you named
- scaling requires explicit configuration decisions
- rebalancing is no longer doing the partition choice for you
That is why direct partition assignment is best used for special-purpose consumers rather than as a default architecture for every service.
Integrating the Messages Into a Flow
Once the adapter pushes records into a message channel, the rest of the Spring Integration flow can stay normal.
This shows the main advantage of Spring Integration here. Partition targeting is handled at the Kafka boundary, while the downstream flow stays focused on message processing.
When to Use @KafkaListener Instead
If you do not specifically need Spring Integration channels and adapters, plain Spring Kafka with @KafkaListener and @TopicPartition may be simpler. But if your application already models message routing through integration flows, the adapter-based approach keeps the design consistent.
So the decision is not only about Kafka. It is also about whether the rest of your application already uses Spring Integration patterns.
Common Pitfalls
The most common pitfall is subscribing to a topic normally and expecting a single partition to stay fixed. Standard topic subscription allows Kafka to assign partitions dynamically.
Another mistake is pinning a partition without thinking about offsets. Partition choice and starting point are related but separate decisions.
A third issue is using explicit partition assignment in places that really want scalable consumer-group behavior. Fixed partitions reduce flexibility.
Finally, some developers mix up Spring Kafka examples with Spring Integration Kafka configuration. The concepts overlap, but the adapter and channel wiring are specific to Spring Integration.
Summary
- To consume from a specific partition in Spring Integration Kafka, use explicit partition assignment.
- Build the container with
TopicPartitionOffsetinstead of plain topic subscription. - Wrap that container in a
KafkaMessageDrivenChannelAdapterto feed the integration flow. - Think separately about partition selection and offset starting position.
- Use this pattern for deliberate partition control, not as the default for every consumer.
Related reading
- spring-kafka application.properties configuration for JAAS/SASL not working
- Spring-Kafka Concurrency Property
- Spring-Kafka How to pass the kafka topic from the application.yml
- Spring-Kafka vs. kafka-clients directly
- Spring-Kafka vs. kafka-clients directly
- spring4.2.1, hibernate5 integrate abstract method error
- Spring / RabbitMQ transaction management
- Spring Actuator + Kafka Streams - Add kafka stream status to health check endpoint

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.