Kafka
KafkaListener
Topic Pattern
Kafka Topics
Software Development

Kafka - how to use @KafkaListener(topicPattern=${kafka.topics}) where property kafka.topics is 'sss.*'?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Spring Kafka lets a listener subscribe to topics by exact name or by regular expression. If you want @KafkaListener to consume from every topic that matches a property such as sss.*, the key point is that topicPattern expects a Java regular expression, not a comma-separated list or a shell-style wildcard.

topicPattern Uses a Regex

This works:

java
1@KafkaListener(topicPattern = "${kafka.topics}", groupId = "demo-group")
2public void listen(String payload) {
3    System.out.println("Received: " + payload);
4}

And this property value is injected before the listener container is created:

properties
kafka.topics=^sss\\..*

That pattern matches topics such as:

  • 'sss.orders'
  • 'sss.payments'
  • 'sss.retry'

Using ^sss\\..* is often clearer than sss.* because it explicitly means “starts with sss.”. A bare sss.* also matches names such as ssstest, which may or may not be what you intended.

A Minimal Spring Boot Example

java
1package example.kafka;
2
3import org.springframework.kafka.annotation.KafkaListener;
4import org.springframework.stereotype.Component;
5
6@Component
7public class PatternConsumer {
8
9    @KafkaListener(
10        topicPattern = "${kafka.topics}",
11        groupId = "demo-group"
12    )
13    public void handle(String message) {
14        System.out.println("message = " + message);
15    }
16}

Configuration:

properties
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=demo-group
kafka.topics=^sss\\..*

With that setup, the listener subscribes to any topic whose name matches the regex. You do not need to list those topics individually in the annotation.

When to Use topicPattern Instead of topics

Use topics when the list is fixed:

java
@KafkaListener(topics = {"sss.orders", "sss.payments"})

Use topicPattern when the set is dynamic or follows a naming convention:

java
@KafkaListener(topicPattern = "^sss\\..*")

Pattern subscriptions are useful when new topics may be created later and should be picked up automatically by consumers in the same application.

That is especially common in multi-tenant systems or pipelines where topic names are generated from consistent prefixes.

Property Files and Escaping

The regex is compiled by the Kafka client, but it first passes through Spring property resolution. That means escaping matters.

In application.properties, the following is usually fine:

properties
kafka.topics=^sss\\..*

In YAML, quote the pattern so it stays a string:

yaml
kafka:
  topics: '^sss\..*'

Without clear escaping, the expression can become hard to read or can mean something slightly different from what you intended.

Testing the Pattern

You can sanity-check the regex in plain Java before wiring it into Kafka:

java
1import java.util.regex.Pattern;
2
3public class PatternCheck {
4    public static void main(String[] args) {
5        Pattern pattern = Pattern.compile("^sss\\..*");
6
7        System.out.println(pattern.matcher("sss.orders").matches());
8        System.out.println(pattern.matcher("sss.retry").matches());
9        System.out.println(pattern.matcher("other.orders").matches());
10    }
11}

Output:

text
true
true
false

If that test fails, the Kafka listener will fail to match the same way.

Common Pitfalls

The most common mistake is treating the pattern like a wildcard syntax rather than a regex. topicPattern does not interpret * by itself as “any topic name.” It only has meaning as part of a valid regular expression.

Another issue is using a pattern that is broader than intended. sss.* and ^sss\\..* are not equivalent if your naming convention depends on the dot after sss.

Developers also sometimes mix topics and topicPattern mentally. Use exact topic names for one case and a regex for the other; do not expect topics = "${kafka.topics}" to behave like a pattern subscription.

Finally, remember that a pattern subscription only helps if the topic names actually follow the naming rule you encoded. If the naming scheme is inconsistent, the listener will miss topics or consume too much.

Summary

  • 'topicPattern expects a Java regular expression.'
  • Property placeholders work, so @KafkaListener(topicPattern = "${kafka.topics}") is valid.
  • Prefer a precise regex such as ^sss\\..* when you mean topics starting with sss..
  • Quote and escape the property correctly in .properties or YAML files.
  • Use topicPattern for naming conventions and topics for fixed explicit topic names.

Course illustration
Course illustration

All Rights Reserved.