Java Programming
Kafka Topic
Coding Tutorials
Application Development
Programming Tips

Check for the existence of a Kafka topic programatically in Java

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

If a Java application depends on a Kafka topic, it is sometimes useful to check whether that topic exists before producing, consuming, or creating supporting resources. The correct modern tool for that job is Kafka's AdminClient, not the producer or consumer API. Once you use the admin API, the problem becomes a straightforward metadata lookup with a few caveats around permissions and race conditions.

Use AdminClient for Topic Metadata

Kafka exposes cluster metadata operations through AdminClient. To check topic existence, create a client with at least the bootstrap server address.

java
1import java.util.Properties;
2import org.apache.kafka.clients.admin.AdminClient;
3import org.apache.kafka.clients.admin.AdminClientConfig;
4
5Properties props = new Properties();
6props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
7
8try (AdminClient admin = AdminClient.create(props)) {
9    System.out.println("Admin client ready");
10}

This client is the right place for administrative checks such as topic listing, topic creation, and configuration inspection.

The Simple Existence Check: listTopics()

The most direct existence check is to ask Kafka for topic names and test membership.

java
1import java.util.Properties;
2import java.util.Set;
3import org.apache.kafka.clients.admin.AdminClient;
4import org.apache.kafka.clients.admin.AdminClientConfig;
5
6public class TopicExistsExample {
7    public static void main(String[] args) throws Exception {
8        Properties props = new Properties();
9        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
10
11        try (AdminClient admin = AdminClient.create(props)) {
12            Set<String> topicNames = admin.listTopics().names().get();
13            boolean exists = topicNames.contains("orders");
14            System.out.println("Topic exists: " + exists);
15        }
16    }
17}

This is easy to read and works well for many applications.

One advantage of this approach is that it is obvious what the code is doing. One downside is that listing all topics may be more metadata than you need if you only care about one name.

A More Targeted Option: describeTopics()

If you want to query a specific topic, you can use describeTopics() and handle the case where Kafka reports that the topic does not exist.

java
1import java.util.Collections;
2import java.util.Properties;
3import java.util.concurrent.ExecutionException;
4import org.apache.kafka.clients.admin.AdminClient;
5import org.apache.kafka.clients.admin.AdminClientConfig;
6
7public class TopicExistsByDescribe {
8    public static boolean topicExists(AdminClient admin, String topic) {
9        try {
10            admin.describeTopics(Collections.singletonList(topic))
11                 .allTopicNames()
12                 .get();
13            return true;
14        } catch (ExecutionException ex) {
15            return false;
16        } catch (InterruptedException ex) {
17            Thread.currentThread().interrupt();
18            throw new RuntimeException(ex);
19        }
20    }
21
22    public static void main(String[] args) {
23        Properties props = new Properties();
24        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
25
26        try (AdminClient admin = AdminClient.create(props)) {
27            System.out.println(topicExists(admin, "orders"));
28        }
29    }
30}

This keeps the check focused on one topic, though in real code you may want to inspect the exception cause instead of treating every ExecutionException as "topic missing."

A Small Helper Method for Real Code

A reusable helper often looks like this:

java
1import java.util.Properties;
2import java.util.Set;
3import java.util.concurrent.ExecutionException;
4import org.apache.kafka.clients.admin.AdminClient;
5import org.apache.kafka.clients.admin.AdminClientConfig;
6
7public final class KafkaTopics {
8    private KafkaTopics() {
9    }
10
11    public static boolean exists(String bootstrapServers, String topic) {
12        Properties props = new Properties();
13        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
14
15        try (AdminClient admin = AdminClient.create(props)) {
16            Set<String> names = admin.listTopics().names().get();
17            return names.contains(topic);
18        } catch (InterruptedException ex) {
19            Thread.currentThread().interrupt();
20            throw new RuntimeException(ex);
21        } catch (ExecutionException ex) {
22            throw new RuntimeException(ex);
23        }
24    }
25}

This is enough for startup checks or health diagnostics.

Topic Existence Checks Are Not a Lock

One subtle but important point: existence checks are only a moment-in-time observation. A topic can be created or deleted immediately after your check completes.

So use these checks for:

  • diagnostics
  • startup validation
  • conditional creation logic

Do not treat them as a permanent guarantee. If the application truly depends on the topic, the rest of the code should still handle failures gracefully.

Permissions Can Affect the Result

If the Kafka principal used by AdminClient lacks permission to describe or list topics, the check may fail even when the topic exists. That means a false result can actually be an authorization problem rather than a missing topic.

So when an existence check behaves unexpectedly, inspect:

  • Kafka ACLs
  • security protocol and SASL settings
  • whether the admin client can reach the cluster at all

Metadata code is only as trustworthy as the connection and permissions behind it.

Common Pitfalls

One common mistake is trying to infer topic existence by creating a producer or consumer and hoping for a failure. That is indirect and less clear than using AdminClient.

Another mistake is swallowing every exception and returning false. A network failure, authentication problem, or timeout is not the same thing as "topic does not exist."

Developers also sometimes forget that topic existence can change after the check. The result is useful, but it is not a lock or reservation.

Finally, if you list all topics, remember that permissions and internal-topic visibility can affect what you see.

Summary

  • Use Kafka's AdminClient to check topic existence in Java.
  • 'listTopics().names() is the simplest direct approach.'
  • 'describeTopics() can be a more targeted check for one topic.'
  • Treat topic existence as a moment-in-time observation, not a long-term guarantee.
  • Distinguish missing topics from connection, timeout, or authorization failures.

Course illustration
Course illustration

All Rights Reserved.