Introduction

In modern Kafka Java code, the right tool for listing topics is AdminClient. It connects to the cluster, fetches metadata, and returns topic names without needing a consumer or producer instance.

Use AdminClient, Not a Consumer Hack

Older examples sometimes use consumers to inspect metadata, but AdminClient is the dedicated API for cluster administration tasks such as listing topics, describing topics, and reading broker metadata.

A minimal example looks like this:

1import java.util.Properties;
2import java.util.Set;
3import java.util.concurrent.ExecutionException;
4import org.apache.kafka.clients.admin.AdminClient;
5import org.apache.kafka.clients.admin.AdminClientConfig;
6import org.apache.kafka.clients.admin.ListTopicsOptions;
7
8public class ListKafkaTopics {
9    public static void main(String[] args) throws ExecutionException, InterruptedException {
10        Properties props = new Properties();
11        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
12
13        try (AdminClient admin = AdminClient.create(props)) {
14            ListTopicsOptions options = new ListTopicsOptions().listInternal(false);
15            Set<String> topics = admin.listTopics(options).names().get();
16
17            for (String topic : topics) {
18                System.out.println(topic);
19            }
20        }
21    }
22}

This prints all non-internal topic names visible to the connected client.

Internal Topics and Permissions

The listInternal(false) setting is important if you do not want Kafka internal topics such as consumer offset storage to appear in the result.

You also need to remember that the result depends on permissions. If the Kafka cluster enforces ACLs, the client may see only the topics it is authorized to describe.

So “missing topics” can be a security issue rather than a code bug.

Add Timeouts and Error Handling

In real applications, do not leave the future blocking forever. Add explicit timeout handling and surface errors clearly.

1import java.time.Duration;
2import java.util.Properties;
3import java.util.Set;
4import java.util.concurrent.TimeUnit;
5import org.apache.kafka.clients.admin.AdminClient;
6import org.apache.kafka.clients.admin.AdminClientConfig;
7
8public class SafeListTopics {
9    public static void main(String[] args) throws Exception {
10        Properties props = new Properties();
11        props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
12        props.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "5000");
13
14        try (AdminClient admin = AdminClient.create(props)) {
15            Set<String> topics = admin.listTopics().names().get(5, TimeUnit.SECONDS);
16            System.out.println(topics);
17        }
18    }
19}

That makes failures easier to diagnose when the broker is down, the bootstrap address is wrong, or network policy blocks the connection.

Security and Production Configuration

If the cluster uses SSL or SASL, you must provide the relevant client properties. Listing topics is still just a normal Kafka client connection, so it inherits the same authentication and encryption requirements as other Kafka APIs.

For example, the property set may include:

• 'security.protocol'
• 'sasl.mechanism'
• 'sasl.jaas.config'
• SSL truststore settings

The core Java code does not change, but the client configuration does.

If you are repeatedly listing topics from an application, be deliberate about how often you do it. Metadata calls are lightweight compared with moving message data, but they are still cluster operations. For dashboards or admin pages, caching the result briefly is often better than querying on every request.

If you need more than names, the same admin client can immediately describe the returned topics and fetch partition or configuration metadata. In practice, topic listing is often the first step in a broader admin or monitoring workflow rather than the final result.

Common Pitfalls

The biggest mistake is using the wrong bootstrap server. If the address points to a dead broker or the wrong environment, topic listing fails before any metadata is returned.

Another mistake is forgetting that ACLs can hide topics. A successful connection does not imply permission to list everything.

A third mistake is using a producer or consumer API for topic discovery when AdminClient already exists for that purpose.

Summary

• Use Kafka AdminClient in Java to list topics.
• 'listTopics().names().get() returns the topic names, and ListTopicsOptions lets you exclude internal topics.'
• Add timeouts and error handling so metadata calls do not hang indefinitely.
• Expect security settings and ACLs to affect what the client can see.
• Reach for AdminClient instead of consumer or producer workarounds for administrative metadata tasks.