Introduction

The Kafka error saying leader not available means the broker metadata currently has no writable leader for one or more topic partitions. Without a leader, producers cannot append records for affected partitions. The fastest fix is to identify whether the problem is topic lifecycle, broker health, replication constraints, or network listener misconfiguration.

What the Error Really Means

Kafka writes always go to partition leaders. A partition can temporarily lose leader during startup, broker outage, metadata propagation delay, or failed leader election.

When console producer sends records during that state, client reports leader not available or similar metadata exceptions.

The key is to debug cluster state first, not producer syntax.

Verify Topic and Partition Metadata

Start by confirming topic exists and has healthy leader assignments.

kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders

In healthy output, each partition should show:

• a leader broker id
• in-sync replica list not empty

If leader field is missing or invalid for partitions, producer failure is expected.

Also list topics to confirm target name is correct.

kafka-topics.sh --bootstrap-server localhost:9092 --list

Typos and wrong environment bootstrap endpoints are common root causes.

Check Broker Availability and Controller Health

If topic metadata is broken, ensure brokers and controller are alive.

kafka-broker-api-versions.sh --bootstrap-server localhost:9092

If this command fails or shows missing expected brokers, cluster itself is unhealthy.

For older ZooKeeper-based clusters, verify ZooKeeper service status. For KRaft clusters, verify controller quorum and broker logs for election issues.

Regardless of mode, broker logs are critical for leader election diagnostics.

Validate Replication Factor Against Live Brokers

A frequent misconfiguration is creating topic with replication factor higher than currently available brokers.

Example failure pattern:

• cluster has one broker running
• topic created with replication factor three
• partitions cannot elect valid leader

Create topic with feasible replication factor for current environment.

1kafka-topics.sh \
2  --bootstrap-server localhost:9092 \
3  --create \
4  --topic orders \
5  --partitions 3 \
6  --replication-factor 1

For production, replication factor should match durability requirements and broker capacity.

Check Listener and Advertised Address Configuration

Producer can connect to bootstrap broker but still fail leader communication if advertised listeners are wrong.

Typical issue:

• broker advertises internal hostname unreachable from producer host
• metadata points client to unreachable leader endpoint

Broker configuration example:

listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://broker1.example.com:9092

Ensure advertised address is reachable from producer network context.

Container and Kubernetes deployments are especially sensitive here.

Reproduce and Test with Console Tools

Use console producer and consumer explicitly to isolate application code.

kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders

In another shell:

kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders --from-beginning

If console tools fail the same way as app producer, problem is cluster-side, not serializer or client code.

Handle Partition Reassignment and Recovery

If leadership problems persist after broker recovery, partition reassignment or replica synchronization may be required.

Generate and execute reassignment plan using official admin tools. Also verify in-sync replica counts before increasing producer acks requirements.

Be cautious with aggressive under-replicated partition tuning in production incidents. Stabilize broker availability first.

Temporary Startup Timing Issues

In local development, leader not available can appear briefly right after broker starts. Retry after metadata settles.

If retries succeed within seconds, issue may be normal startup timing. If persistent, it indicates real cluster inconsistency.

Client-side retries are useful, but they should not hide long-lived cluster failures.

Operational Checklist

Practical order for incident response:

1. describe topic and inspect leaders
2. confirm all expected brokers are reachable
3. validate replication factor compatibility
4. verify advertised listeners from producer network
5. inspect broker logs for election errors
6. re-test with console producer and consumer

This sequence narrows root cause quickly.

Common Pitfalls

A common pitfall is assuming topic auto-creation worked when cluster policy disables it. Another is using replication factor values copied from production in a single-broker local environment. Teams also often overlook advertised listener mismatches, especially in containerized deployments where internal and external hostnames differ. Finally, adding retries in producer code without fixing metadata and leader assignment issues only delays visible failure.

Summary

• Leader not available means producer cannot find a writable partition leader.
• Start debugging from topic metadata and broker health.
• Validate replication settings against active broker count.
• Ensure advertised listeners are reachable from producer network path.
• Use console tools to confirm cluster behavior before changing app code.