Kafka Idempotent Producer: What `enable.idempotence=true` Actually Buys You

The idempotent producer is one of those Kafka features that sounds bigger than it is. Flipping enable.idempotence=true does exactly one thing well, and engineers who assume it does more keep shipping double-charge bugs.

Here is what actually happens on the wire. When the producer connects, the broker hands it a producer ID (PID). Every batch the producer sends is tagged with that PID, an epoch, and a per-partition sequence number that starts at zero and increments. On the broker, the leader keeps the last accepted sequence number per (PID, epoch, partition) tuple. If a duplicate batch arrives, the leader sees sequence 47 already landed, drops the batch, and returns the original ack. The log stays clean.

This is purely a retry guard. If your producer sends a batch, the network drops the ack, and the client library retries, the broker recognizes the duplicate and discards it. If the leader fails over mid-flight, the new leader knows about the same sequence space because it was replicated. The producer epoch fences any old producer instance that tries to resurrect after a restart, which is what prevents reordering.

What it does not do: anything outside a single producer instance writing to a single partition. A second producer is a fresh PID with a fresh sequence space. A consumer that processes the same record twice is not the producer's problem. A cross-cluster mirror that replays records into a downstream cluster is a new producer entirely.

The production failure I have seen most often: a team enabled enable.idempotence=true on the source-side producer and called the system "exactly once." Their downstream consumer was not idempotent. A MirrorMaker 2 rebalance caused the source-side producer to retry a batch the broker had already accepted. The source log stayed clean. But the destination cluster was a separate Kafka, fed by MirrorMaker's own producer, and the destination consumer had not yet committed its offsets when the rebalance happened. On resume, the destination consumer reread a window of records and ran the side effect twice. The producer setting did nothing for them because the duplicate path was on the consume side, not the produce side.

The fix is to treat idempotent producers as one layer in a stack. The producer handles network-retry duplicates on its own writes. For consume-process-produce loops inside one cluster, you need transactions and read_committed consumers. For anything that leaves Kafka, a payment API call, a Postgres insert, an outbound webhook, the consumer must hold its own dedup table keyed by something that survives across runs.

Mental model: idempotent producer is local correctness. Exactly-once is a system property. The flag is necessary, not sufficient.