Exactly-Once Delivery Is a Myth. Aim for Exactly-Once Effects Instead.

Every few months a vendor sells "exactly-once delivery" and a generation of engineers believes it. Then the first network partition arrives, and they find duplicates in the billing system. The promise was never true. It cannot be.

The reason is a watered-down version of the Two Generals Problem. A sender transmits a message and waits for an ack. The ack does not arrive within the timeout. The sender has no way to tell whether the message was delivered and the ack got lost, or whether the message itself was dropped. Two distinct universes look identical from the sender's side. Pick one and you are wrong half the time.

The sender has exactly two options. Retry, which can deliver twice. Do not retry, which can deliver zero times. There is no third option that delivers exactly once. The network does not give you one.

So engineers stop trying to make the network do something it cannot, and instead make the application immune to the ambiguity. This is what "exactly-once effects" means. The wire sees the same record more than once. The business state changes only the first time. The second delivery is observed, deduped, and dropped.

Concretely, here is what a healthy pipeline looks like.

Producers send messages with a stable event ID. Not a Kafka offset, not a timestamp. A UUID minted by the producer at the moment of the business event, written into the payload, and never regenerated on retry.

Consumers maintain a dedupe store keyed on that ID. Before applying the effect, they check whether the ID has been processed. If yes, ack and move on. If no, apply the effect and record the ID atomically with the effect. Postgres makes this trivial: insert the ID into a processed_events table in the same transaction as the business update, with a unique constraint on the ID. The second delivery hits the unique constraint, the transaction rolls back, the consumer acks.

The other half is the producer side. The classic bug is a service that writes an order to its database and then publishes an event in a second step. The database commit succeeds. The publish fails. The order exists but nobody downstream knows. The outbox pattern fixes this: write the event to an outbox table in the same transaction as the order. A relay polls the outbox and publishes. If the relay crashes mid-publish, it republishes. Duplicates are normal, and the consumer's dedupe store catches them.

The lesson worth keeping. Stop chasing exactly-once delivery. Build at-least-once transport and idempotent consumers. The effects are exactly-once. The wire is not. That is the only design that survives a real network.