Exactly-Once Semantics

Table of Contents

🔴 P0 — the holy grail of distributed message processing; critical for payments

Problem #

In a distributed system, a message may be delivered more than once (at-least-once) or not at all (at-most-once). Exactly-once delivery is impossible in the general case (proved by the FLP impossibility result and two-generals problem). Yet payments absolutely require “process this exactly once.”

How to Achieve “Effectively Exactly-Once” #

You don’t achieve exactly-once delivery. You achieve exactly-once processing through idempotent consumers:

At-least-once delivery + Idempotent processing = Effectively exactly-once

Message arrives (possibly a duplicate)
  ↓
Check: have I processed this message ID before?
  ↓ YES → return stored result, skip processing
  ↓ NO  → process, store result + message ID atomically

Implementation Approaches #

Approach	Mechanism	Limitation
Idempotency key + DB	Store processed keys, check before processing	Need key TTL / cleanup
Transactional outbox	Atomically write result + outbox in one txn	Requires CDC or polling
Kafka transactions	Atomic read-process-write across partitions	Kafka-to-Kafka only
Deduplication service	Centralised dedup (Redis/DynamoDB)	Additional dependency

Instinct #

“Exactly-once” is an end-to-end property, not a middleware feature. Even if Kafka provides exactly-once between producers and consumers, the end-to-end guarantee requires idempotent processing at the application level. This is the end-to-end argument from distributed systems theory.

INTERVIEW: This single sentence demonstrates understanding of the distinction and saves time explaining the mechanism. The key phrase to deploy:
Exactly-once delivery is impossible; exactly-once processing is achievable through idempotent consumers.

References #

Exactly-Once Semantics Are Possible — Confluent
You Cannot Have Exactly-Once Delivery — Tyler Treat

DDIA 2e Reference #

Chapter 11: Exactly-once semantics in stream processing
Chapter 12: The end-to-end argument for exactly-once