Kafka Exactly-Once Semantics: What It Actually Means

dTechStack Team

May 1, 2026

6 min read·Apache Kafka

KafkaEOSExactly-OnceTransactions

Idempotent producers prevent duplicates within a session. Transactions extend this across partitions and topics.

📬 Weekly Newsletter

Get one deep-dive article every Saturday. No spam. Unsubscribe anytime.

"Exactly-once" is the most misunderstood phrase in distributed systems. In Kafka specifically, it means something very precise — and it comes with conditions that many engineers don't realise until data is already duplicated in production.

The Three Delivery Guarantees

At-most-once: Fire and forget. The producer doesn't wait for acknowledgment. Messages may be lost, never duplicated. Used for metrics where loss is acceptable.

At-least-once: The producer retries on failure. Messages are never lost, but may be delivered multiple times if the producer crashes after sending but before receiving ack. This is the default.

Exactly-once: Every message is processed exactly once. No duplicates, no loss. This requires both producer and consumer coordination.

Step 1: Idempotent Producers

With retries enabled (retries=MAX_INT), a network blip can cause the broker to receive the same message twice. By default, both copies are written.

Idempotent producers solve this for a single producer session:

java

# Producer config — one property enables idempotence
props.put("enable.idempotence", "true");
# Automatically sets: acks=all, retries=MAX_INT, max.in.flight=5

⚡dTechStack Pro

Get unlimited access to all Apache Kafka tutorials & animated diagrams

1,000+ animated tutorials · 100+ AI tools · Mock interviews · Certifications

Try Pro free →

Step 2: Transactions

Transactions extend exactly-once across multiple partitions and topics — and atomically link a consumer offset commit to the write:

java

// Producer side
props.put("transactional.id", "order-processor-1");
producer.initTransactions();
producer.beginTransaction();
try {
    producer.send(new ProducerRecord<>("enriched-orders", key, value));
    producer.send(new ProducerRecord<>("audit-log", key, auditValue));
    producer.sendOffsetsToTransaction(offsets, consumerGroupMetadata);
    producer.commitTransaction();
} catch (Exception e) {
    producer.abortTransaction();
}

Where EOS Actually Breaks Down

Cross-system transactions are not supported. Kafka EOS only works within Kafka. If you write to Kafka AND to a database, you need a two-phase commit or saga pattern.

Consumer-side processing is not idempotent by default. If your consumer calls an external API or writes to a database, Kafka EOS doesn't protect that operation. You need idempotent processing logic on the consumer side.

transactional.id collisions. If two producers use the same transactional.id simultaneously, the newer one fences the older one — all pending transactions from the old producer are aborted.