Kafka 10 consumer
message consumption
committing messages
data streaming
real-time processing

Consume messages without committing from Kafka 10 consumer

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Consuming messages without committing offsets means you want to read data but not advance the stored position for the consumer group. That is useful for debugging, replay testing, and audit-style reads. The key point is that disabling commits does not freeze the consumer in place forever; it only means Kafka will not persist new progress for that group unless you commit explicitly.

Turn Off Auto Commit

In the Kafka 0.10 Java consumer API, the first step is disabling automatic commits.

java
1import java.time.Duration;
2import java.util.Collections;
3import java.util.Properties;
4import org.apache.kafka.clients.consumer.ConsumerConfig;
5import org.apache.kafka.clients.consumer.ConsumerRecord;
6import org.apache.kafka.clients.consumer.ConsumerRecords;
7import org.apache.kafka.clients.consumer.KafkaConsumer;
8
9Properties props = new Properties();
10props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
11props.put(ConsumerConfig.GROUP_ID_CONFIG, "debug-reader");
12props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
13    "org.apache.kafka.common.serialization.StringDeserializer");
14props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
15    "org.apache.kafka.common.serialization.StringDeserializer");
16props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false");
17
18KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
19consumer.subscribe(Collections.singletonList("events"));

With enable.auto.commit=false, Kafka will not periodically store offsets on your behalf.

Do Not Call Commit Methods

The second requirement is obvious but important: do not call commitSync() or commitAsync().

java
1while (true) {
2    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
3    for (ConsumerRecord<String, String> record : records) {
4        System.out.printf("offset=%d key=%s value=%s%n",
5            record.offset(), record.key(), record.value());
6    }
7    // no commit here
8}

If the consumer restarts later with the same group id, Kafka will resume from the last committed offset for that group, not from the messages you merely read.

Understand What Still Changes Locally

Even without committing, the consumer still advances its in-memory position as it polls. That means within one running process, repeated polls continue moving forward through the topic.

If you want to reread messages in the same session, you must explicitly seek.

java
consumer.poll(Duration.ofMillis(500));
consumer.seekToBeginning(consumer.assignment());

This is where people often get confused. “No commit” does not mean “the consumer never moves.” It means the movement is not persisted for the group.

Restart Behavior Depends on Existing Committed Offsets

On restart, Kafka checks whether the group already has committed offsets. If it does, the consumer resumes there. If it does not, then auto.offset.reset decides what happens next, usually earliest or latest.

That means a no-commit debug run with a brand-new group id may start from the beginning when auto.offset.reset=earliest, but the same code with an older group id may jump to an older committed position instead.

So if you want predictable replay behavior, control all three of these together:

  • group id
  • commit behavior
  • 'auto.offset.reset'

Consider assign for Read-Only Inspection

If the goal is inspection rather than participating in consumer-group rebalancing, using direct partition assignment can be cleaner.

java
1import org.apache.kafka.common.TopicPartition;
2
3TopicPartition partition = new TopicPartition("events", 0);
4consumer.assign(Collections.singletonList(partition));
5consumer.seekToBeginning(Collections.singletonList(partition));

With assign, you avoid some consumer-group behavior entirely and gain explicit control over the offset position.

That can be a better pattern for replay tools, diagnostics, or one-off scripts.

Choose the Right Group Id Strategy

If you consume without committing while using a shared production group id, you can still interfere with that group through rebalancing. For safe inspection work, use a separate group id or direct assignment.

That distinction matters more than many examples admit. The danger is not only whether offsets are committed, but whether the consumer joins a live group you did not intend to affect.

Common Pitfalls

  • Disabling auto commit but still calling commitSync() somewhere else in the code.
  • Assuming “no commit” means the in-memory consumer position never advances.
  • Joining a production consumer group just for debugging.
  • Forgetting to seek when you want to replay data in the same session.
  • Confusing group-persisted offsets with the consumer’s current local position.

Summary

  • Disable enable.auto.commit to stop automatic offset persistence.
  • Do not call commit methods if you want a true no-commit read.
  • The consumer still advances locally while running.
  • Restarts depend on existing committed offsets and auto.offset.reset.
  • For debugging or audit reads, prefer a separate group id or direct partition assignment.

Course illustration
Course illustration

All Rights Reserved.