Kafka Consumer Group
Offset Resetting
Kafka Operations
Kafka Setup
Kafka Consumers

How to reset offsets to arbitrary value in Kafka Consumer Group?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Kafka lets you reset committed consumer group offsets to an arbitrary value, but it is an administrative operation that should be done carefully. The usual tools are the kafka-consumer-groups.sh CLI for manual resets and the Admin API for programmatic resets, and in both cases the group must be inactive or empty before the change can succeed.

The CLI Tool for Manual Resets

The standard operational tool is kafka-consumer-groups.sh with --reset-offsets. Kafka's current operations guide recommends previewing the change first, then applying it with --execute.

To reset one partition to an exact offset:

bash
1bin/kafka-consumer-groups.sh \
2  --bootstrap-server localhost:9092 \
3  --group my-group \
4  --topic my-topic:0 \
5  --reset-offsets \
6  --to-offset 12345

That command previews the change. To actually apply it, add --execute.

bash
1bin/kafka-consumer-groups.sh \
2  --bootstrap-server localhost:9092 \
3  --group my-group \
4  --topic my-topic:0 \
5  --reset-offsets \
6  --to-offset 12345 \
7  --execute

Resetting Multiple Partitions

If each partition should move to a different arbitrary offset, use the --from-file scenario with a CSV file. This is the best fit when a simple single offset is not enough.

csv
1topic,partition,offset
2orders,0,100
3orders,1,250
4orders,2,400

Then run:

bash
1bin/kafka-consumer-groups.sh \
2  --bootstrap-server localhost:9092 \
3  --group my-group \
4  --reset-offsets \
5  --from-file offsets.csv \
6  --execute

Kafka's reset-offsets tooling supports this specifically for arbitrary per-partition resets.

Always Stop the Consumers First

This is the most important operational rule. Kafka's docs state that consumer instances must be inactive before reset operations are performed. If the group is still active, the reset may fail or the running consumers may immediately overwrite the offsets again.

A good sequence is:

  1. stop the consumers
  2. describe the group and confirm it is inactive or empty
  3. preview the reset
  4. execute the reset
  5. restart the consumers

Programmatic Reset With the Admin API

If you need to automate offset changes, use the Kafka Admin client. The Admin API exposes alterConsumerGroupOffsets, and the current Javadoc states that the group must be empty for the operation to succeed.

java
1import java.util.Map;
2import java.util.Properties;
3import org.apache.kafka.clients.admin.Admin;
4import org.apache.kafka.clients.consumer.OffsetAndMetadata;
5import org.apache.kafka.common.TopicPartition;
6
7public class ResetOffsetsDemo {
8    public static void main(String[] args) throws Exception {
9        Properties props = new Properties();
10        props.put("bootstrap.servers", "localhost:9092");
11
12        try (Admin admin = Admin.create(props)) {
13            Map<TopicPartition, OffsetAndMetadata> offsets = Map.of(
14                new TopicPartition("orders", 0), new OffsetAndMetadata(100L),
15                new TopicPartition("orders", 1), new OffsetAndMetadata(250L)
16            );
17
18            admin.alterConsumerGroupOffsets("my-group", offsets).all().get();
19        }
20    }
21}

This is useful for internal admin tools or controlled recovery workflows.

Previewing and Safety Checks

Before executing a reset, inspect current offsets and lag:

bash
1bin/kafka-consumer-groups.sh \
2  --bootstrap-server localhost:9092 \
3  --describe \
4  --group my-group

Also verify that the target offsets make sense for the topic retention window. Kafka will clamp out-of-range reset requests to available boundaries in some scenarios, so do not assume the request always lands exactly where you imagined without checking the output.

Resetting to Logical Positions Versus Arbitrary Numbers

Kafka also supports non-arbitrary reset modes such as:

  • '--to-earliest'
  • '--to-latest'
  • '--shift-by'
  • '--to-datetime'
  • '--by-duration'

Those are useful, but when the question is "reset to this exact offset number," the correct tools are --to-offset and --from-file.

Common Pitfalls

The biggest pitfall is trying to reset offsets while consumers are still running. Even if the command appears to work, the active group can immediately race against your administrative change.

Another issue is resetting one partition and forgetting that the consumer group spans multiple partitions. A partial reset may produce confusing replay behavior if the other partitions remain at old offsets.

Developers also forget to preview the reset before executing it. The dry-run style output is there for a reason, and it is much cheaper than replaying the wrong data range.

Finally, do not confuse auto.offset.reset with manual administrative reset. auto.offset.reset only applies when no committed offset exists or the stored offset is invalid.

Summary

  • Use kafka-consumer-groups.sh --reset-offsets for manual offset resets.
  • Use --to-offset for a single exact value and --from-file for arbitrary per-partition values.
  • Stop consumers first so the group is inactive or empty before resetting.
  • Use the Admin API alterConsumerGroupOffsets for programmatic control.
  • Preview the reset and verify the new offsets before restarting the group.

Course illustration
Course illustration

All Rights Reserved.