KafkaConsumer position() vs committed()?
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
position() and committed() are both about offsets, but they answer different questions. position() tells you the next offset the consumer will read from a partition. committed() tells you the offset that has been stored as the consumer group's recovery point.
The Short Mental Model
Use this distinction:
- '
position()= where this consumer is now' - '
committed()= where this group would resume after restart'
Those values are often equal in simple demos, but they can diverge during normal operation.
What position() Returns
position(partition) returns the next offset the consumer plans to fetch from that partition.
That means:
- if you already consumed offset 15
- the next record to read is offset 16
- then
position()is 16
It reflects the consumer's local progress in the current session.
Example:
After polling records, position() usually advances even if you have not committed yet.
What committed() Returns
committed(partition) returns the last committed offset metadata for that partition and group. This is the durable checkpoint Kafka stores for restart and rebalance recovery.
Example:
If nothing has been committed yet, the result may be null.
Why They Differ
Suppose your consumer polls records up to offset 50 but commits only every 10 messages.
At some moment:
- '
position()may be 51' - '
committed()may still be 41'
That means the consumer has locally read farther than the checkpoint Kafka has stored. If the process crashes at that moment, the group will resume from the committed offset, not from the live in-memory position.
That difference is exactly why the two methods both exist.
Example with Manual Commit
This illustrates the common flow:
- polling moves the live position
- committing updates the durable checkpoint
When to Use Each
Use position() when you want to know:
- how far the current consumer instance has progressed
- what the next fetch offset is
- whether local processing is advancing
Use committed() when you want to know:
- what the group has durably stored
- where restart or rebalance recovery will begin
- whether offset commits are working as expected
These are related but not interchangeable.
Rebalances and Restarts
During a rebalance or restart, the committed offset matters more than the previous in-memory position of one consumer instance. That is why relying only on position() can give a false sense of safety if commits are lagging behind processing.
Operationally:
- lag monitoring often compares end offset with committed offset
- session-progress diagnostics often inspect current position too
Both are useful, but for different questions.
Common Pitfalls
Assuming position() is the same as the recovery checkpoint is wrong. It is only the consumer's current next-read offset.
Reading committed() and expecting it to advance automatically after every poll leads to confusion when manual commit or batched auto-commit is in play.
Ignoring null from committed() can break startup logic for new groups that have not stored offsets yet.
Comparing lag using the wrong offset source can make dashboards or troubleshooting conclusions misleading.
Summary
- '
position()is the next offset the current consumer will read.' - '
committed()is the last offset stored for the consumer group.' - '
position()can move ahead ofcommitted()between commits.' - After a crash or rebalance, recovery uses the committed offset, not the last in-memory position.
- Use each method according to whether you care about live progress or durable checkpoint state.
Related reading
- KafkaConsumer `seekToEnd()` does not make consumer consume from latest offset
- KafkaConsumer.close() Why?
- KafkaConsumer.commitAsync() behavior with a lower offset than previous
- kafka.consumer.SimpleConsumer Reconnect due to socket error java.nio.channels.ClosedChannelException
- kafka.errors.KafkaTimeoutError KafkaTimeoutError Failed to update metadata after 60.0 secs
- KafkaException jdk.internal.loader.ClassLoaders can’t find org.apache.kafka.common.security.plain.PlainLoginModule
- KafkaIO checkpoint - how to commit offsets to Kafka
- Kafka.JS refuses to connect <<[BrokerPool] Failed to connect to seed broker, trying another broker from the list>>

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.