Introduction

Kafka does not implement "TTL" as a per-message countdown in the way some queues do. For a topic, what people usually mean by TTL is retention, most commonly the retention.ms topic configuration, and changing it affects replay windows, disk usage, and recovery behavior for every consumer of that topic.

Think in Retention, Not Message Expiration

The key setting for time-based retention is retention.ms. Kafka deletes old log segments when they are older than the retention window, subject to the topic's cleanup policy and the timing of segment cleanup.

That means two important things:

• data is removed by retention policy, not by a per-record timer
• deletion is not always immediate at the exact millisecond boundary

Other settings also matter:

• 'retention.bytes can delete data based on size'
• 'cleanup.policy can be delete, compact, or both'
• segment settings influence when cleanup is actually possible

If retention.bytes is small, increasing retention.ms alone may not keep data around longer.

Inspect the Current Topic Configuration First

Before changing anything, describe the topic so you know whether a topic-level override already exists.

1kafka-configs.sh \
2  --bootstrap-server broker1:9092 \
3  --entity-type topics \
4  --entity-name orders-events \
5  --describe

Capture that output in your ticket or runbook. It gives you a rollback target and makes it obvious whether the topic currently inherits broker defaults or already has explicit overrides.

Set a Topic-Specific Retention Window

To change the retention window for one topic, alter retention.ms directly.

For example, thirty-six hours is 129600000 milliseconds:

1kafka-configs.sh \
2  --bootstrap-server broker1:9092 \
3  --entity-type topics \
4  --entity-name orders-events \
5  --alter \
6  --add-config retention.ms=129600000

Then verify the effective value:

1kafka-configs.sh \
2  --bootstrap-server broker1:9092 \
3  --entity-type topics \
4  --entity-name orders-events \
5  --describe

Do not skip the verification step. Unit conversion mistakes and wrong cluster targets are common, especially when operators think in hours or days and Kafka expects milliseconds.

Remove the Override to Return to Broker Defaults

If you want the topic to inherit the broker-level retention policy again, delete the topic override:

1kafka-configs.sh \
2  --bootstrap-server broker1:9092 \
3  --entity-type topics \
4  --entity-name orders-events \
5  --alter \
6  --delete-config retention.ms

After that, describe the topic again and confirm that the topic no longer has an explicit retention.ms override.

Check Consumer Recovery Before Shortening Retention

Lowering retention is not just a storage change. It reduces the replay window for consumers, backfills, and incident recovery.

Before shortening retention, check the lag and recovery expectations of important consumer groups:

1kafka-consumer-groups.sh \
2  --bootstrap-server broker1:9092 \
3  --group billing-consumer \
4  --describe

If consumers can fall behind by twelve hours during an outage and you reduce retention to six hours, they may lose the ability to catch up from Kafka alone.

A safe pre-change checklist includes:

• current consumer lag
• required replay window for incident recovery
• current disk pressure
• rollback command ready before execution

Watch Out for Compaction and Size Limits

If the topic uses compaction, retention behavior is more nuanced than a delete-only topic. A cleanup policy such as compact,delete can surprise teams who expect a simple time window.

Also remember that time-based retention is not the only limit. If retention.bytes is set aggressively, Kafka can remove older data before the retention.ms target is reached.

That is why post-change monitoring matters. If data seems to disappear early, inspect both time and size retention settings rather than assuming Kafka ignored your TTL change.

Common Pitfalls

The most common mistake is treating retention.ms as if it were the only retention control. In practice, retention.bytes and cleanup policy can change the observed behavior.

Another frequent issue is shortening retention without checking worst-case consumer lag and replay requirements. Developers also sometimes expect data to vanish immediately after the config update, but Kafka removes old segments asynchronously. Finally, do not call this per-message TTL and assume queue-style semantics. Kafka retention operates at the log-segment level.

Summary

• Topic TTL in Kafka is usually implemented through retention settings, especially retention.ms.
• Use kafka-configs.sh to inspect, change, and verify topic-level retention.
• Convert time units carefully because Kafka expects milliseconds.
• Check consumer lag and recovery needs before reducing retention.
• Remember that retention.bytes and cleanup policy can override what you expect from time-based retention alone.