Apache Kafka
Producer Config
request.timeout.ms
max.block.ms
Kafka properties

Apache Kafka Producer Config 'request.timeout.ms' VS. 'max.block.ms' properties

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

request.timeout.ms and max.block.ms sound similar, but they govern different waiting points in the Kafka producer. One controls how long the client waits for a broker response to a request, while the other controls how long producer API calls may block before they give up waiting for metadata or buffer space.

request.timeout.ms Is About The Broker Response

request.timeout.ms limits how long the producer waits for the response to a request it has already sent to Kafka.

Example:

java
1Properties props = new Properties();
2props.put("bootstrap.servers", "localhost:9092");
3props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
4props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
5props.put("request.timeout.ms", "30000");

If the broker does not respond within that window, the client may retry or fail depending on the rest of the producer configuration.

So this setting matters after the request is on the wire.

max.block.ms Is About Blocking In The Producer Client

max.block.ms is different. It limits how long calls such as send() and partitionsFor() can block while waiting for metadata or buffer allocation inside the client.

java
1Properties props = new Properties();
2props.put("bootstrap.servers", "localhost:9092");
3props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
4props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
5props.put("max.block.ms", "10000");

This setting matters before the request is successfully prepared and sent, for example when:

  • metadata for the topic is not available yet
  • the producer buffer is full
  • transactional API calls are waiting internally

If the wait exceeds max.block.ms, the producer throws an exception such as TimeoutException.

Think Of Them As Different Phases

The easiest mental model is:

  • 'max.block.ms: waiting inside the producer before the request can proceed'
  • 'request.timeout.ms: waiting for the broker after the request has been sent'

That distinction explains why tuning the wrong one often fails to solve the real problem.

For example, if your producer is blocked because the internal buffer is exhausted, increasing request.timeout.ms will not help. The request has not even reached the broker yet.

Example Scenario

Suppose your application is producing faster than the cluster can accept records. The internal producer buffer fills up. The next send() call blocks waiting for memory to free up.

That situation is governed mainly by max.block.ms, not request.timeout.ms.

On the other hand, suppose the request has already been sent to the broker, but the broker is slow to acknowledge it. That situation is governed by request.timeout.ms.

The two settings sound similar because both involve waiting, but they guard different boundaries in the producer lifecycle.

Do Not Ignore delivery.timeout.ms

Modern Kafka producer tuning also needs delivery.timeout.ms, which is an upper bound on how long a record can take overall before the producer reports success or failure.

That means request.timeout.ms is not the only timing control affecting retries and delivery behavior.

As current Kafka docs note, delivery.timeout.ms should be greater than or equal to request.timeout.ms + linger.ms. So if you tune request timing aggressively, review delivery timing too.

A Minimal Combined Example

java
1Properties props = new Properties();
2props.put("bootstrap.servers", "localhost:9092");
3props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
4props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
5props.put("request.timeout.ms", "30000");
6props.put("max.block.ms", "10000");
7props.put("delivery.timeout.ms", "120000");

This configuration says:

  • wait up to 10 seconds if send() must block for metadata or buffer space
  • wait up to 30 seconds for a broker response to a sent request
  • fail overall delivery after 120 seconds if the record still cannot be delivered successfully

Common Pitfalls

One common mistake is increasing request.timeout.ms to fix producer buffer exhaustion. That does not address client-side blocking.

Another issue is raising max.block.ms without understanding why the producer is blocking in the first place. If the buffer is constantly full, the real issue may be throughput imbalance or cluster health.

A third problem is tuning these values in isolation and forgetting delivery.timeout.ms, which still caps end-to-end delivery behavior.

Finally, teams often diagnose a timeout only from the exception text without checking whether the delay happened before sending or after the broker received the request.

Summary

  • 'max.block.ms controls how long producer API calls may block inside the client.'
  • 'request.timeout.ms controls how long the client waits for a broker response to a request.'
  • They apply to different phases of the producer lifecycle and are not interchangeable.
  • Buffer pressure and metadata waits point to max.block.ms; slow broker acknowledgments point to request.timeout.ms.
  • Review delivery.timeout.ms alongside both settings for a complete producer timeout strategy.

Course illustration
Course illustration

All Rights Reserved.