Kafka
Kafka Producer Performance Test
Kafka Configuration
Kafka 2.10-0.8.2.0
Kafka Producer Config

use kafka-producer-perf-test.sh how to set producer config at kafka_2.10-0.8.2.0

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

For Kafka 2.10-0.8.2.0, kafka-producer-perf-test.sh does not use the newer --producer-props style that many current examples show. In that older release, the tool exposes a fixed set of command-line options such as --broker-list, --request-num-acks, --producer-num-retries, and --compression-codec, and it builds the producer configuration internally from those flags.

The Key Version Difference

This is the part that trips people up: if you search today, you will mostly find examples for newer Kafka versions. Those versions added different CLI flags and a different producer stack.

In Kafka 0.8.2.0, the performance tool expects older option names such as:

  • '--broker-list'
  • '--topics'
  • '--messages'
  • '--message-size'
  • '--request-num-acks'
  • '--producer-num-retries'
  • '--producer-retry-backoff-ms'
  • '--compression-codec'
  • '--sync'

So the first rule is to match the script syntax to the Kafka version you are actually running.

Example Command for 0.8.2.0

bash
1bin/kafka-producer-perf-test.sh \
2  --broker-list localhost:9092 \
3  --topics perf-topic \
4  --messages 100000 \
5  --message-size 100 \
6  --batch-size 200 \
7  --compression-codec 1 \
8  --request-num-acks -1 \
9  --producer-num-retries 3 \
10  --producer-retry-backoff-ms 100 \
11  --threads 1

That command tells the script where the broker is, which topic to target, how many messages to send, what compression to use, and a few core reliability settings.

In this old tool, those flags are translated into old producer properties such as:

  • 'metadata.broker.list'
  • 'compression.codec'
  • 'request.required.acks'
  • 'message.send.max.retries'
  • 'retry.backoff.ms'

You are not passing those keys directly. The script maps the CLI values to them.

What You Can and Cannot Configure

The old perf script is not a general-purpose producer launcher. It only exposes the settings the tool authors chose to wire into the parser.

That means:

  • if a setting has a dedicated flag, use the flag
  • if a setting has no flag, you usually cannot pass it directly through the CLI in the same flexible way that newer versions allow

For example, if you need to vary acknowledgements, retries, or sync versus async behavior, the script already has options for that. If you need an arbitrary producer property that the old parser never exposed, the tool may not support it without modifying source or using a different benchmark harness.

Choosing the Right Flags

Some common old-version tuning choices are:

  • '--request-num-acks -1 for stronger durability'
  • '--request-num-acks 1 for lower latency'
  • '--sync to force synchronous sends'
  • omit --sync to use async mode
  • '--compression-codec 0 for none'
  • '--compression-codec 1 for gzip, depending on the codec mapping in that release'

You should interpret the results in light of those tradeoffs. Higher durability settings often lower throughput, while async sending and batching usually raise throughput at the cost of different reliability and latency characteristics.

Read the Output as a Benchmark, Not a Full Production Model

The script is useful for comparing rough throughput and latency under controlled settings, but it does not automatically mirror a real application’s serialization, partitioning, message keys, or business logic.

Treat it as a targeted benchmark tool:

  • compare acks settings
  • compare batching choices
  • compare compression behavior
  • compare sync and async sending

That is where it adds the most value.

Common Pitfalls

The biggest mistake is copying a modern example that uses --producer-props into Kafka 0.8.2.0. That flag belongs to newer tooling, not this old script.

Another issue is assuming every producer property can be injected freely. In the older perf tool, only a limited set of producer settings is exposed through dedicated command-line options.

Developers also sometimes forget that Kafka 0.8.2.0 uses older property names under the hood, such as metadata.broker.list, rather than the more modern bootstrap.servers.

Finally, benchmark settings should match the question you are trying to answer. If you change retries, acks, batching, threads, and compression all at once, the result becomes hard to interpret.

Summary

  • In Kafka 0.8.2.0, kafka-producer-perf-test.sh uses old version-specific CLI flags.
  • Use options such as --broker-list, --topics, --request-num-acks, and --producer-num-retries.
  • Do not expect the newer --producer-props workflow to work on this release.
  • The script exposes only a subset of producer settings through dedicated flags.
  • Benchmark one variable at a time so the performance results stay meaningful.

Course illustration
Course illustration

All Rights Reserved.