Kafka Consumer
Producer Client ID
Kafka Programming
Message Brokers
Data Streaming

Can I get producer client id in kafka consumer?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

A Kafka consumer does not automatically receive the producer’s client.id as part of the message metadata. The producer client.id is mainly used by Kafka brokers and monitoring systems for logging, metrics, and quotas, not as message payload that consumers can read later.

What Metadata Consumers Actually Get

A consumer can read standard record metadata such as:

  • topic
  • partition
  • offset
  • timestamp
  • key
  • value
  • headers

The producer’s configured client.id is not one of those record fields. It is a client-side identifier attached to the producer connection, not a built-in property of every stored Kafka message.

Why Kafka Does Not Expose It Automatically

Kafka is designed so records remain relatively simple and portable. Broker-side information such as producer identity for metrics does not become part of the persisted user-visible record contract by default.

That means a consumer reading a topic cannot ask Kafka, "Which producer client.id wrote this exact record?" unless the producer explicitly included that information in the record itself.

The Practical Solution: Use Headers

If consumers need to know the producer identity, put it into record headers explicitly.

Producer example in Java:

java
1import java.nio.charset.StandardCharsets;
2import org.apache.kafka.clients.producer.KafkaProducer;
3import org.apache.kafka.clients.producer.ProducerRecord;
4
5ProducerRecord<String, String> record =
6    new ProducerRecord<>("events", "key1", "value1");
7
8record.headers().add("producer-client-id", "order-service".getBytes(StandardCharsets.UTF_8));
9producer.send(record);

Consumer side:

java
1import java.nio.charset.StandardCharsets;
2import org.apache.kafka.clients.consumer.ConsumerRecord;
3import org.apache.kafka.common.header.Header;
4
5Header header = record.headers().lastHeader("producer-client-id");
6if (header != null) {
7    String producerId = new String(header.value(), StandardCharsets.UTF_8);
8    System.out.println(producerId);
9}

This makes the producer identity part of the message contract instead of relying on hidden broker-side state.

Alternatives To Headers

You could also include the producer identity in:

  • the message key
  • the value payload
  • a structured envelope object

Headers are usually the cleanest place when the information is metadata rather than business data.

Why Headers Are Usually Better

Headers let you add metadata without polluting the main payload schema. They are especially useful when the payload format is reused across multiple services but you still want traceability, origin tags, or processing hints.

That said, only put information there if consumers really need it. Extra metadata is still extra bytes and extra coupling.

Logging And Auditing Perspective

If your actual goal is auditing or tracing producer behavior, broker logs, metrics, tracing systems, or message envelopes may be more appropriate than depending on client.id specifically. The producer client.id was not designed as a durable business-level identity system.

So ask first whether you need per-record metadata for consumers or just operational observability for operators.

Common Pitfalls

The most common mistake is assuming the Kafka record model automatically contains the producer configuration. It does not.

Another mistake is using client.id as though it were a security identity. It is useful operational metadata, but it should not be treated as proof of who authored a message.

A third issue is stuffing too much operational metadata into every message when only broker-side logging would have been sufficient.

Summary

  • Kafka consumers cannot directly read the producer’s client.id from standard record metadata.
  • 'client.id is mainly for producer-side logging, metrics, and quotas.'
  • If consumers need producer identity, include it explicitly in headers or the payload.
  • Headers are usually the cleanest way to attach this kind of metadata.
  • Decide whether your real need is message-level metadata or operational observability.
  • If producer identity matters for downstream auditing, define it as part of the event schema deliberately instead of assuming Kafka will surface it implicitly.

Course illustration
Course illustration

All Rights Reserved.