Monitoring Java Management Extensions (JMX) metrics of a Kafka broker provides insights into the broker’s performance, resource usage, and operational health. Apache Kafka, which is widely used for building real-time data pipelines and streaming applications, exposes a wide range of metrics that are crucial for monitoring and managing Kafka effectively.

Enabling JMX in Kafka

Before you can monitor any JMX metrics, you need to ensure that JMX is enabled on your Kafka brokers. By default, Kafka brokers start with JMX enabled but without remote access. To enable remote JMX access and set the JMX port, you need to add the following lines to your Kafka broker configuration (server.properties):

1# Kafka Configurations
2listeners=PLAINTEXT://your.host.name:9092
3# JMX remote access configuration
4JMX_PORT=9999
5KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=your.host.name"

Accessing JMX Metrics via Command Line

Once JMX is enabled and configured, you can use various tools to monitor these metrics from the command line. The most straightforward tool is jconsole, which is included in the JDK.

Using Jconsole

1. Open a terminal.
2. Start jconsole by entering the command:

   jconsole localhost:9999

1. This command connects jconsole to the JMX server running on the Kafka broker at localhost on port 9999.

Using JMX Term

For a more command-line focused tool, you can use jmxterm, which is an open-source tool that provides a CLI for interacting with JMX servers.

Installing JMX Term

You can download jmxterm from its official repository (https://github.com/jiaqi/jmxterm). Extract and run it using a Java command:

java -jar jmxterm-*-uber.jar

Monitoring Kafka with JMX Term

1. Open jmxterm:

   java -jar jmxterm-*-uber.jar --url localhost:9999

1. You can then use commands to watch various metrics, for example:

1   # view all domains
2   domains
3   # change domain to kafka.server
4   domain kafka.server
5   # list available metrics
6   beans
7   # get specific metric data
8   get -b kafka.server:name=MessagesInPerSec,type=Broker TopicMessagesInPerSec

Exporting Metrics for Prometheus

Prometheus is another powerful tool that can scrape JMX metrics if exposed correctly. Using the jmx_exporter is a popular method to expose these metrics.

Setting up JMX Exporter

1. Download the jmx_prometheus_javaagent.jar from its GitHub repository.
2. Add the following line to your Kafka’s KAFKA_OPTS environment variable:

   export KAFKA_OPTS="$KAFKA_OPTS -javaagent:/path/to/jmx_prometheus_javaagent.jar=7071:/path/to/config.yaml"

1. The config.yaml includes rules for what metrics to expose and how.

Querying Metrics in Prometheus

After configuring, Prometheus can scrape these metrics from http://your.kafka.broker.host:7071/metrics.

Best Practices and Key Metrics to Monitor

Some of the crucial Kafka JMX metrics to track include:

• Messages in / sec: The rate at which messages are being received.
• Bytes in / sec: The rate at which bytes are being received.
• Bytes out / sec: The rate at which bytes are being sent to consumers.
• Partition count: The total number of partitions on the broker.

Summary Table

By routinely monitoring these metrics, you can gain insights into the performance and stability of your Kafka cluster and promptly address any issues that arise.