Apache Kafka is a distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Testing whether your Kafka server is installed correctly is essential to ensure that it behaves as expected, serving your streaming and message brokering needs efficiently.

Checking Kafka Installation

After installing Kafka, the most straightforward test to verify if it is running correctly is by using Kafka's own command-line tools to create a topic, produce some messages, and consume them.

Prerequisites

Before executing any command, ensure:

• Apache Kafka and ZooKeeper are installed on your server.
• Both Kafka and ZooKeeper services are running.

You can start Kafka and ZooKeeper using the following commands (assuming you are in the Kafka installation directory):

bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties

Part 1: Create a Kafka Topic

Creating a topic is the first step to check if Kafka can store and manage data.

Command:

bin/kafka-topics.sh --create --topic testTopic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1

Details:

• --create: Command to create a topic.
• --topic testTopic: Specifies the name of the topic as testTopic.
• --bootstrap-server localhost:9092: Points to your Kafka server running on localhost with port 9092.
• --replication-factor 1: Each message in the topic will be replicated to only one node.
• --partitions 1: Topic will have only one partition.

Output:

Created topic testTopic.

Part 2: Produce Messages to the Topic

Produce some messages to see if they can be sent to the topic without any issues.

Command:

bin/kafka-console-producer.sh --topic testTopic --bootstrap-server localhost:9092

Details:

• You can type messages into the console after running the command and each new line you enter will be sent as a new message.

Example:

>Hello Kafka
>Testing Kafka Installation

Type CTRL+C to exit the producer prompt.

Part 3: Consume Messages from the Topic

Finally, read the messages back to ensure that they were stored correctly in the Kafka topic.

Command:

bin/kafka-console-consumer.sh --topic testTopic --from-beginning --bootstrap-server localhost:9092

Details:

• --from-beginning: This tells Kafka to fetch messages from the beginning of the log.

Output:

Hello Kafka
Testing Kafka Installation

Summary Table

Additional Considerations

Logging & Monitoring

Once confirmed that Kafka is working as expected, setting up proper monitoring and logging can aid in production diagnostics and performance tuning. Tools like Prometheus, Grafana, or Kafka’s own JMX metrics can be helpful.

Security Aspects

In a production environment, securing Kafka using SSL/TLS, SASL, or Kerberos can protect sensitive data and guard against unauthorized access.

Conclusion

Validating that your Kafka installation works involves checking that all components function as expected—from creating topics to producing and consuming messages. This ensures data integrity and availability in your real-world applications. By following the above steps, one can systematically verify the success of a Kafka installation and move forward with configuring Kafka for their specific use cases.