Apache Kafka is a powerful distributed streaming platform capable of handling trillions of events a day. Implementing security in Kafka is essential to prevent unauthorized access to data. One common security configuration is SASL_SSL, which combines SASL (Simple Authentication and Security Layer) for authentication and SSL (Secure Sockets Layer) for encryption. This article walks through the steps and components involved in setting up Kafka with SASL_SSL.

Step 1: Configure Kafka Server

1. Configure server.properties

First, you need to set up the Kafka broker configuration to use SASL_SSL. Modify the server.properties file as follows:

1listeners=SASL_SSL://your.server.address:9093
2security.inter.broker.protocol=SASL_SSL
3sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
4ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks
5ssl.keystore.password=yourkeystorepass
6ssl.key.password=yourkeypass
7ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
8ssl.truststore.password=yourtruststorepass
9sasl.enabled.mechanisms=SCRAM-SHA-256

Step 2: Configure JAAS for Kafka Brokers

You must create a JAAS file that Kafka will use for SASL configuration. Typically named kafka_server_jaas.conf, it might look like this:

1KafkaServer {
2   org.apache.kafka.common.security.scram.ScramLoginModule required
3   username="kafka"
4   password="kafka-secret"
5   user_kafka="kafka-secret";
6};

Set the environment variable KAFKA_OPTS to ensure Kafka uses this JAAS configuration:

export KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/kafka_server_jaas.conf"

Step 3: Configure Kafka Client

Clients need to authenticate with the Kafka server using SASL_SSL. Update your client's client.properties file:

1bootstrap.servers=your.server.address:9093
2security.protocol=SASL_SSL
3sasl.mechanism=SCRAM-SHA-256
4ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks
5ssl.truststore.password=yourtruststorepass

Create a JAAS config file for the client kafka_client_jaas.conf:

1KafkaClient {
2   org.apache.kafka.common.security.scram.ScramLoginModule required
3   username="kafka"
4   password="kafka-secret";
5};

Set the environment variable to use this file:

export KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/kafka_client_jaas.conf"

Step 4: Generate SSL Key and Certificate

For both the Kafka server and clients, you need to create a keystore and a truststore. Use keytool for this purpose:

1# Create server keystore
2keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 -genkey -keyalg RSA -storepass yourkeystorepass -keypass yourkeypass
3
4# Create client truststore
5keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca-cert -storepass yourtruststorepass

Step 5: Starting Kafka Server

With configurations done, start your Kafka broker:

./bin/kafka-server-start.sh config/server.properties

Step 6: Testing Kafka with SASL_SSL

To ensure everything is functioning as expected, produce and consume messages with your SSL and SASL settings:

1# Producer
2bin/kafka-console-producer.sh --broker-list your.server.address:9093 --topic test --producer.config config/client.properties
3
4# Consumer
5bin/kafka-console-consumer.sh --bootstrap-server your.server.address:9093 --topic test --from-beginning --consumer.config config/client.properties

Key Configuration Parameters

Conclusion

Implementing SASL_SSL in Kafka enhances security by combining authentication and encryption, essential for protecting sensitive data in transit. By carefully configuring each component and testing connectivity, you can ensure a robust security setup in your Kafka deployment.