Apache Kafka, a distributed streaming platform, provides a robust security framework which includes Access Control Lists (ACLs) to manage permissions over Kafka resources. ACLs are essential when you want to secure your Kafka cluster by specifying which users or applications are allowed to perform operations on topics, consumer groups, and other Kafka resources.

Understanding Kafka ACLs

ACLs in Kafka are used to grant or deny permissions to Kafka users. These permissions control the ability to read, write, and configure Kafka resources. Kafka uses an authorization module as part of its security features, which checks ACLs to determine whether a given action (produce, consume, admin operations) by a user or application is allowed.

Setting Up Kafka ACLs

To manage ACLs, Kafka provides a command-line tool called kafka-acls.sh. This tool interacts with the Kafka cluster to add, remove, or list ACLs.

Here's an overview of how to use the kafka-acls.sh utility:

Adding ACLs

To add an ACL, you will specify the allowed operation, the user or application it applies to, and the Kafka resource. Here is an example command:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
  --add --allow-principal User:alice --operation Read --topic my-topic

This command allows the user alice to read from the my-topic topic.

Removing ACLs

To remove an ACL, you might use a command like this:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
  --remove --allow-principal User:alice --operation Read --topic my-topic

This command removes the read permission for alice on my-topic.

Listing ACLs

To list the ACLs active in your cluster:

bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list

This will show all ACLs currently configured in the cluster.

Different Types of Operations

ACLs in Kafka can control several types of operations such as:

• Read: Allows consuming messages from a topic.
• Write: Allows producing messages to a topic.
• Create: Allows creating a topic.
• Delete: Allows deleting a topic.
• Alter: Allows modifying topic configurations.
• Describe: Allows viewing topic configurations.

Practical Example

Suppose you want to set up ACLs for a scenario where user bob should only be able to write to my-topic, and user alice should be able to read from the same topic. You would execute:

1# Allow bob to write
2bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
3  --add --allow-principal User:bob --operation Write --topic my-topic
4
5# Allow alice to read
6bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 \
7  --add --allow-principal User:alice --operation Read --topic my-topic

Best Practices

• Principle of Least Privilege: Always grant the minimum permissions necessary for users and applications.
• Regular Audits: Periodically review and audit your ACL settings to ensure they still align with your security policies.
• Use Secure Connections: When modifying ACLs, ensure your connections are secure to prevent interception.

Summary Table

Here’s a quick reference table for common ACL commands:

Kafka ACLs are a powerful tool for securing your Kafka cluster. Understanding and utilizing them effectively can help you maintain robust security and operational efficiency.