Kafka custom AuthenticateCallbackHandler

Apache Kafka

Kafka Security

AuthenticateCallbackHandler

Custom CallbackHandler

Kafka Configuration

Kafka custom AuthenticateCallbackHandler

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Kafka & Streaming Security & Auth

Introduction

Kafka exposes AuthenticateCallbackHandler for SASL flows that need custom login or validation behavior. Most Kafka deployments never implement it directly because built-in mechanisms like SCRAM and PLAIN are configured through JAAS, but it becomes important when you integrate with OAUTHBEARER or another custom authentication flow.

What the Interface Does

AuthenticateCallbackHandler extends Java's CallbackHandler and adds lifecycle configuration through configure and close. Kafka calls handle during authentication and passes SASL-specific callback objects that your code must understand.

At a high level:

'configure receives Kafka configs, the SASL mechanism, and JAAS entries'
'handle processes callback requests'
'close releases resources'

That contract is simple, but the exact callback types depend on whether the handler is running on a client or a broker and which SASL mechanism is active.

A Minimal Handler Skeleton

The following example shows the structure of a custom handler. It is intentionally small, but it follows the real interface contract.

java

1package com.example.kafka.security;
2
3import java.io.IOException;
4import java.util.List;
5import java.util.Map;
6import javax.security.auth.callback.Callback;
7import javax.security.auth.callback.UnsupportedCallbackException;
8import javax.security.auth.login.AppConfigurationEntry;
9import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler;
10
11public class DemoCallbackHandler implements AuthenticateCallbackHandler {
12
13    private String saslMechanism;
14
15    @Override
16    public void configure(
17            Map<String, ?> configs,
18            String saslMechanism,
19            List<AppConfigurationEntry> jaasConfigEntries) {
20        this.saslMechanism = saslMechanism;
21    }
22
23    @Override
24    public void handle(Callback[] callbacks)
25            throws IOException, UnsupportedCallbackException {
26        for (Callback callback : callbacks) {
27            throw new UnsupportedCallbackException(
28                    callback,
29                    "Unsupported callback for mechanism " + saslMechanism
30            );
31        }
32    }
33
34    @Override
35    public void close() {
36    }
37}

This class compiles with Kafka client dependencies and gives you the correct starting point.

A More Practical `OAUTHBEARER` Example

One real use case is supplying an OAuth bearer token on the client side. Kafka's OAuth support uses callback handlers to obtain the token during login.

java

1package com.example.kafka.security;
2
3import java.io.IOException;
4import java.time.Instant;
5import java.util.Collections;
6import java.util.List;
7import java.util.Map;
8import java.util.Set;
9import javax.security.auth.callback.Callback;
10import javax.security.auth.callback.UnsupportedCallbackException;
11import javax.security.auth.login.AppConfigurationEntry;
12import org.apache.kafka.common.security.auth.AuthenticateCallbackHandler;
13import org.apache.kafka.common.security.oauthbearer.OAuthBearerToken;
14import org.apache.kafka.common.security.oauthbearer.OAuthBearerTokenCallback;
15
16public class StaticTokenLoginCallbackHandler implements AuthenticateCallbackHandler {
17
18    @Override
19    public void configure(
20            Map<String, ?> configs,
21            String saslMechanism,
22            List<AppConfigurationEntry> jaasConfigEntries) {
23    }
24
25    @Override
26    public void handle(Callback[] callbacks)
27            throws IOException, UnsupportedCallbackException {
28        for (Callback callback : callbacks) {
29            if (callback instanceof OAuthBearerTokenCallback tokenCallback) {
30                tokenCallback.token(new SimpleToken("demo-token"));
31            } else {
32                throw new UnsupportedCallbackException(callback);
33            }
34        }
35    }
36
37    @Override
38    public void close() {
39    }
40
41    private static final class SimpleToken implements OAuthBearerToken {
42        private final String value;
43
44        private SimpleToken(String value) {
45            this.value = value;
46        }
47
48        @Override
49        public String value() {
50            return value;
51        }
52
53        @Override
54        public Set<String> scope() {
55            return Collections.emptySet();
56        }
57
58        @Override
59        public long lifetimeMs() {
60            return Instant.now().plusSeconds(300).toEpochMilli();
61        }
62
63        @Override
64        public String principalName() {
65            return "service-account";
66        }
67
68        @Override
69        public Long startTimeMs() {
70            return Instant.now().toEpochMilli();
71        }
72    }
73}

This example is intentionally simple. In production, the token would normally come from an OAuth provider rather than being hardcoded.

Wiring the Handler Into Kafka

A client-side login handler is configured with the login callback property.

properties

security.protocol=SASL_SSL
sasl.mechanism=OAUTHBEARER
sasl.login.callback.handler.class=com.example.kafka.security.StaticTokenLoginCallbackHandler

Broker-side handlers use listener-scoped properties, and server-side validation handlers are a different concern from client-side login handlers. Make sure you are configuring the right side of the connection.

When You Actually Need a Custom Handler

You usually do not need this interface for:

SCRAM with username and password
PLAIN with JAAS config
standard TLS client certificates

You do need it when Kafka expects a callback-driven authentication exchange, especially with OAuth token retrieval or custom plugins.

Common Pitfalls

The most common mistake is implementing a handler for the wrong callback type. Kafka will call your code with specific callback objects tied to the mechanism, so a handler written for OAUTHBEARER does not make sense for SCRAM.

Another pitfall is putting secret retrieval, HTTP calls, or token parsing into handle without thinking about latency and failure handling. Authentication paths should be predictable and well-instrumented.

Developers also confuse login callback handlers with server callback handlers. The configuration property names are similar, but they serve different roles on different sides of the connection.

Finally, do not swallow unsupported callbacks. Throwing UnsupportedCallbackException is often the safest behavior because it reveals misconfiguration early.

Summary

'AuthenticateCallbackHandler is Kafka's hook for callback-driven SASL authentication flows.'
Most deployments do not need it for SCRAM or PLAIN.
Implement configure, handle, and close, then support the exact callback types your mechanism uses.
Client login handlers and broker validation handlers are different pieces of configuration.
Keep authentication code explicit, fast, and strict about unsupported callbacks.

System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Kafka custom AuthenticateCallbackHandler

System Design practice on Codemia

Introduction

What the Interface Does

A Minimal Handler Skeleton

A More Practical OAUTHBEARER Example

Wiring the Handler Into Kafka

When You Actually Need a Custom Handler

Common Pitfalls

Summary

Related reading

System Design Fundamentals

Track what you have practised

System Design practice on Codemia

A More Practical `OAUTHBEARER` Example