Add jar to general Kafka Connect classpath in the Confluent Docker
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Introduction
In Confluent's Kafka Connect Docker images, there are two different questions people often mix together: "how do I add a connector plugin" and "how do I add a library to the worker-wide classpath." For connector JARs, the preferred mechanism is usually CONNECT_PLUGIN_PATH, while CLASSPATH is the more general but riskier escape hatch for worker-level libraries.
Prefer Plugin Isolation for Connectors
If you are adding a connector, converter, or SMT plugin, the normal approach is to place it in its own directory and point Kafka Connect at that plugin path.
A Docker Compose example looks like this:
Each plugin should ideally live in its own subdirectory under the mounted plugin path. That keeps dependency isolation cleaner and matches how Kafka Connect expects plugin discovery to work.
When You Actually Need General Classpath
Sometimes you truly need a library available to the worker process itself, not just inside one isolated plugin directory. In that case, you can extend the image and set CLASSPATH.
This makes the JARs visible to the general JVM classpath for the Connect worker.
That can be useful for:
- custom security providers
- worker-wide libraries
- special dependencies not resolved well through plugin isolation
But it is also easier to create classpath conflicts this way.
Why CLASSPATH Is Not the First Choice
Kafka Connect plugin isolation exists to reduce dependency collisions between connectors. If you push too much into the general classpath, you undo some of that isolation and increase the chance of:
- version conflicts
- class shadowing
- surprising runtime behavior between connectors
So the practical rule is:
- use
CONNECT_PLUGIN_PATHfor connector-style additions - use
CLASSPATHonly when you really need worker-wide visibility
Custom Image Versus Volume Mount
For development, a mounted volume can be fine. For production, a custom image is usually safer because it is reproducible and easier to promote through environments.
Example custom image with plugin directory:
This avoids depending on a host-mounted directory layout at runtime.
Verify That Connect Sees the Plugin
After the worker starts, confirm discovery through the REST API:
If the plugin does not appear, the most common causes are:
- wrong directory structure
- wrong
CONNECT_PLUGIN_PATH - missing transitive JARs
- stale container image
For general classpath issues, the plugin may appear but fail later with ClassNotFoundException or dependency conflicts in the worker logs.
That distinction matters because discovery failures and runtime linkage failures have different fixes. One is usually a path problem, while the other is usually a dependency or classloader problem.
Common Pitfalls
- Using
CLASSPATHfor connector plugins when plugin isolation is the better mechanism. - Dumping multiple unrelated connector JARs into one flat directory and making dependency conflicts harder to diagnose.
- Forgetting to rebuild the Docker image after adding new JARs.
- Assuming plugin discovery and worker classpath are the same thing.
- Ignoring
/connector-pluginsoutput and debugging blindly from connector creation failures.
Summary
- In Confluent Docker, connector plugins should usually be added through
CONNECT_PLUGIN_PATH. - Use
CLASSPATHonly when you need worker-wide library visibility. - Prefer custom images over ad hoc mounted JAR directories for reproducible deployments.
- Keep plugins isolated in their own directories to reduce dependency conflicts.
- Verify plugin discovery through the Kafka Connect REST API after startup.

