Kafka
Spring Boot
YAML
Auto Configuration
Dependencies Management

How to disable all Kafka related auto configuration from yaml/properties file in spring-boot-2 without removing dependencies?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

In Spring Boot 2, Kafka support is usually activated automatically when the relevant dependencies are on the classpath. If you want to keep those dependencies but stop Boot from creating Kafka-related beans for you, the normal solution is to exclude the Kafka auto-configuration classes in application.yml or application.properties.

The Main Property to Use

Spring Boot lets you disable auto-configuration through spring.autoconfigure.exclude. That property accepts one or more fully qualified class names.

For a simple Kafka exclusion in application.yml, use:

yaml
1spring:
2  autoconfigure:
3    exclude:
4      - org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration

The application.properties version is the same idea in a single line:

properties
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration

This tells Boot not to apply the default Kafka setup even though the Kafka libraries are still present.

When a Single Exclusion Is Not Enough

In many applications, excluding KafkaAutoConfiguration is enough because that is the class that creates the usual producer, consumer, and template wiring. If your application also uses annotations such as @EnableKafka or imports custom Kafka configuration, those parts can still create beans separately.

If you want to disable every Kafka-related Boot contribution, it is common to exclude more than one class:

yaml
1spring:
2  autoconfigure:
3    exclude:
4      - org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration
5      - org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration

The exact set depends on what your application has enabled. The key point is that excluding auto-configuration only stops Boot from doing its automatic setup. It does not override explicit configuration that you wrote yourself.

What You Need to Configure Manually

Once auto-configuration is excluded, Boot will no longer create helpful defaults such as ProducerFactory, ConsumerFactory, KafkaTemplate, or listener container support. If your application still needs Kafka in some environments, you must register those beans yourself.

Here is a minimal producer configuration:

java
1package example.kafka;
2
3import java.util.HashMap;
4import java.util.Map;
5
6import org.apache.kafka.clients.producer.ProducerConfig;
7import org.apache.kafka.common.serialization.StringSerializer;
8import org.springframework.context.annotation.Bean;
9import org.springframework.context.annotation.Configuration;
10import org.springframework.kafka.core.DefaultKafkaProducerFactory;
11import org.springframework.kafka.core.KafkaTemplate;
12import org.springframework.kafka.core.ProducerFactory;
13
14@Configuration
15public class ManualKafkaProducerConfig {
16
17    @Bean
18    public ProducerFactory<String, String> producerFactory() {
19        Map<String, Object> props = new HashMap<>();
20        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
21        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
22        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
23        return new DefaultKafkaProducerFactory<>(props);
24    }
25
26    @Bean
27    public KafkaTemplate<String, String> kafkaTemplate() {
28        return new KafkaTemplate<>(producerFactory());
29    }
30}

That configuration is useful when you want full control over when and how Kafka is enabled, instead of letting Spring Boot do it globally.

Why Keep the Dependencies at All

This pattern is common in modular systems, shared libraries, and test-heavy services. You might want Kafka classes available for a profile-specific configuration, for integration tests, or for optional features that are enabled only in some deployments.

For example, you may keep the dependency in the build file but activate your manual configuration only under a profile:

java
1package example.kafka;
2
3import org.springframework.context.annotation.Configuration;
4import org.springframework.context.annotation.Profile;
5
6@Configuration
7@Profile("kafka")
8public class KafkaFeatureProfile {
9}

Combined with the exclusion property, this gives you predictable behavior. Kafka is not started accidentally, but it is still available when the kafka profile is enabled and your manual beans are registered.

YAML Versus Annotation Exclusion

You can also exclude auto-configuration in Java with @SpringBootApplication(exclude = ...), but the property-based approach is often better when the goal is environment control rather than code-level design. A YAML or properties entry can be changed per environment without recompiling the application.

That is especially useful when you want Kafka disabled in local development but enabled through a different configuration package in staging or production.

Common Pitfalls

The most common mistake is excluding KafkaAutoConfiguration and expecting that to disable custom Kafka beans too. It does not. If you have @Bean methods, @EnableKafka, or imported configuration classes, those can still activate Kafka pieces.

Another mistake is using a single comma-separated value in YAML where a list is clearer. Boot can parse either style in many cases, but the YAML list form is less error-prone when you exclude multiple classes.

A third issue is forgetting downstream dependencies. If another bean expects KafkaTemplate or a listener container factory and you excluded auto-configuration without replacing those beans, the application may fail at startup with missing-bean errors.

Finally, do not confuse disabling auto-configuration with removing network access to Kafka. Excluding Boot wiring only changes bean creation inside the application context.

Summary

  • Use spring.autoconfigure.exclude to disable Kafka auto-configuration while keeping dependencies installed.
  • 'KafkaAutoConfiguration is the usual first class to exclude in Spring Boot 2.'
  • Excluding auto-configuration does not disable custom Kafka beans you created yourself.
  • If the application still needs Kafka, define ProducerFactory, ConsumerFactory, or KafkaTemplate manually.
  • Property-based exclusion is often cleaner than hard-coding exclusions in annotations when behavior changes by environment.

Course illustration
Course illustration

All Rights Reserved.