Introduction

Cannot import name KafkaConsumer usually means Python is importing the wrong module or the expected package is not installed in the environment you are actually using. The fix is rarely in Kafka itself. It is almost always about package selection, virtual environment confusion, or a local file shadowing the real library.

The Expected Import

If you are using the kafka-python package, the normal import is:

from kafka import KafkaConsumer

A minimal example looks like this:

1from kafka import KafkaConsumer
2
3consumer = KafkaConsumer(
4    "orders",
5    bootstrap_servers=["localhost:9092"],
6    auto_offset_reset="earliest",
7    group_id="demo-group",
8)
9
10for message in consumer:
11    print(message.value)

If that import fails, the next step is not to rewrite the code. It is to verify what kafka module Python is actually loading.

Check the Installed Package

The correct package name for that import is kafka-python, not just kafka.

python -m pip show kafka-python
python -m pip show kafka

If kafka-python is missing, install it into the interpreter you are using.

python -m pip install kafka-python

Using python -m pip is safer than running bare pip, because it ties the installation to the same interpreter that runs your script.

Look for a Shadowing File

One of the most common causes is a local file or folder named kafka.py or kafka/ in your project. Python may import that instead of the installed package.

For example, this layout causes trouble:

project/
  kafka.py
  app.py

Then this code in app.py can fail in confusing ways:

from kafka import KafkaConsumer

To debug, inspect what module is being imported.

import kafka
print(kafka.__file__)

If the printed path points into your project instead of the site-packages directory, you found the problem.

Virtual Environment Mismatch

Another common issue is installing kafka-python in one environment and running the script in another.

Check the interpreter path first:

python -c "import sys; print(sys.executable)"
python -m pip show kafka-python

If you use an IDE, Jupyter, or a task runner, confirm that it points at the same environment. Import issues that look mysterious from the terminal are often just environment mismatches in the editor.

Package Conflicts and Old Installs

Sometimes the wrong package is installed. The package named kafka is not the same thing as kafka-python, and having both can create confusion.

A clean fix is often:

python -m pip uninstall kafka -y
python -m pip uninstall kafka-python -y
python -m pip install kafka-python

Then retest the import in a fresh shell or environment.

Minimal Verification Script

Once the package situation is fixed, verify the import in a tiny script before going back to the full application.

from kafka import KafkaConsumer

print("KafkaConsumer imported successfully")

If this works, the environment is correct and you can move on to broker connectivity or runtime configuration.

When the Library Choice Is Different

Some Kafka Python examples use confluent-kafka instead of kafka-python. That library has a different API.

For confluent-kafka, the import looks like:

from confluent_kafka import Consumer

So if you copied code from one client library and installed another, the import will fail even though both libraries relate to Kafka.

Make sure the code and the installed package belong to the same client library.

Common Pitfalls

A common mistake is installing the wrong package, especially kafka instead of kafka-python.

Another mistake is naming a local file kafka.py, which shadows the real package.

Developers also often forget that their IDE, notebook, or service runner is using a different interpreter than the shell where they installed the package.

Finally, do not mix sample code from kafka-python and confluent-kafka. Their imports and APIs are not interchangeable.

Summary

• The expected import for kafka-python is from kafka import KafkaConsumer.
• Verify that kafka-python is installed in the same interpreter that runs your code.
• Check for local files or folders named kafka that shadow the package.
• Remove conflicting packages if the environment is messy.
• Confirm which Kafka client library your code is actually written for.