Installing librdkafka on Ubuntu 16.04 can sometimes present challenges, primarily because of its dependencies and how it interfaces with various versions of PHP or Python through extensions like php-rdkafka and confluent-kafka-python. Here we will cover common issues and solutions in installing librdkafka, and subsequently php-rdkafka or confluent-kafka-python.

Understanding librdkafka

librdkafka is a C library implementation of the Apache Kafka protocol, providing Producer, Consumer, and Admin clients. It is designed for high performance and works in a variety of Unix-based systems, including Linux.

Prerequisites

Before installing librdkafka, ensure that you have the following prerequisites installed:

• build-essential
• libsasl2-dev
• libssl-dev
• zlib1g-dev
• pkg-config

You can install these using the command:

sudo apt-get install build-essential libsasl2-dev libssl-dev zlib1g-dev pkg-config

Installing librdkafka from Source

The librdkafka GitHub repository often has the most recent version. To compile from source:

1. Clone the repository:

    git clone https://github.com/edenhill/librdkafka.git

1. Change to the librdkafka directory:

    cd librdkafka

1. Build and install:

    ./configure
    make
    sudo make install

Potential Issues and Resolutions

During installation, you might encounter some issues related to library paths or missing dependencies:

Issue: Missing dependency packages

Resolution: Ensure all required dependencies mentioned earlier are installed.

Issue: librdkafka.so not found by linker

Resolution: Sometimes after installation, the linker might not find librdkafka. To resolve this:

• Update the LD_LIBRARY_PATH environment variable:

    export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

• Alternatively, run sudo ldconfig after installing librdkafka.

Installing Kafka PHP Extension (php-rdkafka)

If you are working with PHP, you may want to install the php-rdkafka extension, which depends on librdkafka:

1. Install PHP development tools and php-pear if not already installed:

    sudo apt-get install php-dev php-pear

1. Install php-rdkafka via PECL:

    pecl install rdkafka

1. Add extension=rdkafka.so to your php.ini.

Installing Kafka Python Extension (confluent-kafka-python)

For Python developers using Kafka, confluent-kafka-python provides Kafka integration:

1. Ensure you have pip installed:

    sudo apt-get install python-pip

1. Install the Python package:

    pip install confluent-kafka

Summary

Conclusion

Installing librdkafka and its related extensions on Ubuntu 16.04 involves careful management of dependencies and library paths. By following the detailed steps and troubleshooting common issues, developers can successfully leverage Kafka’s capabilities within their applications on older versions of Ubuntu.