5 ways to check Apache Kafka version

Introduction

Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. It is widely used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, and incredibly fast. This guide will walk you through several methods of checking the version of Apache Kafka you have running, from basic commands to more advanced topics.

Using Kafka Command Line Tools

The simplest way to check the version of Kafka is by utilizing the command line tools that come with Kafka. In the bin directory of your Kafka installation, you should see a variety of scripts to run Kafka. To check the version, you can do the following:

$ ./kafka-topics.sh --version

This will print out the version of Kafka that you are running. The output should look something like this:

$ Kafka 2.6.0 (Commit:3a48f11)

Inspecting Kafka Logs

Another method to find out the version of Kafka is by inspecting the logs when Kafka starts up. The logs usually have an entry that states the Kafka version. Since the location of logs can vary depending on your setup, a common location for these logs is:

$ cat /usr/local/kafka/logs/server.log

Search for an entry that resembles the following:

[2023-01-01 00:00:00,000] INFO Kafka version: 2.6.0 (org.apache.kafka.common.utils.AppInfoParser)

Querying Kafka Broker Through Zookeeper Shell

If your Kafka cluster uses Zookeeper, you can connect to the Zookeeper instance and query Kafka version-related information from the broker information that Zookeeper maintains. You can use the Zookeeper shell that ships with Kafka to connect to Zookeeper and issue queries:

$ ./zookeeper-shell.sh localhost:2181
ls /brokers/ids
get /brokers/ids/1

After executing the ‘get’ command against the broker id, you will see JSON output that includes version information:

{"jmx_port":9999,"timestamp":"1609459200000","endpoints":["PLAINTEXT://your.kafka.broker:9092"],"host":"your.kafka.broker","version":5,"port":9092}

Note that in the above output, the version field represents the broker’s API version and not the actual Kafka distribution version. This can be less straightforward for correlating with Kafka releases.

Using Kafka Broker REST APIs

If you have enabled REST APIs on your Kafka brokers, you can make REST request to endpoints provided by Kafka to get broker information which includes the version as well. Here’s an example using curl:

$ curl http://your.kafka.broker:8080/v2/kafka/info
curl http://your.kafka.broker:8080/v2/kafka/cluster

The response you get back will be in JSON format, containing various metadata including the version of Kafka:

{
    "version": "2.6.0",
    ...
}

Programmatically Accessing Kafka Version

Another advanced way to check the Kafka version is programmatically using Kafka’s own client libraries:

import org.apache.kafka.common.utils.AppInfoParser;

public class KafkaVersionChecker {

    public static void main(String[] args) {
        System.out.println("Kafka version: " + AppInfoParser.getVersion());
    }
}

When you run the above Java code, it will output the version of the Kafka client library in use. This is typically the same version as your Kafka broker version, but it’s important to ensure that your client and server versions are compatible.

Conclusion

Being able to verify the Apache Kafka version running in your environment is crucial for maintaining compatibility with other systems and within your Kafka cluster. The methods outlined here range from simple command line checks to programmatically checking the version. Each method has its own use case depending on the level of detail required and the environment setup.