Apache Kafka Foundation Course - How to compile Kafka Code

Welcome back to Apache Kafka Tutorial. This video is an update for Apache Kafka 1.x. In all my tutorial videos, I am using SBT to compile and execute my examples. Many of my viewers are new to SBT. SBT is Scala build tool. However, It is equally useful for Java as well. I keep getting complains from many of my viewers about SBT installation.

Installing SBT for Kafka

In this video, I will show you step by step process to install, compile and execute one of my examples in GCP Kafka VM. So, let’s start.

  1. SSH to your Kafka VM.
  2. Check if you already have a Java compiler. Just type javac on the command prompt. You might see a message as command not found. You can fix this problem by installing open JDK.
  3. Use below command to install JDK.
    sudo apt-get install openjdk-8-jdk                                          
  1. Next step is to install SBT. Start your browser and visit Scala SBT home page
  2. Navigate to SBT documentation and visit latest SBT documentation.
  3. Look for installing SBT on Linux. Your GCP Kafka machine is using Debian OS. So, we follow the installation procedure for Debian.
  4. First step is to setup SBT repository.
    echo "deb https://dl.bintray.com/sbt/debian /" | sudo tee -a /etc/apt/sources.list.d/sbt.list      
  1. Update the apt-get, and finally install SBT.
    sudo apt-get update
    sudo apt-get install sbt                                        

That's it. You are ready to start SBT for the first time. Create a working directory, change your working directory and start SBT by typing sbt command.

Compile and Execute Kafka Producer

Now let's compile and execute our first example. Follow these steps.

  1. Create a build.sbt file, and place following content in the file.
    name := "KafkaTest"

    scalaVersion := "2.11.8"
    libraryDependencies ++= Seq(
        "org.apache.kafka" % "kafka-clients" % "1.0.0")                                     
  1. Create a source code file and copy the below code in the file.
    //File Name-SimpleProducer.java
    import java.util.*;
    import org.apache.kafka.clients.producer.*;
    public class SimpleProducer {
        public static void main(String[] args) throws Exception{
            String topicName = "SimpleProducerTopic";
            String key = "Key1";
            String value = "Value-1";
            Properties props = new Properties();
            props.put("bootstrap.servers", "localhost:9092,localhost:9093");
            props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
            Producer<String, String> producer = new KafkaProducer<>(props);
            ProducerRecord<String, String> record = new ProducerRecord<>(topicName,key,value);
            System.out.println("SimpleProducer Completed.");

The source code is same as we used in one of my tutorial video. The code is also available in Learning Journal GitHub repository.

  1. You are now ready to compile our simple producer example. Execute the command as shown below to compile your program.
    sbt compile                                          
  1. You should see a success message in the end. You can execute your program using sbt run command.

Great. If you want to see the message, you can start a console consumer. Use below command to execute the consumer.

    kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic SimpleProducerTopic --from-beginning 

We sent just one message, so your consumer shows a single message.
Great. That's it for this session.
Thank you for watching learning journal. Keep learning and keep growing.

You will also like:

Kafka Core Concepts

Learn Apache Kafka core concepts and build a solid foundation on Apache Kafka.

Learning Journal

Scala placeholder syntax

What is a scala placeholder syntax and why do we need it? Learn from experts.

Learning Journal

Pattern Matching

Scala takes the credit to bring pattern matching to the center.

Learning Journal

Pure Function benefits

Pure Functions are used heavily in functional programming. Learn Why?

Learning Journal

Immutability in FP

The literal meaning of Immutability is unable to change? How to program?

Learning Journal