Great. Let's start.
Installing Kafka in Google Cloud
I am assuming you already have access to Google Cloud Account. You can follow these steps to set up a single node Kafka VM in Google Cloud.
- Login to your GCP account.
- Go to GCP products and services menu.
- Click Cloud Launcher
- Search for Kafka. You will see multiple options. For a single node setup, I prefer to use the Google VM Image. You can also try the single node Bitnami image. There is a multi-node Bitnami Image as well. However, they designed it for production usage with larger VM configurations. For your learning purposes, the Google image is good enough.
- Select the Kafka VM Image.
- You will notice the Kafka version, Operating system, and other packages.
- Your Kafka usage is free, but you have a cost associated with the VM for CPU, Memory and the disk space. But don't worry about it. Google is charging you on a per hour basis. You also have a free credit for a year.
- Click the launch button. You can review and change some configurations, but the default settings are fair enough.
- Click the Deploy button at the bottom of the page. That's it. Just wait for few minutes and GCP will launch your single node Kafka VM.
- You can SSH to the VM from your deployment page, or you can go to your homepage, then visit the compute engine page, and SSH to your VM.
- Once you finish your work, you can select the VM and stop it. Your billing will stop. You can come back next day, pick the VM and start it again.
- You Kafka VM comes preconfigured. All services are up and running. You can start using it right away.
Kafka Quick Start
Let's do some simple things. Go to Kafka documentation quick start page. You don't need to perform step 1 and step 2. We already completed the setup, and the services are already running. You can copy the command to create a Kafka topic and execute it. Our Kafka VM comes with appropriate PATH settings. So, we just copy the command. There is no need to specify the path.
Next command is to list the available topics.
You can open two SSH windows. Start a console consumer in one window and a console producer in another window. Let's do that. Start the producer.
Start a consumer.
Now, you can type some messages in the producer window. You will see them arriving in the
consumer window. Press
to terminate the consumer and similarly the producer.
If you no longer need the Kafka VM, navigate to deployment manager and delete it. Removing your VM directly, doesn't clean up everything. So, instead of just deleting your VM from compute instance page, make sure to delete it from your deployments page.
Great. That's it for this session.
Thank you for watching Learning Journal. Keep learning and Keep growing.