How to create Synchronous Kafka Producer

Default Kafka producer send API is asynchronous and nonblocking. When you call the send API, it merely adds the ProducerRecord into the buffer and returns immediately. Asynchronous and non-blocking send is efficient. However, it also leaves the possibility to lose some messages without even knowing that some of your messages never reached the Kafka broker. If your use case demands, you can easily turn your send method into a synchronous blocking call and wait for the acknowledgment or an exception. Let’s create an example to understand the mechanics.
We created a Hello Producer in an earlier post. The example uses the default asynchronous send() method to deliver some Kafka messages. Now, we want to take the same example and change the send() method call to a synchronous blocking call. We also want to capture the metadata acknowledgment and print the offset number at which the message is persisted by the broker.
This example is an excerpt from the Book Kafka Streams – Real-time Stream Processing
For a detailed explanation of the example and much more, you can get access to the Book using below link.

Solution – Sync Producer

Implementing synchronous send() is straightforward. Code Listing below shows the code snippet to achieve synchronous send and collect metadata sent by the broker.

You can access fully function project in our GitHub folder.

Read More

Author : Prashant Pandey -

You will also like:

When to use <figure>

The why and where of figure tag in HTML5.

Learning Journal

What is Programming and Programming Language

"PROGRAMMING" is the word which we all come across quite often. Let us see what that means.

Learning Journal

Scala Variable length arguments

How do you create a variable length argument in Scala? Why would you need it?

Learning Journal

Local Functions

How do you implement private methods in a functional programming language.

Learning Journal

Statements and Expressions

Statements and Expressions in Scala. How are they different?

Learning Journal