How to create Synchronous Kafka Producer
Default Kafka producer send API is asynchronous and nonblocking. When you call the
send API, it merely adds the ProducerRecord into the buffer and returns
immediately.
Asynchronous and non-blocking send is efficient. However, it also leaves the
possibility to lose some messages without even knowing that some of your messages
never reached the Kafka broker. If your use case demands, you can easily turn your
send method into a synchronous blocking call and wait for the acknowledgment or an
exception. Let’s create an example to understand the mechanics.
We created a Hello
Producer
in an earlier post.
The example uses the default asynchronous send() method to
deliver some Kafka messages. Now, we want to take the same example and change the
send() method call to a synchronous blocking call. We also want to capture
the
metadata acknowledgment and print the offset number at which the message is
persisted by the broker.
This example is an excerpt from the Book Kafka Streams –
Real-time Stream Processing
For a detailed explanation of the example and much more, you can get access to
the Book using below link.
Solution – Sync Producer
Implementing synchronous send() is straightforward. Code Listing below shows the code snippet to achieve synchronous send and collect metadata sent by the broker.
You can access fully function project in our GitHub folder.