How to create Avro Kafka Consumer
In the earlier post, we created an Avro
producer that sends stock data objects to a Kafka broker. In this post, we
will create a consumer that reads those stock data objects and prints the content on
the console. For the sake of simplicity, we will not perform any processing on the
received messages but merely print them at the console. However, instead of simply
writing them on the console, you can implement necessary processing logic depending
upon your use case.
This example is an excerpt from the Book Kafka Streams –
Real-time Stream Processing
For a detailed explanation of the example and much more, you can get access to
the Book using below link.
Solution – Avro Consumer
Creating a Kafka consumer is a four-step process, and it is very similar to creating a Kafka producer. Code Listing below shows the necessary code snippet.
You can access fully function project in our GitHub folder.
A typical consumer will process the records that are returned by the
poll() and
poll() again for more data. This infinite loop of polling and processing is
the
fundamental nature of a real-time stream processing application. A typical real-time
application is an always running application that keeps pooling the Kafka broker for
new records and processes them as they arrive.