{"id":14928,"date":"2018-05-04T04:39:18","date_gmt":"2018-05-04T04:39:18","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=14928"},"modified":"2018-05-04T04:39:18","modified_gmt":"2018-05-04T04:39:18","slug":"kafka-streams","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/kafka-streams\/","title":{"rendered":"Kafka Streams | Stream, Real-Time Processing & Features"},"content":{"rendered":"

In our previous Kafka tutorial<\/strong>, we discussed ZooKeeper in Kafka<\/strong>. Today, in this Kafka Streams tutorial, we will learn the actual meaning of Streams in Kafka.<\/p>\n

Also, we will see Kafka Stream architecture, use cases, and Kafka streams feature. Moreover, we will discuss stream processing topology in Apache Kafka.<\/p>\n

Kafka Streams is a client library for building applications and microservices, especially, where the input and output data are stored in Apache Kafka Clusters<\/strong>. <\/span><\/p>\n

Basically, with the benefits of Kafka’s server-side cluster technology, Kafka Streams combines the simplicity of writing and deploying standard Java <\/strong>and Scala<\/strong> applications on the client side.\u00a0<\/span><\/p>\n

So, let’s begin with Apache Kafka Streams.<\/p>\n

First, let’s discuss a little about Stream and Real-Time Kafka Processing<\/p>\n

Stream & Real-Time Processing in Kafka\u00a0<\/span><\/h2>\n

The real-time processing of data continuously, concurrently, and in a record-by-record fashion is what we call Kafka Stream processing.<\/span><\/p>\n

Real-time processing in Kafka is one of the applications of Kafka<\/strong>.\u00a0<\/span><\/p>\n

Basically, Kafka Real-time processing includes a continuous stream of data. Hence, after the analysis of that data, we get some useful data out of it.<\/span><\/p>\n

Now, while it comes to Kafka, real-time processing typically involves reading data from a topic (source), doing some analysis or transformation work, and then writing the results back to another topic (sink). To do this type of work, there are several options.\u00a0<\/span><\/p>\n

Either we can write our own custom code with a Kafka Consumer<\/strong> to read the data and write that data via a Kafka Producer.\u00a0<\/strong><\/span><\/p>\n

Or we use a full-fledged stream processing framework like Spark Streaming<\/strong>, Flink<\/strong>, Storm<\/strong>, etc.<\/span>
\nHowever, there is an alternative to the above options, i.e. Kafka Streams. So, let\u2019s learn about Kafka Streams.<\/span><\/p>\n

What is Kafka Streams?<\/span><\/h2>\n

Kafka Streams, a client library, we use it to process and analyze data stored in Kafka. <\/span><\/p>\n

It relied on important streams processing concepts like properly distinguishing between event time and processing time, windowing support, and simple yet efficient management and real-time querying of application state.\u00a0<\/span><\/p>\n

In addition, Kafka Streams has a low barrier to entry that means we can quickly write and run a small-scale proof-of-concept on a single machine. For that, we only need to run additional instances of our application on multiple machines to scale up to high-volume production workloads.<\/span><\/p>\n

Moreover, by leveraging Kafka’s parallelism model, it transparently handles the load balancing of multiple instances of the same application.<\/span><\/p>\n

Some key points related to Kafka Streams:<\/strong><\/p>\n