- This topic has 1 reply, 1 voice, and was last updated 5 years, 7 months ago by
.
-
Posts
-
-
-
Spark Streaming
Data arriving continuously, in an unbounded sequence is a data stream. Continuously flowing input data is divided into discrete units with the help of streaming for further processing. Through Stream processing analyzing of streaming data is possible. Also, it is a low latency processing.In the year 2103 Spark Streaming was introduced to Apache Spark. It is an extension of the core Spark API. Streaming offers scalable, high-throughput and fault-tolerant stream processing of live data streams. It is possible to do Data ingestion from many sources. For Example Apache Flume, Kafka, Amazon Kinesis or TCP sockets. And, By using complex algorithms that are expressed with high-level functions processing can be done. For example reduce, map, join and window. Afterwards, processed data can be pushed out to live dashboards, filesystems and databases.
Streaming’s Key abstraction is Discretized Stream. It is also known as Spark DStream. A stream of data divided into small batches is represented by it. DStreams are built on Spark’s core data abstraction”RDDs“. Streaming allows integration with any other Apache Spark components like Spark SQL and Spark MLlib.
To know more about Spark Streaming, follow the link: Spark Streaming Tutorial for Beginners
-
- You must be logged in to reply to this topic.
