Explain the level of parallelism in spark streaming.

[DataFlair Team]

Explain the level of parallelism in spark streaming.

[DataFlair Team]

> In order to reduce the processing time, one need to increase the parallelism.
> In Spark Streaming, there are three ways to increase the parallelism :
(1) Increase the number of receivers : If there are too many records for single receiver (single machine) to read in and distribute so that is bottleneck. So we can increase the no. of receiver depends on scenario.
(2) Re-partition the receive data : If one is not in a position to increase the no. of receivers in that case redistribute the data by re-partitioning.
(3) Increase parallelism in aggregation :

for complete guide on Spark Streaming you may refer to Apache Spark-Streaming guide

[Author]

Posts