In the famous word count example for spark streaming, the spark configuration object is initialized as follows:
/* Create a local StreamingContext
with two working thread and batch
interval of 1 second.
The master requires 2 cores
to prevent from a starvation scenario. */
val sparkConf = new SparkConf().
setMaster("local[2]").
setAppName("WordCount")
Here if I change the master from local[2] to local or does not set the Master, I do not get the expected output and in fact word counting doesn’t happen at all.
The comment says “The master requires 2 cores to prevent from a starvation scenario” that’s why they have done setMaster(“local[2]”).
Can somebody explain me why it requires 2 cores and what is starvation scenario ?