- This topic has 1 reply, 1 voice, and was last updated 5 years, 7 months ago by
.
Viewing 1 reply thread
-
Posts
-
-
-
- Driver program is responsible for launching various parallel operations on the cluster.
- Driver program contains application’s main() function.
- It is the process which is running the user code which in turn create the SparkContext object, create RDDsand performs transformation and action operation on RDD.
- Driver program access Apache Spark through a SparkContext object which represents a connection to computing cluster (From Spark 2.0 onwards we can access SparkContext object through SparkSession).
- Driver program is responsible for converting user program into the unit of physical execution called task.
- It also defines distributed datasets on the cluster and we can apply different operations on Dataset (transformation and action).
- Spark program creates a logical plan called Directed Acyclic graph which is converted to physical execution plan by the driver when driver program runs.
-
Viewing 1 reply thread
- You must be logged in to reply to this topic.
