Front Page Forums Apache Spark What are shared variables in Apache Spark?

This topic contains 1 reply, has 1 voice, and was last updated by  dfbdteam5 2 months, 3 weeks ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #6025

    dfbdteam5
    Moderator

    Explain shared variable in Spark.
    What is need of Shared variable in Apache Spark?

    #6027

    dfbdteam5
    Moderator

    Shared variables are nothing but the variables that can be used in parallel operations. By default, when Apache Spark runs a function in parallel as a set of tasks on different nodes, it ships a copy of each variable used in the function to each task. Sometimes, a variable needs to be shared across tasks, or between tasks and the driver program. Spark supports two types of shared variables: broadcast variables, which can be used to cache a value in memory on all nodes, and accumulators, which are variables that are only “added” to, such as counters and sums.

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.