How to identify that, given operation is Transformation/Action in your program?

Live instructor-led & Self-paced Online Certification Training Courses (Big Data, Hadoop, Spark) Forums Apache Spark How to identify that, given operation is Transformation/Action in your program?

Viewing 1 reply thread
  • Author
    Posts
    • #5305
      DataFlair Team
      Moderator

      How to identify that, given operation is Transformation/Action in your program?

    • #5307
      DataFlair Team
      Moderator

      In order to identify the operation, one need to look at the return type of an operation.

      • If operation returns a new RDD in that case an operation is ‘Transformation’
      • If operation returns any other type than RDD in that case an operation is ‘Action’

      Hence, Transformation constructs a new RDD from an existing one (previous one) while Action computes the result based on applied transformation and returns the result to either driver program or save it to the external storage.

      Also refer to operations of RDD in Apache Spark and its Operations

Viewing 1 reply thread
  • You must be logged in to reply to this topic.