What is Map in Apache Spark

Viewing 1 reply thread
  • Author
    • #5732
      DataFlair TeamDataFlair Team

      What is Map transformation operation in Apache Spark?
      What is the need for the Map transformation?
      What processing can be done in the Map in Spark explain with example

    • #5733
      DataFlair TeamDataFlair Team

      Map is a transformation applied to each element in a RDD and it provides a new RDD as a result. In Map transformation, user-defined business logic will be applied to all the elements in the RDD.
      It is similar to FlatMap, but unlike FlatMap Which can produce 0, 1 or many outputs, Map can only produce one to one output.
      Map operation will transforms an RDD of length N into another RDD of length N.

      Map Operation

      Map transformation will not shuffle data from one partition to many. It will keep the operation narrow.

Viewing 1 reply thread
  • You must be logged in to reply to this topic.