map() transformation

[DataFlair Team]

Explain the map() transformation?

[DataFlair Team]

> map() transformation takes a function as input and apply that function to each element in the RDD.
> Output of the function will be a new element (value) for each input element.
Ex.
val rdd1 = sc.parallelize(List(10,20,30,40))
val rdd2 = rdd1.map(x=>x*x)
println(rdd2.collect().mkString(“,”))

[DataFlair Team]

> map()’s return type need not be the same as its input type.
> i.e. input type may be String but Output type may be of type int

Ex.
val m1 = sc.parallelize(List(1,2,3,4))
val m22 = m1.map(z=>z*0.5)
println(m2.collect().mkString(“,”))

Output :
0.5,1.0, 1.5, 2.0

For more transformation please refer Transformation and Action in Apache Spark RDD

[DataFlair Team]

It passes each element through user-defined function. It returns a new dataset on passing each element to the function. It is applying function on each row / item of RDD. Size of input and output will remain same.
(From <b>http://data-flair.training/blogs/rdd-transformations-actions-apis-apache-spark/#Related_Posts</b&gt;)

[Author]

Posts