map instead of flatMap

[DataFlair Team]

In the code for Stack Exchange data analysis 2nd KPI (Monthly questions count), I tried replacing map for flatMap in the line “flatMap{line=> {val xml = XML.loadString”. I got an error and the execution failed because the result of the map operation was not parseable by DateFormat. I checked the outputs of map and flatMap functions before moving to the date format step.

flatMap gives the output as given below:
List(2015-11-22T16:18:47.337)
List(2015-11-23T05:47:01.023)
List(2015-11-23T09:40:01.580) and so on.

map gives the output as given below:
Some(List(2014-05-13T23:58:30.457))
Some(List(2014-05-14T00:11:06.457))
Some(List(2014-05-14T01:25:59.677)) and so on.

Please explain this difference.

[DataFlair Team]

> Please note that, flatMap is not replacement for map.
> map() transformation, takes a function and applied that function to each element in the dataset and produce output for each element which is one-to-one of input element. One more difference is, map() output type need not be the same as input type.
> While flatMap() transformation produce an iterable output. Let’s take the example of WordCount example, in WordCount example we take one line and split the no. of words in a line so using flatMap() we can achieve the same (you can observe that you can iterate over the output i.e. (dataflair, 1), (technology,1) and so on) while the same is not possible with map().

> For more explanation, please go through map() and flatMap() example given by me in this forum.

[Author]

Posts