Free Online Certification Courses – Learn Today. Lead Tomorrow. › Forums › Apache Flink › map instead of flatMap
- This topic has 1 reply, 1 voice, and was last updated 5 years, 5 months ago by DataFlair Team.
-
AuthorPosts
-
-
October 10, 2018 at 4:03 pm #6583DataFlair TeamSpectator
In the code for Stack Exchange data analysis 2nd KPI (Monthly questions count), I tried replacing map for flatMap in the line “flatMap{line=> {val xml = XML.loadString”. I got an error and the execution failed because the result of the map operation was not parseable by DateFormat. I checked the outputs of map and flatMap functions before moving to the date format step.
flatMap gives the output as given below:
List(2015-11-22T16:18:47.337)
List(2015-11-23T05:47:01.023)
List(2015-11-23T09:40:01.580) and so on.map gives the output as given below:
Some(List(2014-05-13T23:58:30.457))
Some(List(2014-05-14T00:11:06.457))
Some(List(2014-05-14T01:25:59.677)) and so on.Please explain this difference.
-
October 10, 2018 at 4:03 pm #6584DataFlair TeamSpectator
> Please note that, flatMap is not replacement for map.
> map() transformation, takes a function and applied that function to each element in the dataset and produce output for each element which is one-to-one of input element. One more difference is, map() output type need not be the same as input type.
> While flatMap() transformation produce an iterable output. Let’s take the example of WordCount example, in WordCount example we take one line and split the no. of words in a line so using flatMap() we can achieve the same (you can observe that you can iterate over the output i.e. (dataflair, 1), (technology,1) and so on) while the same is not possible with map().> For more explanation, please go through map() and flatMap() example given by me in this forum.
-
-
AuthorPosts
- You must be logged in to reply to this topic.