In which case one cannot implement a Combiner?

Free Online Certification Courses – Learn Today. Lead Tomorrow. Forums Apache Hadoop In which case one cannot implement a Combiner?

Viewing 1 reply thread
  • Author
    Posts
    • #5336
      DataFlair TeamDataFlair Team
      Spectator

      Combiner is used as an optimization technique. But what are the prerequisites to implement combiner ?
      In which case one cannot implement a Combiner?

    • #5338
      DataFlair TeamDataFlair Team
      Spectator

      The combiner is as semi-reducer in MapReduce. It is an optional class which can be specified in MapReduce driver class to process the output of map tasks before submitting it to reducer tasks.

      The main purpose of combine class is, usually the output from map task is huge and data transfer from map to reduce will be high. Since data transfer is expensive and to limit the volume of data transfer between map and reduce tasks Combiner
      function summarizes the map output records with the same Key. The output of combiner will be sent over to reducer as input.

      • Combiner does not have its own interface, it implements Reduce Interface and reduce () method.
      • There is no guarantee on how many times a combiner function will be called for each map output key. At times it may not be executed at all.

      Follow the link to learnj more about Combiner in Hadoop

Viewing 1 reply thread
  • You must be logged in to reply to this topic.