How many Mapper and Reducer will run if the input directory of the job is empty?

Free Online Certification Courses – Learn Today. Lead Tomorrow. Forums Apache Hadoop How many Mapper and Reducer will run if the input directory of the job is empty?

Viewing 2 reply threads
  • Author
    Posts
    • #5880
      DataFlair TeamDataFlair Team
      Spectator

      What will happen if in a Map-Reduce Job if there is no data/file in input directory ?

      How many Mapper and Reducer will run in this case?

    • #5881
      DataFlair TeamDataFlair Team
      Spectator

      If input directory is empty, then no mappers or Reducer will run.
      As a number of mappers depend upon the number of InputSplits, as no data no input splits hence no mappers.
      Without any mapper, a number of the reducer is also 0.

      If we try to run map/reduce job on Hadoop cluster without specifying any input file it will throw following exception:

      java.io.IOException: No input paths specified in job

    • #5883
      DataFlair TeamDataFlair Team
      Spectator

      Since the number of mappers depends upon the Input-Splits which in turn depends on the data, no mappers will be running in this job.

      And since the Reducer receives the input from mapper, hence no reducers will be running as well.
      When trying to run this job with no files under the input directory specified, an error will occur saying :
      java.io.IOException: No input paths specified in job

Viewing 2 reply threads
  • You must be logged in to reply to this topic.