Live instructor-led & Self-paced Online Certification Training Courses (Big Data, Hadoop, Spark) Forums Hadoop In MapReduce how to change the name of output file from part-r-00000

This topic contains 1 reply, has 1 voice, and was last updated by  dfbdteam3 1 year, 6 months ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #4684

    dfbdteam3
    Moderator

    I want to change the name of output file of MapReduce Job, rather than part-*, I want to give a custom name. Actually I want to map the output file name with the hive table name.

    #4685

    dfbdteam3
    Moderator

    By following 2 ways we can change the name of output file from part-r-00000:

    1. Using a Java class that derives from MultipleOutputFormat as the jobs output format allows control of the output file names.

    // job.setOutputFormatClass(TextOutputFormat.class);
    LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class);
    MultipleOutputs.addNamedOutput(job,“text”, TextOutputFormat.class,
    Text.class, IntWritable.class);

    2. Using (this I have tested and working)
    job.getConfiguration().set(“mapreduce.output.basename”, “text”);
    part name will change and file will be created as text-r-00000

    For more details, please follow: MapReduce Tutorial

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.