What is slot in Hadoop v1 Map-Reduce ?

[DataFlair Team]

What are the slots?
how many types or slots are there?
What are the issues with the slot based mechanism ?

why it was removed from Hadoop 2 ?

[DataFlair Team]

Each Map Reduce Jobs are split into task and Task tracker runs each task on a fixed number of map and reduce slots inside a data node based on a static configuration.

In Hadoop 1.0 we need to specify in mapred-site.xml the following parameter to Configure the number of map slots and reduce slots

mapreduce.tasktracker.map.tasks.maximum
mapreduce.tasktracker.reduce.tasks.maximum

The Following are the problem with Slot

• Slots are wasted, anytime the cluster workload does not fit the static configuration.
• In a slot-based model makes it hard for non-MapReduce applications to be scheduled appropriately.
• Jobs never leave their original slots and cannot be move to free slots.

For more details about MapReduce Jobs, please refer: MapReduce Tutorial

[Author]

Posts