Map task

[DataFlair Team]

The map task copy is sent to all the blocks,that means same task copy is sent to all replicated blocks as well,so does it process all the replicated blocks also…this will consume more resources..how is it managed?

[DataFlair Team]

The Map task is sent to all the replicated blocks, but it does not process the Map tasks simultaneously across all the replicated blocks.

For example, if a block is replicated across three other blocks (named B1, B2, B3), then initially, B1 would start processing the Map task. Say supposing, if a node fails in a cluster which contains block B1, only then will the Map task start executing on the replicated block B2 and if the node containing block B2 fails, then the Map task will start running on block B3.
While the Map task is being run on any of the replicated blocks, the data team will work on bringing back the failed node alive so that the block B1 starts working and the replication factor of three is maintained across the clusters.

To conclude, the HDFS data blocks are created across different nodes across the cluster and so the resource consumption need not be taken into consideration as there will be enough resources to facilitate the Map tasks on each node.

[DataFlair Team]

The map task is sent to all the blocks but it doesn’t process all blocks suppose if there are three duplicate(same) blocks it processes only one block.

While the Map taskis being run on any of the replicated blocks, the data team will work on bringing back the failed node alive so that the block B1 starts working and the replication factor of three is maintained across the clusters.

The HDFS data blocks are created across different nodes across the cluster and so the resource consumption need not be taken into consideration as there will be enough resources to facilitate the Map tasks on each node.
Learn More about : MapTask

[Author]

Posts