What is different between the InputSplit and HDFS Block in Hadoop?

[DataFlair Team]

What is the difference between hdfs block and input split? What is given as input to Mapper Block or Split? If Split then why?
What is the fundamental difference between a MapReduce InputSplit and HDFS block

[DataFlair Team]

Block–
Default size of the HDFS block is 128 MB which we can configure as per our constraint. All blocks of the file are of the same size except the last block, which can be of same size or smaller. The files are split into 128 MB blocks and then stored into HDFS.
InputSplit– By default, split size is approximately equal to block size(128 MB). InputSplit is user defined and the user can control split size based on the size of data in MapReduce program.

Block- It is the physical representation of data. It contains a minimum amount of data that can be read or write.

InputSplit
 It is the logical representation of data present in the block. It is used during data processing in MapReduce program or other processing techniques.
 InputSplit doesn’t contain actual data, but a reference to the data.

Learn more about InputSplit vs Block in Hadoop.

[DataFlair Team]

Input Split
An inputsplit is a logical division of records. User can control the size of the inputsplit for the MapReduce Program. Each inputsplit is assigned to individual mappers for processing. The number of inputsplit is equal to the number map tasks for the MapReduce program. Inputsplit is defined by the InputFormat class.

HDFS Block
HDFS Block is a physical division of data. The default block size in HDFS is 128MB. HDFS stores these blocks among several nodes. Each block of the data will be of 128MB except the last one, depending on the total size of the data and block size.

[Author]

Posts