Does HDFS allow a client to read a file which is already opened for writing?

[DataFlair Team]

In HDFS,can we read from the file which is already open for writing?
What happens, when client try to read from file already opened for writing in HDFS?

[DataFlair Team]

Yes, the client can read the file which is already opened for writing.
But, the problem in reading a file which is currently being written, lies in the consistency of data i.e. Hadoop HDFS does not provide the surety that the data which has been written into the file will be visible to a new reader before the file has been closed.

For this, one can call the hflush operation explicitly which will push all the data in the buffer into write pipeline and then the hflush operation will wait for acknowledgments from the datanodes. Hence, by doing this, the data that has been written into the file before the hflush operation visible to the reader for sure.

For more details, please refer: HDFS read-write operations

[Author]

Posts