Live instructor-led & Self-paced Online Certification Training Courses (Big Data, Hadoop, Spark) Forums Hadoop Does HDFS allow a client to read a file which is already opened for writing?

This topic contains 1 reply, has 1 voice, and was last updated by  dfbdteam3 12 months ago.

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • #4774

    dfbdteam3
    Moderator

    In HDFS,can we read from the file which is already open for writing?
    What happens, when client try to read from file already opened for writing in HDFS?

    #4776

    dfbdteam3
    Moderator

    Yes, the client can read the file which is already opened for writing.
    But, the problem in reading a file which is currently being written, lies in the consistency of data i.e. Hadoop HDFS does not provide the surety that the data which has been written into the file will be visible to a new reader before the file has been closed.

    For this, one can call the hflush operation explicitly which will push all the data in the buffer into write pipeline and then the hflush operation will wait for acknowledgments from the datanodes. Hence, by doing this, the data that has been written into the file before the hflush operation visible to the reader for sure.

    For more details, please refer: HDFS read-write operations

Viewing 2 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.