How to change the replication factor of data which is already stored in HDFS ?

Free Online Certification Courses – Learn Today. Lead Tomorrow. Forums Apache Hadoop How to change the replication factor of data which is already stored in HDFS ?

Viewing 2 reply threads
  • Author
    Posts
    • #5520
      DataFlair TeamDataFlair Team
      Spectator

      If data is present in HDFS and Replication Factor is defined, then how can we change Replication Factor of data already stored in hdfs? The global RF is 4 but for few files, I want to increase the replication factor to 6, since I want to do this for limited files, not all the data in hdfs, no need to change the configuration parameter in hdfs-site.xml. Without moving the data can we increase it dynamically?

    • #5523
      DataFlair TeamDataFlair Team
      Spectator

      Default replication factor in HDFS is 3
      In order to change this we can issue below command :
      where /user/hadoop/sample.txt is path of file in HDFS

      4 : new replication factor

      $ hadoop dfs -setrep -R -w 4 /user/hadoop/sample.txt

    • #5524
      DataFlair TeamDataFlair Team
      Spectator

      The command to change the replication factor of existing file to 2
      $./bin/hadoop dfs -setrep -w <2> <path of file>
      The command to change the replication factor of entire HDFS to 2

      $.hadoop dfs -setrep -R -w 2 /user/hadoop/sample.txt

Viewing 2 reply threads
  • You must be logged in to reply to this topic.