Hadoop HDFS Commands with Examples and Usage – Part II

Hadoop Quiz

1. HDFS Commands – Objective

In this Hadoop HDFS Commands tutorial, we are going to learn the remaining important and frequently used Hadoop commands with the help of which we will be able to perform HDFS file operations like copying a file, changing files permissions, viewing the file contents, changing files ownership, creating directories, etc. To learn more about world’s most reliable storage layer follow this HDFS introductory guide.

Hadoop HDFS Commands with Examples and Usage

Hadoop HDFS Commands with Examples and Usage – Part II

2. Hadoop HDFS Commands

Hadoop file system shell commands are used to perform various Hadoop HDFS operations and in order to manage the files present on HDFS clusters. In this frequently used Hadoop DFS commands are discussed with their expample and usage. All the Hadoop basic commands are invoked by the bin/hdfs script. If face any doubt in Hadoop HDFS Commands, Please Ask us in comments.

2.1 moveFromLocal

HDFS moveFromLocal Command Usage
moveFromLocal <localSrc> <dest>

HDFS moveFromLocal Command Example
hdfs dfs -moveFromLocal /home/dataflair/Desktop/sample /user/dataflair/dir1

HDFS moveFromLocal Command Description
This HDFS command copies the file or directory from the local file system identified by the local source to destination within HDFS, and then deletes the local copy on success.

Learn more about Internals of HDFS Data Write Pipeline and File write execution flow

2.2. moveToLocal

HDFS moveToLocal Command Usage
moveToLocal <src> <localDest>

HDFS moveToLocal Command Example
hdfs dfs -moveToLocal /user/dataflair/dir2/sample /user/dataflair/Desktop

HDFS moveToLocal Command Description
This hadoop basic command works like -get, but deletes the HDFS copy on success.

Read: Top 10 Hadoop Hdfs Commands Part-I – An HDFS Tutorial

2.3. tail

HDFS tail Command Usage
hdfs dfs -tail [-f] <filename>

HDFS tail Command Example

 "hdfs dfs -tail /user/dataflair/dir2/purchases.txt
hdfs dfs -tail -f /user/dataflair/dir2/purchases.txt" 

HDFS tail Commnad Description
Shows the last 1KB of the file on console or stdout.

2.4. rm

HDFS rm Command Usage
rm <path>

HDFS rm Command Example
hdfs dfs -rm /user/dataflair/dir2/sample

HDFS rm Command Description
This Hadoop command removes the file or empty directory present on the path provided by the user.

HDFS rm Command Example
hdfs dfs -rm -r /user/dataflair/dir2

HDFS rm Command Description
Recursive version of delete.

2.5. expunge

HDFS expunge Command Usage
hdfs dfs -expunge

HDFS expunge Command Example
hdfs dfs -expunge

HDFS expunge Command Description
This Hadoop shell commmand is used to empty the trash.

Read: Hadoop Commands with Examples and Usage Part-III

2.6. chown

HDFS chown Command Usage
hdfs dfs -chown [-R] [OWNER][:[GROUP]] URI [URI ]

HDFS chown Command Example
hdfs dfs -chown -R dataflair /opt/hadoop/logs

HDFS chown Command Description
This Hadoop HDFS Commmand changes the owner of files. With -R, changes are made recursively by way of the structure of the directory. A user should be the superuser.

2.7. chgrp

HDFS chgrp Command Usage
hdfs dfs -chgrp [-R] <NewGroupName> <file or directory name>

HDFS chgrp Command Example
hdfs dfs -chgrp [-R] New Group sample

HDFS chgrp Description
Hadoop chgrp shell command is used to change the files group association. Also, you can try -R option to make changes recursively by the way of structure of the directory.

Read: Interact with HDFS using CLI & Perform Various Operations Part-IV

2.8. setrep

HDFS setrep Command Usage
setrep [-R] [-w] rep <path>

HDFS setrep Command Example
hdfs dfs -setrep -w 3 /user/dataflair/dir1

HDFS setrep Command Description
This HDFS command is used to change the replication factor of a file. If the entered path is a directory, then this command changes the replication factor of all the files present in the directory tree rooted at path provided by user recursively.

Options:
The -w flag requests that the command waits for the replication process to get completed. This may likely take a very long time to get completed.

The -R flag is accepted for backward compatibility. It does not make any changes.
Any Doubt yet in Hadoop HDFS Commands? Please Comment.

2.9. du

HDFS du Command Usage
du <path>

HDFS du Command Example
hdfs dfs -du /user/dataflair/dir1/sample

HDFS du Command Description
This HDFS dfs command shows disk usage, in bytes, for all the files present on the path provided by the user; reporting of filenames are done with the full HDFS protocol prefix.

HDFS du Command Example
hdfs dfs -du -s /user/dataflair/dir1/sample

HDFS du Description
This Hadoop basic command is like -du, but it prints a summary of the amount of disk usage of all files/directories in the path.

2.10. df

HDFS df Command Usage
hdfs dfs -df [-h] URI [URI ...]

HDFS dfs Command Example
hdfs dfs -df -h

HDFS dfs Command Description
This Hadoop fs displays free space.
This was all on Hadoop HDFS Commands.

Related Links

If you find these Hadoop HDFS commands helpful or if you have any queries let us know in the comments and our support team will get back to you. Hope You like the Hadoop HDFS Commands Tutorial.

8 Responses

  1. Brinu says:

    Thank you for sharing HDFS commands. This really helped me a lot.

  2. Jenni says:

    Really appreciate your blog that has shared many details. I like your other blogs also and regularly visit your page. Keep it up!!

    • Data Flair says:

      Thank you, Jenni, for complimenting on Hadoop HDFS Commands tutorial. Each Hadoop HDFS tutorial from Data Flair is an effort for every reader to provide them with latest and researched information.
      It is nice to know that you regularly visit Data Flair. Still, if you want more articles and information, you can follow us our FB page also.

  3. Madhavi says:

    This blog is the source of knowledge to enhance the career in Bigdata for the lifetime.

  4. DEVASHISH KUMAR CHOUDHARY says:

    thank you

Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.