1. Hadoop Commands – Objective
In this HDFS Hadoop commands tutorial, we are going to learn the remaining important and frequently used HDFS commands with the help of which we will be able to perform HDFS file operations like copying a file, changing files permissions, viewing the file contents, changing files ownership, creating directories, etc. To learn more about world’s most reliable storage layer follow this HDFS introductory guide.
Looking to learn HDFS, follow these detailed tutorials:
2. HDFS Hadoop Commands Tutorial
Hadoop HDFS commands are used to perform various Hadoop HDFS operations and in order to manage the files present on HDFS clusters. In this HDFS dfs commands tutorial, we will discuss the hadoop basic commands, hadoop shell commands and frequently use Hadoop commands with examples and description. All the Hadoop file system shell commands are invoked by the bin/hdfs script. If you encounter any query in this HDFA Hadoop tutorial, Please Comment.
Hadoop touchz Command Usage
Hadoop touchz Command Example
hdfs dfs -touchz /user/dataflair/dir2
Hadoop touchz Command Description
This HDFS basic creates a file at the path containing the current time as a timestamp. Fails if a file already exists at a path, unless the file is already size 0.
Hadoop test Command Usage
hdfs dfs -test -[ezd] URI
Hadoop test Command Example
"hdfs dfs -test -e sample hdfs dfs -test -z sample hdfs dfs -test -d sample"
Hadoop test Command Description
The Hadoop fs shell command test is used for file test operations.
It gives 1 output if a path exists; it has zero length, or it is a directory or otherwise 0.
-d: if the path given by the user is a directory, then it gives 0 output.
-e: if the path given by the user exists, then it gives 0 output.
-f: if the path given by the user is a file, then it gives 0 output.
-s: if the path given by the user is not empty, then it gives 0 output.
-z: if the file is zero length, then it gives 0 output.
Hadoop text Command Usage
hdfs dfs -text <source>
Hadooo text Command Example
hdfs dfs -text /user/dataflair/dir1/sample
Hadoop text Command Description
This hdfs fs command takes a source file and outputs the file in text format. The allowed formats are zip and TextRecordInputStream.
Hadoop stat Command Usage
hdfs dfs -stat path
Hadoop stat Command Example
hdfs dfs -stat /user/dataflair/dir1
Hadoop stat Command Description
This HDFS File system command prints information about the path.
%b: If the format is a string which accepts file size in blocks.
%o: Block size
%y, %Y: modification date.
Hadoop tail Command Usage
hdfs dfs -tail [-f] <filename2>
Hadoop tail Command Example
"hdfs dfs -tail /user/dataflair/dir1/sample hdfs dfs -tail -f /user/dataflair/dir1/sample"
Hadoop tail Command Description
This Hadoop File system command shows the last 1KB of the file on console or stdout.
Read: HDFS Commands Part – 4
Hadoop chown Command Usage
hdfs dfs -chown [-R] [OWNER][:[GROUP]] URI [URI ]
Hadoop chown Command Example
hdfs dfs -chown -R dataflair /opt/hadoop/logs
Hadoop chown Command Description
This Hadoop command changes the owner of files. With -R, makes the change according to the order of the directory structure recursively. The user must be the superuser.
Hadoop chmod Command Usage
chmod [-R] mode,mode,... <path>...
Hadoop chmod Command Example
hdfs dfs -chmod 777 /user/dataflair/dir1/sample
Hadoop chmod Command Description
This Hadoop shell command changes the permissions of files. With -R, makes the change recursively by way of the directory structure. The user must be the owner of the file or the superuser.
Hadoop appendToFile Command Usage
hadoop fs -appendToFile <localsource> ... <dst>
Hadoop appendToFile Command Example
hadoop fs -appendToFile /home/dataflair/Desktop/sample /user/dataflair/dir1
Hadoop appendToFile Command Description
This Hadoop fs command append single sources or multiple sources from local file system to the file system at the destination. It also reads input from standard input and adds it to destination file system.
Any doubt yet in the HDFS Hadoop Commands, Please Comment.
Hadoop checksum Command Usage
hadoop fs -checksum URI
Hadoop checksum Command Example
hadoop fs -checksum /user/dataflair/dir1/sample
Hadoop checksum Command Description
This Hadoop basic command Returns the checksum information of a file.
Hadoop count Command Usage
hdfs dfs -count [-q] <paths>
Hadoop count Command Example
hdfs dfs -count /user/dataflair
Hadoop count Command Description
This HDFS DFS command Counts the number of directories, number of files present and bytes under the paths that match the specified file pattern.
This was all on HDFS Hadoop Commands.
3. Related Links
- Top 10 Useful Hdfs Commands Part-IV
- Data Blocks in Hadoop HDFS – Hadoop Distributed File System
- Hadoop HDFS data read and write operations.
- Introduction to HDFS Federation in Hadoop
For any queries or feedback regarding Hadoop Commands just leave a comment in the below section. I hope you like the Hadoop Commands tutorial.