Hadoop Commands with Examples and Usage Part-III 2


1. Hadoop Commands – Objective

In this HDFS Hadoop commands tutorial, we are going to learn the remaining important and frequently used HDFS commands with the help of which we will be able to perform HDFS file operations like copying a file, changing files permissions, viewing the file contents, changing files ownership, creating directories, etc. To learn more about world’s most reliable storage layer follow this HDFS introductory guide.

HDFS Hadoop commands tutorial

Hadoop Commands Tutorial

Looking to learn HDFS, follow these detailed tutorials:

HDFS High Availability

HDFS Fault Tolerance

2. HDFS Hadoop Commands Tutorial

Hadoop HDFS commands are used to perform various Hadoop HDFS operations and in order to manage the files present on HDFS clusters. In this HDFS dfs commands tutorial, we will discuss the hadoop basic commands, hadoop shell commands and frequently use Hadoop commands with examples and description. All the Hadoop file system shell commands are invoked by the bin/hdfs script. If you encounter any query in this HDFA Hadoop tutorial, Please Comment.

2.1. touchz

Hadoop touchz Command Usage

touchz <path>

Hadoop touchz Command Example

hdfs dfs -touchz /user/dataflair/dir2

Hadoop touchz Command Description
This HDFS basic creates a file at the path containing the current time as a timestamp. Fails if a file already exists at a path, unless the file is already size 0.

Read: Read write Operations in HDFS

2.2. test

Hadoop test Command Usage

hdfs dfs -test -[ezd] URI

Hadoop test Command Example

 "hdfs dfs -test -e sample
hdfs dfs -test -z sample
hdfs dfs -test -d sample" 

Hadoop test Command Description
The Hadoop fs shell command test is used for file test operations.
It gives 1 output if a path exists; it has zero length, or it is a directory or otherwise 0.
Options:
-d: if the path given by the user is a directory, then it gives 0 output.
-e: if the path given by the user exists, then it gives 0 output.
-f: if the path given by the user is a file, then it gives 0 output.
-s: if the path given by the user is not empty, then it gives 0 output.
-z: if the file is zero length, then it gives 0 output.

2.3. text

Hadoop text Command Usage

hdfs dfs -text <source>

Hadooo text Command Example

hdfs dfs -text /user/dataflair/dir1/sample

Hadoop text Command Description
This hdfs fs command takes a source file and outputs the file in text format. The allowed formats are zip and TextRecordInputStream.

2.4. stat

Hadoop stat Command Usage

hdfs dfs -stat path

Hadoop stat Command Example

hdfs dfs -stat /user/dataflair/dir1

Hadoop stat Command Description
This HDFS File system command prints information about the path.

%b: If the format is a string which accepts file size in blocks.

%n: Filename

%o: Block size

%r: replication

%y, %Y: modification date.

Read: HDFS Commands Part – 1

2.5. tail

Hadoop tail Command Usage

hdfs dfs -tail [-f] <filename2>

Hadoop tail Command Example

 "hdfs dfs -tail /user/dataflair/dir1/sample
hdfs dfs -tail -f /user/dataflair/dir1/sample" 

Hadoop tail Command Description
This Hadoop File system command shows the last 1KB of the file on console or stdout.

Read: HDFS Commands Part – 4

2.6. chown

Hadoop chown Command Usage

hdfs dfs -chown [-R] [OWNER][:[GROUP]] URI [URI ]

Hadoop chown Command Example

hdfs dfs -chown -R dataflair /opt/hadoop/logs

Hadoop chown Command Description
This Hadoop command changes the owner of files. With -R, makes the change according to the order of the directory structure recursively. The user must be the superuser.

Read: HDFS Commands Part – 2

2.7. chmod

Hadoop chmod Command Usage

chmod [-R] mode,mode,... <path>...

Hadoop chmod Command Example

hdfs dfs -chmod 777 /user/dataflair/dir1/sample

Hadoop chmod Command Description
This Hadoop shell command changes the permissions of files. With -R, makes the change recursively by way of the directory structure. The user must be the owner of the file or the superuser.

2.8. appendToFile

Hadoop appendToFile Command Usage

hadoop fs -appendToFile <localsource> ... <dst>

Hadoop appendToFile Command Example

hadoop fs -appendToFile /home/dataflair/Desktop/sample /user/dataflair/dir1

Hadoop appendToFile Command Description
This Hadoop fs command append single sources or multiple sources from local file system to the file system at the destination. It also reads input from standard input and adds it to destination file system.

Any doubt yet in the HDFS Hadoop Commands, Please Comment.

2.9. checksum

Hadoop checksum Command Usage

hadoop fs -checksum URI

Hadoop checksum Command Example

hadoop fs -checksum /user/dataflair/dir1/sample

Hadoop checksum Command Description
This Hadoop basic command Returns the checksum information of a file.

Read: HDFS Commands Part – 4

2.10. count

Hadoop count Command Usage

hdfs dfs -count [-q] <paths>

Hadoop count Command Example

hdfs dfs -count /user/dataflair

Hadoop count Command Description
This HDFS DFS command Counts the number of directories, number of files present and bytes under the paths that match the specified file pattern.

This was all on HDFS Hadoop Commands.

3. Related Links

For any queries or feedback regarding Hadoop Commands just leave a comment in the below section. I hope you like the Hadoop Commands tutorial.


2 thoughts on “Hadoop Commands with Examples and Usage Part-III

  • Evelyn Serrell

    I came to your Play with HDFS using Commands Part-III – DataFlair Blogs page by searching on Google and I really like your blog. Nice explanation.

Comments are closed.