{"id":110396,"date":"2022-09-02T09:00:46","date_gmt":"2022-09-02T03:30:46","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=110396"},"modified":"2022-09-02T09:41:39","modified_gmt":"2022-09-02T04:11:39","slug":"awk-command-in-linux","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/awk-command-in-linux\/","title":{"rendered":"AWK Command in Linux"},"content":{"rendered":"

In this article, you will learn all there is to the AWK command in Linux-based operating systems. We will go through what the awk command is, why it is used, how it works, syntax, options, and the built-in variables of the awk command.<\/p>\n

We will also look into the technical side of the awk command as we go through awk patterns, awk statements, awk variables, awk actions, and finally some practical commands of the awk command in the terminal. In this article, we will look into some of the coolest features of the awk command, so pay attention and read right to the end!<\/p>\n

What is AWK?<\/h3>\n

Awk is not a command-line-based utility, instead, it is a scripting language used for manipulating and generating reports. The awk language, unlike c programing, needs no compiling. AWK is the abbreviation of the names of the developers – Aho, Weinberger, and Kernighan.<\/p>\n

Awk is designed for advanced text processing and is mostly used as a tool for reporting and analysis. Since awk is data-driven, you can define a set of actions to be performed against the input text. Awk takes in and transforms the input data, and sends it to stdout (standard output).<\/p>\n

The awk scripting language is used very commonly by programmers, as they write scaled-down programs in the form of a statement that defines text designs and patterns.<\/p>\n

How does awk work?<\/h3>\n

The primary purpose of the awk command is to make text manipulation and information retrieval an easy job in Linux distributions. The command works by scanning a set of input lines and then searches for the lines that match the pattern specified by the user.<\/p>\n

The awk command accepts input data, which gets transformed and sent out to the standard output. For each pattern that awk recognizes, the user can describe an action to perform on each line. Awk can easily process complex log files and output a readable output.<\/p>\n

Why use awk?<\/h3>\n

As we have seen awk is used for processing and manipulating text. It enables a programmer to write tine program in the form of a statement where he or she can specify text patterns that are searched for in each line of a file.<\/p>\n

Awk also has the ability to search more than one file to see if they contain the lines that match the specified pattern and perform the defined actions.<\/p>\n

Let us look at what we can do with the awk command:<\/p>\n

1. Operations capable of AWK<\/h4>\n

a. Scan files line by line.<\/p>\n

b. Divides each input line into fields.<\/p>\n

c. awk compares the fields of the input lines to patterns<\/p>\n

d. On successful matching, awk can perform specified actions.<\/p>\n

2. What is awk useful for?<\/h4>\n

a. Transform data files<\/p>\n

b. Produce formatted reports<\/p>\n

c. processing text<\/p>\n

d. Manipulating text<\/p>\n

3. Programming constructs<\/h4>\n

a. Producing formatted output lines<\/p>\n

b. Performing arithmetic and string operations<\/p>\n

c. loops and conditionals<\/p>\n

Syntax of the awk command<\/h3>\n

When you first look at the syntax of the awk command, it may look a little intimidating, but once you understand what each field is for, and see a few examples, you will get the hang of it. The syntax for th awk command is:<\/p>\n

awk <options> <\u2019selection criteria <action>\u2019 input filename> output file<\/pre>\n
Let us look at each field in the syntax briefly:<\/p>\n
1. Options<\/h4>\n
This field takes in the options available with the awk command, which we will look into in the next section.<\/p>\n
2. Selection criteria<\/h4>\n
This field specifies the pattern of string we are searching for.<\/p>\n
3. Action<\/h4>\n
This field specifies the action that has to be carried out if it found the match<\/p>\n
4. Input filename<\/h4>\n
This field specifies the name of the file you want the search of the string to happen in.<\/p>\n
5. Input filename<\/h4>\n
This field specifies the name of the files you want the formatted or manipulated print in.<\/p>\n
Options used with the awk command<\/h3>\n
We have seen that in the syntax of the awk command, there is a field for entering options. Options specify how awk should work or how the output should be formatted. Let us look into the option available with the awk command.<\/p>\n
1. -f <program file><\/h4>\n
This option reads the awk program source from the file specified instead of the first command-line argument.<\/p>\n
2. -F <fs><\/h4>\n
This option uses fs for the input field separator.<\/p>\n
Fields and records<\/h3>\n
In earlier sections, we were speaking about awk dividing input lines into fields. Let us look at what files and records are in slightly greater depth. We saw that awk processes text files and streams, and divides this input data into records and files.<\/p>\n
Awk operators only one record at a time. These records are separated by a character called separators and the default separator is a new line. This means that each line in the text file is a record.<\/p>\n
These records consist of fields. Just like records, fields are also separated by a separator, except in this case the default separator is a space. Fields in each record are referred using a dollar sign ($) Followed by the filed number starting with 1. In layman’s language, a record is a sentence and a field is a word.<\/p>\n
Statements of the AWK command<\/h3>\n
Just like every programming language, awk also has the standard loops like if-else, while, for, and many more, in this section we will be looking at each of them in detail.<\/p>\n
If -else statement<\/h4>\n
The if-else statement works just like you expect it, if a statement is true it will perform a specific set of instructions and if it is false, it will perform another set of instructions.<\/p>\n
Let us write a simple program in one line: to check if the second field and 3rd field of each record are the same, if they are we will print what they are and if not we will print a message saying they are not equal.<\/p>\n
To write the program as expected as above, use the command shown below:<\/p>\n
awk -F ‘,’ ‘{if($2==$3){print $1″,”$2″,”$3} else {print “Not equal”}}’ if .txt<\/p>\n
To execute this program, we obviously need a sample text file, for the time being, let us consider the following text file:<\/p>\n
\"sample<\/a><\/p>\n
And upon running the command above, we get the expected output.<\/p>\n
\"linux<\/a><\/p>\n
While loop<\/h4>\n
The while loop also works just like anticipated, it performs a specific task until the specified condition is false. Let us write a simple while loop to break down the field in every record, and also precede it with a number. To do so, use the command shown below:<\/p>\n
awk '{i=0; while(i<=NF) { print i \":\"$i; i++;}}' if.txt<\/pre>\n
For this statement also let us take the text file we used in the previous example.<\/p>\n
\"sample<\/a><\/p>\n
Upon running the while loop, we get the expected output:<\/p>\n
\"while<\/a><\/p>\n
For loop<\/h4>\n
The for loop is exactly the same as the while loop, it performs a specific set of actions only as long as the condition is true. Let us write a simple program for printing the squares of numbers till 10 using for loop. To do so, use the command shown below:<\/p>\n
awk 'BEGIN{for(i=1; i<=10; i++) print \"The square of\", i, \"is\", i*i;}'<\/pre>\n
This specific example does not need any text file to work on as we are directly printing in on the terminal and have no input data. Upon running the command, we get the following output:<\/p>\n
\"for<\/a><\/p>\n
Break statement<\/h4>\n
The break statement is used to exit out of a loop with a specific condition met. For example, let us write a code to print the word \u201cubuntu\u201d only 5 times (using the break statement), which means the 6th time the command goes through the loop, it should break away from it. To do the following, use the command shown below:<\/p>\n
awk 'BEGIN{x=1; while(1) {print \"Example\"; if ( x==6 ) break; x++; }}'<\/pre>\n
Even this example does not need any text file to work on as we are directly printing in on the terminal and have no input data. Upon running the command, we get the following output:<\/p>\n
\"break<\/a><\/p>\n
Patterns of AWK<\/h3>\n
You can insert a pattern in front of the awk command so that it acts as a selector. The selector determines whether to perform the specified action or not. Let us look at the different types of patterns<\/p>\n
1. Regular expressions<\/p>\n
2. Arithmetic and relational expressions<\/p>\n
3. String valued expressions<\/p>\n
4. Arbitrary boolean combinations of the above expressions.<\/p>\n
Let us look at each expression in detail.<\/p>\n
Regular expressions using linux AWK Command<\/h4>\n
These expressions are the simplest form of expressions containing a string of characters enclosed in slashes (\/). For example, let us write a command to print out the letters starting with the letter capital \u2018A\u2019 in a specific text file. To do so type the command shown below in the terminal:<\/p>\n
awk '$1 ~ \/^A\/ {print $0}' expression.txt<\/pre>\n
Upon running the command, we get the expected output: All the content starting with the letter \u201cA\u201d gets printed out.<\/p>\n
\"regular<\/a><\/p>\n
Relational expressions using AWK Linux<\/h4>\n
As the name suggests relational expressions relate 2 quantities, they include:<\/p>\n