AWK Command Examples in Linux

by

awk command examples in linux

AWK is a very useful scripting language for text processing. This language is executed by the awk interpreter. It allows the user to process some input, define variables, use logical operators, string and numeric functions, extract data and generate formatted reports. AWK’s syntax is very familiar to C language and it is a direct predecessor of Perl. All AWK scripts can be converted into Perl scripts using a2p utility. In this tutorial, we will show you several practical examples of awk command examples in Linux.

Prerequisites

The AWK interpreter is a standard tool found on every Linux distribution. The gawk package contains the open source version of AWK, and depending on the Linux distribution, it can be installed from a source file or using the gawk or mawk packages included with the specific Linux distribution.

How to install AWK in Linux

Login to your  server via SSH as user root

ssh root@IP_Address

In order to install the AWK command line utility on CentOS/Fedora or any other RPM-based Linux distribution, run the following command:

yum install gawk

On Ubuntu/Debian, you need to invoke this command to install gawk:

apt-get install gawk

AWK Command Examples in Linux

Simple awk commands can be easily run from the command line, and for more complex tasks should be written as awk scripts to a file. Listed below are some useful examples of awk commands and executable scripts.

  • You can use the AWK command to print only certain columns from the input field. For example, using the command given below you can find out the list IP addresses which are connected to your server:
netstat -anp|grep tcp|awk '{print $5}'| cut -d : -f1 | sort | uniq -c | sort -n

This is quite useful if you are investigating if your server is under a DoS or DDoS attack.

  • In the following example, we use AWK to search for a particular pattern in certain columns and do some action based on the result:
exim -bpr | grep frozen | awk {'print $3'} | xargs exim -Mrm

The command above will delete all frozen emails from the Exim mail queue.

  • AWK is often used to perform useful and practical text processing and manipulation. For example, we can use awk to remove duplicates in a text file without sorting:
awk '!x[$0]++' file-with-duplicates > new-file-without-duplicates
  • The following command will print five random numbers from 0 to 999:
awk 'BEGIN { for (i = 1; i <= 5; i++) print int(1000 * rand()) }'
  • Use the following command to count the number of lines in a file named ‘sample_file’:
awk 'END { print NR }' sample_file
  • The following command will print all lines in a file named sample-file which contain strings starting with either ‘A’ or ‘a’, followed by ‘re’:
awk '/[Aa]re/{print}' /opt/sample_file
  • You can use the AWK command for more complex operations. If your website looks somewhat slow, you may use the following command to check if there is some problem with the disk I/O (and/or network in some rare cases):
tac /proc/stat | awk '/^btime/ {up=systime()-$2;print "up " up/86400 "d"}; /^cpu / {print "user " $2/up "%, nice " $3/up "%, sys " $4/up "%, idle " $5/up "%, iowait " $6/up "%, steal " $9/up "%\niowait/used " $6 / ($2+$3+$4) ", steal/used " $9 / ($2+$3+$4) }'

IOWAIT means how long processes are blocked by busy I/O, mostly disk storage or perhaps network. STEAL means how long processes are blocked by luck of CPU timeslice on the server. Higher iowait perused CPU time (=USER + NICE + SYSTEM) shows busy I/O, higher STEAL perused shows busy CPU.

  • The following script uses a simple awk command that searches the input file ‘/etc/passwd’ and provides an output with the username followed by the date and time of the last login:
vi login-check
#!/bin/bash

for user in `awk -F: '{print $1}' /etc/passwd`
do
echo -n "$user: "
finger $user | grep Last
if [ $? != 0 ]; then
echo
fi
done

Make the script executable:

chmod 755 login-check

Run the script:

./login-check

You should be able to see the user accounts available on your server, followed by the date and time of the last login of each user.

Conclusion

There are some newer languages such as Perl and Python which can be used instead of AWK, but using AWK commands in Linux, has some advantages because:

  • AWK is very easy to learn.
  • AWK can be used to solve certain types of problems faster and write more efficient scripts than using other tools/languages.
  • AWK comes very handily when working with large files like logs etc. because using AWK command/scripts you can produce a filtered and readable report.

awk command exampleOf course, you don’t have to do any of this, if you use one of our Optimized VPS Hosting Solutions, in which case you can simply ask our expert Linux admins to help you with this. They are available 24×7 and will take care of your request immediately.

PS. If you liked this post on AWK command examples in Linux, please share it with your friends on the social networks using the buttons on the left or simply leave a reply below. Thanks.

Leave a Comment