How to Use awk

awk is one of the most used commands in bash environment. I can’t possibly cover everything about awk but I will write about an example that can be applied to many situations.

When I list files and directories in a directory, I execute…

ls -lah

That shows outputs the list like below.

total 296
drwxr-xr-x+ 50 hiriumi  staff   1.6K Aug 27 21:44 .
drwxr-xr-x   5 root     admin   160B Dec  5  2019 ..
-r--------   1 hiriumi  staff     7B Aug 18 22:06 .CFUserTextEncoding
-rw-r--r--@  1 hiriumi  staff    20K Aug 26 20:47 .DS_Store
drwx------  86 hiriumi  staff   2.7K Aug 26 16:57 .Trash
-rw-------   1 hiriumi  staff    36K Aug 29 17:44 .bash_history
-rw-r--r--   1 hiriumi  staff   788B Jul 19 17:31 .bash_profile
-rw-r--r--   1 hiriumi  staff   272B Mar 10 23:35 .bash_profile.backup
drwx------   3 hiriumi  staff    96B Apr 14 22:46 .cache
drwx------   5 hiriumi  staff   160B May 21 19:54 .config
drwx------   3 hiriumi  staff    96B Mar  7 13:40 .cups
drwxr-xr-x  14 hiriumi  staff   448B Mar  6 22:17 .dropbox
drwxr-xr-x  16 hiriumi  staff   512B Aug  1 14:25 .dvdcss
-rw-r--r--   1 hiriumi  staff    56B Mar  9 00:01 .gitconfig
drwxr-xr-x  15 hiriumi  staff   480B May 10 09:09 .iterm2
-rwxr-xr-x   1 hiriumi  staff    22K May 10 09:10 .iterm2_shell_integration.bash
drwxr-xr-x   6 hiriumi  staff   192B Mar  9 00:26 .kivy
<SNIP>

As you can see, if it’s a file, the line starts with “-” and if it’s a directory, it starts with “d”. Let’s just list the files using grep. I will show an example in awk after that.

ls -lah | grep '^-'

The following command outputs exactly the same list.

ls -lah | awk '/^-/ {print}'
-r--------   1 hiriumi  staff     7B Aug 18 22:06 .CFUserTextEncoding
-rw-r--r--@  1 hiriumi  staff    20K Aug 26 20:47 .DS_Store
-rw-------   1 hiriumi  staff    36K Aug 29 17:44 .bash_history
-rw-r--r--   1 hiriumi  staff   788B Jul 19 17:31 .bash_profile
-rw-r--r--   1 hiriumi  staff   272B Mar 10 23:35 .bash_profile.backup
-rw-r--r--   1 hiriumi  staff    56B Mar  9 00:01 .gitconfig
-rwxr-xr-x   1 hiriumi  staff    22K May 10 09:10 .iterm2_shell_integration.bash
-rw-------   1 hiriumi  staff    62B Jul 28 13:44 .lesshst
-rw-------   1 hiriumi  staff    29K Aug 27 21:44 .viminfo
-rw-r--r--   1 hiriumi  staff    10B Jun 29 21:40 .vimrc
-rw-r--r--   1 hiriumi  staff    32B May 22 20:29 .vuerc
-rw-------   1 hiriumi  staff    35B Mar  6 23:26 .zsh_history

What if you want to just output the file names? Here is what you can do. The command means print the 9th column separated by space.

ls -lah | awk '/^-/ {print $9}'
.CFUserTextEncoding
.DS_Store
.bash_history
.bash_profile
.bash_profile.backup
.gitconfig
.iterm2_shell_integration.bash
.lesshst
.viminfo
.vimrc
.vuerc
.zsh_history

The following command outputs the 1st and the 9th columns.

ls -lah | awk '/^-/ {print $1,$9}'
-r-------- .CFUserTextEncoding
-rw-r--r--@ .DS_Store
-rw------- .bash_history
-rw-r--r-- .bash_profile
-rw-r--r-- .bash_profile.backup
-rw-r--r-- .gitconfig
-rwxr-xr-x .iterm2_shell_integration.bash
-rw------- .lesshst
-rw------- .viminfo
-rw-r--r-- .vimrc
-rw-r--r-- .vuerc
-rw------- .zsh_history

What if you want a custom output for your documentation? I just want to list the file names with vertical bars. This can be used as a table in JIRA.

ls -lah | awk '/^-/ {print "|"$9"|"}'

Here is the sample output.

|.CFUserTextEncoding|
|.DS_Store|
|.bash_history|
|.bash_profile|
|.bash_profile.backup|
|.gitconfig|
|.iterm2_shell_integration.bash|
|.lesshst|
|.viminfo|
|.vimrc|
|.vuerc|
|.zsh_history|

Recap

awk is an old but very useful command on Linux/Mac. There is much more to this command so I will write in this blog as I come across more usages.

Author: admin

A software engineer in greater Seattle area

Leave a Reply

Your email address will not be published. Required fields are marked *