linux_awk.md

• AWK
  • Syntax
  • Example
  • Specify field separator
  • Use Regular Expression
  • BEGIN / END

AWK

Syntax

Think of awk as just a simple program that iterates over your file one line at a time.

For every line in the file it checks to see whether that line has a certain pattern, if that line match a pattern, some action is performed.

NOTE: awk won't stop when if match a pattern, it will keep matching the rest patterns.

# PSEUDO
awk 'if(PATTERN1){ACTION1} if(PATTERN2){ACTION2} ...'

Since this way of doing things is so common, the if statement is never written, and the brackets (), even the PATTERN, can be omitted.

# syntax
awk 'PATTERN1{ACTION1} PATTERN2{ACTION2} ...'

Example

• print the first 2 lines
  • NR stands for Row Number

  $ ls -l | awk 'NR==1{print $0} NR==2{print}'
  total 0
  drwx------@  3 root  staff    96 Dec  4  2021 Applications

• print full information of the first 3 lines, and only last field for the rest lines
  • NR stands for Number of Fields

  ls -l | awk 'NR<=3{print} NR>3{print $NF}'
  total 0
  drwx------@  3 root  staff    96 Dec  4  2021 Applications
  drwx------@ 14 root  staff   448 Jan 31 18:43 Desktop
  Documents
  Downloads
  Library
  Movies
  Music

• print the last 2nd filed

  awk '....{print $(NF-1)}'

Specify field separator

• awk treats space as the column delimiter
• use -F to specify a delimiter

  $awk -F ':' '{...}'

• The delimiter can be a regular expression.

  $awk -F '[.:]' '{...}'

Use Regular Expression

• print folder names which ends with s

  ls -l | awk '/s$/{print $NF}'
  Applications
  Documents
  Downloads
  Movies
  Pictures

• you can specify which field RE to be match

  ls -l | awk '$0 ~ /s$/{print $NF}'

BEGIN / END

• BEGIN / END are special pattern in awk. It doesn't match any specific line, but instead, cause the action to execute when awk starts up / shut down.

ls -l | awk '
  BEGIN{temp_sum=0; total_records=0; print "Begin calculating average file size"}
  NR>1{temp_sum += $5; total_records +=1;}
  END{print "Average file size:" temp_sum/total_records }
'

Begin calculating average file size
Average file size:506.273