BashGuide

A more complete and extensive guide is Advanced Bash-Scripting Guide Many examples are based hereof.

Bash Google Coding Guidelines

Command separators

Operator &

executes the command in the background in a subshell

  # For example:
  for n in {1..30}; do
  wget "https://i11www.iti.kit.edu/_media/teaching/winter2012/algo2/vorlesung${n}.pdf" &
  #                                                             do in subshell   -----^^^
  done

does not wait for the command to finish, and the return status is 0

Operator ;
- Commands separated by a ; are executed sequentially
- The shell waits for each command to terminate in turn
- The return status is the exit status of the last command executed

Special chararcters / compound commands

End of options (`--`)

A double-dash defines the end of options:

ls -lh -- -badName/

Comment ( `#` )

# introduceds a line comment

`$var1` vs. `${var1}` (dollar sign + curly braces)

Variable substitution (contents of a variable) (e.g. var1=5; echo $var1 # outputs: 5)

Use parameter substitution (${var1}) instead (same effect but resolves some disambiguities e.g. when concatenating variables with strings).

Non-/No-op command (`:`)

: is a null command. It can be interpreted as a synonym for the shell builtin true.

Exit status is true (/0).

Last exit code (`$?`)

$? returns the exit code of the last command.

`$$` / PID

echo "pid is $$" prints the process ID.

Control structures

while

inputFile="./diff.txt"

while read file; do
    cp ${file}  "./diffedFiles"
done < ${inputFile}

Note that we also use our variable to redirect the contents to the loop (works also to store loop outputs in a variable).

for

Example using an array:

# Declare an array variable
declare -a arr=("element1" "element2" "element3")

for i in "${arr[@]}"; do
    echo "$i"
    # Or do whatever with individual element of the array
done
# You can access them using echo "${arr[0]}", "${arr[1]}" also

Examples

Using ranges with leading zeros:

for i in {000..999}; do
    echo ${i}
done

`if` conditions (`if` and `fi`)

if [[ "$(ls -A ${pathTemp})" ]]     # True if length of string is non-zero
    then                            # Or in one line: `if [[ "$(ls -A ${pathTemp})" ]]; then`
        echo "Path is not empty"
        # ...
    else
        echo "path is EMPTY"
        # ...
elif [[ <cond> ]]
    :   # Elif branch (optional)
else
    :   # Else branch (optional)
fi

Extended test command - `[[ expression ]]`

Return a status of 0 or 1 depending on the evaluation of the conditional expression expression. The spaces are mandatory.

([ ]: test command; Same as Extended test command / [[ ]]. Prefer [[ ]] over [ ] (see also http://mywiki.wooledge.org/BashFAQ/031). This is more safe but does not work in POSIX-only environments.)

Example

Check if folder /tmp is empty:

[[ "$(ls -A /tmp)" ]] && echo "Not Empty" || echo "Empty"

RegEx-if condition

read -p "Press \`Y\` or \`y\` to continue: " -n 1 -r
echo                           # Print newline after command was entered
if [[ "${REPLY}" =~ ^[Yy]$ ]]
    then
         : # Continue
    else
        exit 1
fi

Strings etc.

Here document

HEADER= cat <<'EOF'
Some longe multiline string
       with indentations.
EOF

# or for commands:
VAR=$(cat <<EOF
blah
EOF
)

EOF is just an arbitrary delimiter for the start/end of the here document
Quote (like above) your start/end sequence (e.g. EOF -> 'EOF') to avoid escaping
-EOF to lstrip any leading whitespaces

Functions

function noParams {
    echo "blah"
    exit
}

function withParams {
    echo $1 
}

# Function call
noParams

# Function call **with** one param
withParams 'Hello'
withParams 'World'

Bash does not have return statements. Processing values of a function call can be achieved by echoing the "return"-value.

res=$(noParams)
# res = "blah"

Traps / traptest

Traps react on signals (show std signals with kill -l; EXIT = 0).

Set trap: trap 'cmdOrFctName' <SIGNAL>
Signals can be ignored: trap '' SIGINT
Reset trap: trap SIGINT (no longer execute traps for SIGINT)

Examples

Catch `Ctrl-C`

Prints and increments counter until Ctrl-C is pressed (kill with: kill -s SIGKILL <PID>).

counter="0"
trap "echo \$((counter++))" SIGINT SIGTERM
echo "PID is $$"

while :
do
	sleep 3
done

Catch non-zero exit codes and handle in function

#!/bin/bash
set -e

trap 'trapFctName $? $LINENO' EXIT
trapFctName() {
    if [ "$1" != "0" ]; then
      # error handling goes here
      printf "Error %s occurred in line %s\n" $1 $2
    fi
}

`

--------------------------------------------------------------------------------------


# Redirecting
* `>` : output - write (to file (overwrites existing ones)) **redirects only stdout**
* `>>` : append (to file)
* `<` : input - read (from file)

## Example - appending to a file
Appends all `.txt` files in current directory:
    
```bash
cat *.txt >> ~/anotherDirectoryToAvoidInfiniteAppending/all.txt

Example - store std::out to variable but keep outputting to std::out

PIP_STATUS="$(pip3 install --upgrade pip | tee /dev/tty)"

Stderr, Stdout, Stdin

command   identifier1>identifier2
            ^^^      ^    ^^^
redirect    id1     to    id2

std level	identifier
Stdin	`0`
Stdout	`1`
Stderr	`2`

Redirecting stderr to stdout: &>word, >word 2>&1, and >&word are exactly the same.

Examples

# Pipe to "null device" (this discards everything it gets)
ls -R * 2> /dev/null

# or redirect output of stdout to a file (no space after the number!!)
find ./ -name "newfile*" 1> log.txt

# stderrr to stdout
programm 2>&1

# stderr and stdout to file
programm &> Datei.txt

Printing (to Stdout, ...)

Use printf in favour of echo (here is why)

# Prints var as a string
printf '%s' "$var"

Exit codes

Some exit codes have special meanings. See also /usr/include/sysexits.h. $? is used to find the return value of the last executed command.

Exit Code Number	Meaning	Example	Comments
`0`	Success	---	---
`1`	Catchall for general errors	let "var1 = 1/0"	Miscellaneous errors, such as "divide by zero" and other impermissible operations
`2`	Misuse of shell builtins (according to Bash documentation)	empty_function() { }	Missing keyword or command, or permission problem (and diff return code on a failed binary file comparison).
`126`	Command invoked cannot execute	/dev/null	Permission problem or command is not an executable
`127`	"command not found"	illegal_command	Possible problem with `$PATH` or a typo
`128`	Invalid argument to exit	exit 3.14159	exit takes only integer args in the range 0 - 255 (see first footnote)
`128+n`	Fatal error signal "n"	kill -9 `$PPID` of script	`$?` returns 137 (128 + 9)
`130`	Script terminated by Control-C	Ctl-C	Control-C is fatal error signal 2, (130 = 128 + 2, see above)
`255*`	Exit status out of range	exit -1	exit takes only integer args in the range 0 - 255

Piping

Piping to another process:

command1 | command2

This will send the output of command1 as the input of command2.

Usually, the last item in a pipeline determines the result (return value).

Argument passing

Arguments

Sends the output of one process as command line arguments to another process

command2 `command1`

For example:

cat `find . -name '*.foo' -print`

This will send the output of command1 into command2 as command line arguments. Note that file names containing spaces (newlines, etc.) will be broken into separate arguments, though.

mSearch() {
	grep -nC "${2}" -i "$1" 'blub.txt'
	#           ^--------^--------- second and first argument (actually it begins with zero $0 = script name)
}

Optional arguments

Syntax:

"${<argumentNumb>-${varName}}"
"${<argumentNumb>-<default-value>}"

Example:

mSearch() {
	numPadLines=7
	# "${2-${numPadLines}} ---> Use numPadLines if 2nd argument is unset
	tac "notes.md" | grep -nC "${2-${numPadLines}}" -i "$1" --color=auto --group-separator=***********************************************************
}

Pattern matching

//TODO: For now see: https://www.gnu.org/software/bash/manual/html_node/Pattern-Matching.html#Pattern-Matching

Variables / Parentheses

Synthax	Explanation
`{ ... }`	Brace expansion
`$var` or `${var}`	Variable/parameter expansion
`( ... )`	Command group
`(( ... ))`	C-style manipulation + arithmetic expansion and evaluation

Readonly variables

Define constants as read-only

readonly TIMEOUT_S=10

Curly Braces

Braces are reserved words, so they must be separated from the list by blanks or other shell metacharacters.

Expanding arrays, as in ${array[42]}
Parameter expansion operations, as in ${filename%.*} (remove extension)
Expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Ranges:

$ echo {0..20}
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Special usage in find -exec
- {} substitutes the matches found
- -exec takes a program and arguments and runs (-> it does not execute a full bash)
- To overcome this: $ find $felix -name "PQIS*.zip" -exec bash -c 'echo $(dirname {})' \;

Examples

$var      # use the variable
${var}    # same as above
${var}bar # expand var, and append "bar" too
$varbar   # same as ${varbar}, i.e expand a variable called varbar, if it exists.

OUTPUT="$(ls -1)"
echo "${OUTPUT}"    # Quoting (") does matter to preserve multi-line values

`( )` / Single Parentheses

When commands are grouped, redirections may be applied to the entire command list.

Command group ( e.g. (a=hello; echo $a))
Array init (e.g. Array=(element1 element2 element3))

Examples

1. Colon ( `:` ) in `if fi`

Since if requires a non-empty then clause and comments don't count as non-empty, [: serves as a no-op](#:\ /\ Colon\ /\ non-op\ command).

if command; then :; else ...; fi

2. `{ }` vs. `( )`

count_tmp() { cd /tmp; files=(*); echo "${#files[@]}"; }
pwd; count_tmp; pwd         # pwd : "print working directory"

Output

/home/mrspaceinvader
11
/tmp

Function body executes in a subshell

cd ; unset files
count_tmp() (cd /tmp; files=(*); echo "${#files[@]}")
pwd; count_tmp; pwd

Output

/home/mrspaceinvader
11
/home/mrspaceinvader

Double Parentheses `(( ))`

Permits arithmetic expansion and evaluation (e.g. a=$(( 5 + 3 )); would set a to 8). Allows C-style manipulation of variables in Bash, for example, (( var++ )) or (( var = 42 )) or ((var+=42)).

Strings

Everything is a string (no typing in bash)

Length of a string

$ blah="blub"
$ echo ${#blah}
4

Slicing

Starts the string at position 1 (bash starts counting at 0) until the end (legth of string)

$ echo ${blah:1:${#blah}}
lah

Or get rid of the last character

$ echo ${blah::-1}
bla

However,

$ echo ${blah:1:}

prints nothing.

Quoting

Better quote everything (e.g. rm "$2" << might contain spaces; without quotes you'll get a list of files to delete)

Double Quotes

Variable references are instantly expanded.

If you want selective expansion inside a string - i.e., expand some variable references, but not others - do use double quotes, but prefix the $-references which you do not want to expand with \; e.g., \$var).

Single Quotes

Strings inside single quotes are not expanded or interpreted in any way by the shell.

Comparison Operators

See [if fi chapter](#if\ /\ Tests\ /\ Expression\ evaluation\ (if\ and\ fi)) for usage.

Integers

-eq : is equal to (f.ex.: if [ "$a" -eq "$b" ])
-ne : is not equal to
-gt : is greater than
-ge : is greater than or equal to
-lt : is less than
-le : is less than or equal to
< : is less than (within double parentheses) (f.e.x.: (("$a" < "$b")))
<= : is less than or equal to (within double parentheses)
> : is greater than (within double parentheses)
>= : is greater than or equal to (within double parentheses)

Strings

= / == : is equal to (f.ex.: if [ "$a" = "$b" ]; Note the whitespace framing of = )

Note: The == comparison operator behaves differently within a double-brackets test than within single brackets:

[[ $a == z* ]]   # True if $a starts with an "z" (pattern matching).
[[ $a == "z*" ]] # True if $a is equal to z* (literal matching).

[ $a == z* ]     # File globbing and word splitting take place.
[ "$a" == "z*" ] # True if $a is equal to z* (literal matching).

!= : is not equal to
< / > : is less/greater than, in ASCII alphabetical order (< / > needs to be escaped within a [ ] construct)
-z : string is null, that is, has zero length
-n : string is not null

Famous :(){ :|:& };:

Long form

:()
{
	:|:&
};
:

Explanation

:() #Define new function
	#named ':'
{ #Begin function definition
#block
:|:& #Pipe the very ':' function through itself,
		#creating two processes, and make the
		#resulting copy run in the background
		#(the & part)
} #End function definition block
;: #Call ':' for the first time, initiating a chain
#reaction: each instance of ':' will create two
#more instances, ad infinitum

Debugging and useful commands

Exit on first failure

set -e

Print/echo each command before execution

set -o xtrace

-> Also prints evaluation of

Additional Ressources

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License *.

Code (snippets) are licensed under a MIT License *.

* Unless stated otherwise

Home

Python 3

^(un)fold

Snippets

General

Libs

Linux/bash

^(un)fold

Guides

Scripts

Git

^(un)fold

C/C++

^(un)fold

Video

^(un)fold

Databases

^(un)fold

PostgreSQL

Misc

^(un)fold

Windows

^(un)fold

Mac

^(un)fold

General

SW recommendations

^(un)fold

(Angular) Dart

^(un)fold

BashGuide

Command separators

Special chararcters / compound commands

End of options (--)

Comment ( # )

$var1 vs. ${var1} (dollar sign + curly braces)

Non-/No-op command (:)

Last exit code ($?)

$$ / PID

Control structures

while

for

Examples

Using ranges with leading zeros:

if conditions (if and fi)

Extended test command - [[ expression ]]

Example

RegEx-if condition

Strings etc.

Here document

Functions

Traps / traptest

Examples

Catch Ctrl-C

Catch non-zero exit codes and handle in function

Example - store std::out to variable but keep outputting to std::out

Stderr, Stdout, Stdin

Examples

Printing (to Stdout, ...)

Exit codes

Piping

Argument passing

Arguments

Optional arguments

Pattern matching

Variables / Parentheses

Readonly variables

Curly Braces

Examples

( ) / Single Parentheses

Examples

1. Colon ( : ) in if fi

2. { } vs. ( )

Double Parentheses (( ))

Strings

Length of a string

Slicing

Quoting

Double Quotes

Single Quotes

Comparison Operators

Integers

Strings

Famous :(){ :|:& };:

Long form

Explanation

Debugging and useful commands

Exit on first failure

Print/echo each command before execution

Additional Ressources

Home

Python 3

Linux/bash

Git

C/C++

Video

Databases

Misc

Windows

Mac

SW recommendations

(Angular) Dart

Becoming a Software Eng

Clone this wiki locally

End of options (`--`)

Comment ( `#` )

`$var1` vs. `${var1}` (dollar sign + curly braces)

Non-/No-op command (`:`)

Last exit code (`$?`)

`$$` / PID

`if` conditions (`if` and `fi`)

Extended test command - `[[ expression ]]`

Catch `Ctrl-C`

`( )` / Single Parentheses

1. Colon ( `:` ) in `if fi`

2. `{ }` vs. `( )`

Double Parentheses `(( ))`