Backup with Borg

Contents

1. Contents
2. Installation
3. Usage
   1. Init a new backup repository with encryption enabled
   2. Encryption modes
   3. Create an actual backup
   4. List created backups
   5. Diff borg snapshots
   6. Query/get information about Borg repository
   7. Extract a specific backup
   8. Mount a specific backup
      1. Intall additional dependencies
      2. Mount
      3. Unmount with
4. Automation
   1. Backup
   2. Portable Backup Script
   3. Systemd Unit File
   4. Systemd Timer

---

Borg is a software for backup creation under linux, see https://borgbackup.readthedocs.io. It uses deduplication in a similar way like git.

Separate bash scripts for each process as automatisation would make sense.

Installation

sudo pacman -S borg

See also: https://borgbackup.readthedocs.io/en/stable/installation.html.

Usage

Init a new backup repository with encryption enabled

mkdir the directory (where the repo data should reside) first.

borg init --encryption=<mode> /path/to/backup/repo


# For remote repositories (via ssh)
borg init --encryption=<mode> user@hostname:/path/to/repo           # See: https://borgbackup.readthedocs.io/en/stable/quickstart.html#remote-repositories

Initialises a backup repository with a repository key - it will be asked for a key - and a path.

Encryption modes

Hash func	Auth-only args	Encryption & auth mode (passphrase only)	Encryption & auth mode (passphrase + keyfile)
SHA-256	authenticated	repokey	keyfile
BLAKE2b	authenticated-blake2	repokey-blake2	keyfile-blake2

• To automate backups with passphrase set the BORG_PASSPHRASE env variable
• Or use --encryption none for not encryption
• If no encryption is wanted but still to detect malicious tampering
  • Use --encryption authenticated
  • To work with authenticated repos you will need the passphrase, but there is an emergency workaround: BORG_WORKAROUNDS=authenticated_no_key
• Keyfiles will be stored in .config/borg/keys
• Speed
  • Generally BLAKE2b is faster than SHA-256 (Intel/AMD 64-bit CPUs)
  • authenticated-blake2 is faster than none & authenticated
  • --encryption can be none (no passphrase, no keyfile) -> however SHA-256 is used as chunk ID hash -> not the fastest option
  • See also here in the benchmarking section of the borg docs

If you use keyfiles export them via: borg key export /path/to/borg/repo /export/location/borg-repo.keyfile (you must state a file not a folder).

Create an actual backup

We create a backup of /home/$user but exclude some directories:

borg create /path/to/repo::'{hostname}-{now}' \
    /home/$user \
    --exclude '/home/*/.cache/*'    \
    --exclude "/home/$user/.xsession-errors" \
    --exclude-caches  \
    --compression lz4 \
    --progress \
    --one-file-system

To get a feeling about compression/decompression speed and ratio see here: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO.

List created backups

List all created backups inside the repository via:

borg list /path/to/borg/repo

Diff borg snapshots

Prints a diff of two snapshots. You might want a list before to check the available names.

borg diff /path/to/borg/repo::snapshot_name1 snapshot_name2

Note: if there are equal there is no output.

For checking if your command backups the correct things (like considering excludes correctly) this is a very useful command.

Query/get information about Borg repository

$ borg info /path/to/repo/
Enter passphrase for key /home/<usr>/.config/borg/keys/test_2.2: 
Repository ID: e6cf32e10e7a81e4839e676670d0a5a7326a33edb425b91ead4d3c4fe01038a8
Location: /path/to/repo/
Encrypted: Yes (key file BLAKE2b)
Key file: /home/<usr>/.config/borg/keys/test_2.2
Cache: /home/<usr>/.cache/borg/e6cf32e10e7a81e4839e676670d0a5a7326a33edb425b91ead4d3c4fe01038a8
Security dir: /home/<usr>/.config/borg/security/e6cf32e10e7a81e4839e676670d0a5a7326a33edb425b91ead4d3c4fe01038a8
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:              130.40 MB             65.63 MB              3.46 MB

                       Unique chunks         Total chunks
Chunk index:                     155                 2664

Extract a specific backup

In order to extract a specific backup (point in time) to the current folder you can use the following command:

borg extract /path/to/borg/repo::backupNAME --progress

Notes:

• The whole path will be created. E.g. when your backuped folder is /mnt/test_1/ it will also create the folders mnt and test_1.
• In case you are wondering that you do not need any keyfiles have a look in ~/.config/borg/keys/. During borg init the keyfiles were created at this location.

Mount a specific backup

Install additional dependencies

Borg allows you to directly mount a specific backup to be mounted. In order to work you have to install another dependency: pip3 install llfuse.

Mount

After the installation we are ready to go.

Get the list of available backups:

borg list /path/to/borg/repo

Do the actual mount:

borg mount /path/to/borg/backup/repo::2019-08-05-00:16:15-usr /mnt/temp/
#                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
#                                                ||              ^^^^^^^^^
#                               Snapshot name (from `borg list`)     ||
#                                                                   Will be mounted here

If you get a /mnt/temp/: Mountpoint must be a writable directory error you probably don't have the correct permissions (-> chmod will help).

Unmount with

borg umount /mnt/temp/

Automation

For automation this handy scripts might be useful:

Backup

Copy this file to your prefered location like. For example:

cp borg-automation.sh /opt/borg-backup/borg-automation.sh
sudo chmod 770 /opt/borg-backup/borg-automation.sh

#!/bin/bash
##################################
# Borg backup automation script
##################################
# Note, this script does NOT:
# * mount any disk/partitions
# * initially create your Borg repository

##################################
# User Configuration
PATH_TO_BACKUP="/mnt/test_1"                                 # This path will be backed-up
BORG_REPO="/mnt/test_2"                                      # Path to Borg repository
BACKUP_REPO_DISK_UUID="1CD0AF0C0975E9BA"                     # UUID of disk containing the Borg repository
BORG_PASSPHRASE="test"

BORG_COMPRESSION="lz4"                  # Compression method used for backup
#BORG_COMPRESSION="lzma"
# See also: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO


##################################
# Predefined Configuration
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no
export BORG_PASSPHRASE="${BORG_PASSPHRASE}"
#export BORG_REPOSITORY="${BORG_REPO}"
TIMESTAMP="$(date +"%Y-%m-%d-%H:%M:%S")-$(hostname)"

BORG_PRUNE() {
borg prune \
-v \
--stats \
--list \
--keep-yearly=-1 \
--keep-monthly=25 \
--keep-weekly=53 \
--keep-daily=90 \
${BORG_REPO}
}

BORG_CREATE() {
borg create \
--stats \
--progress \
--one-file-system \
--compression ${BORG_COMPRESSION} \
--exclude-caches \
--exclude '/mnt/some/path/TEMP' \                 # edit excludes
--exclude '/dev' \                                # The following are excludes for a complete system backup
--exclude '/home/*/.cache' \
--exclude '/root' \
--exclude '/home/<user>/share' \                  # Add username!!
--exclude '/home/*/.config/borg/security' \
--exclude '/lost+found' \
--exclude '/tmp' \
--exclude '/mnt' \
--exclude '/proc' \
--exclude '/run' \
--exclude '/snap' \
--exclude '/sys' \
--exclude '/var/cache' \
--exclude '/var/lib/container' \
--exclude '/var/lib/lxcfs' \
--exclude '/var/lib/ooni' \
--exclude '/var/lib/snapd' \
--exclude '/var/lib/ubuntu-advantage' \
--exclude '/var/run' \
--exclude '/var/lock' \
--exclude '/var/spool' \
--exclude '/var/log' \
--exclude '/var/tmp' \
--exclude '/var/snap' \
--exclude '/etc/pacman.d/' \
--exclude '/etc/.pwd.lock' \
--exclude '/usr/bin' \
--exclude '/usr/include' \
--exclude '/usr/lib' \
--exclude '/usr/lib32' \
--exclude '/usr/lib64' \
--exclude '/usr/share' \
--exclude '/usr/src' \
--exclude '/usr/sbin' \
${BORG_REPO}::${TIMESTAMP} \
${PATH_TO_BACKUP}
}

# Reganding excludes for system backup see: https://unix.stackexchange.com/questions/1067/what-directories-do-i-need-to-back-up

####################################################################
SEPARATOR="-------------------------------"
SEPARATOR_LONG="---------------------------------------------------------------------------------------------"

HEADING="#################################
# BORG BACKUP AUTOMATION SCRIPT #
#################################"

echo "${HEADING}"

# Check if correct drive is mounted and avoid bloating drives by accident
foundUUID="$(lsblk -o uuid --noheading | grep ${BACKUP_REPO_DISK_UUID})"
lsblkOutLong="$(lsblk -o name,mountpoint,label,fsused,fssize,size,fstype,uuid)"
foundUUID_LONG="$(grep ${BACKUP_REPO_DISK_UUID} <<< ${lsblkOutLong})"

echo ""
if [[ -z "${foundUUID}" ]]; then
    echo "No backup disk found; exiting ..."
    printf "Mounted disks were:\n\n${lsblkOutLong}\n"
    exit 0
else
    echo "${SEPARATOR_LONG}"
    printf "Found backup disk:\n\n"
    head -n 1 <<< ${lsblkOutLong}
    echo "${foundUUID_LONG}"
    echo "${SEPARATOR_LONG}"
fi
echo ""

# Check if directory to backup is not empty (then it is possibly not mounted)
if [ -n "$(ls -A ${PATH_TO_BACKUP} 2>/dev/null)" ]
then
    echo "All clear; directory to backup is not empty."
else
    echo "Directory to backup is empty. Exiting..."
    exit 0
fi

# Print version for logging
echo "Borg version: $(borg --version)"

echo "Starting pruning. This may take some moments ..."
printf "\tExecution string: \`$(declare -f BORG_PRUNE)\`\n\n"
BORG_PRUNE 
printf "Done with pruning.\n\n"

echo "Starting backup for \`${TIMESTAMP}\`. This may take some moments ..."
printf "\tExecution string: \`$(declare -f BORG_CREATE)\`\n\n"
BORG_CREATE
echo "Done with backup \`${TIMESTAMP}\`."

# Reset password env var
export BORG_PASSPHRASE=""

# Just be sure that something was still cached but not written to disk
sync

Portable backup script

#!/bin/bash
##################################
# Borg backup automation script
##################################
# Note, this script does NOT:
# * mount any disk/partitions
# * initially create your Borg repository


## INIT
# Command used
# borg init --encryption=none /mnt/h/my_backup

echo "Make sure the backup is mounted as /mnt/h and the disk to backup is mounted as /mnt/x"
read -r -p "Are you sure? [y/N] " response
if ! [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]; then
    echo "Exiting..."
    exit
fi

##################################
# User Configuration
PATH_TO_BACKUP="/mnt/x"                                 # This path will be backed-up
BORG_REPO="/mnt/h/my_backup"                            # Path to Borg repository

BORG_COMPRESSION="lz4"                                  # Compression method used for backup
#BORG_COMPRESSION="lzma"
# See also: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO


##################################
# Predefined Configuration
export BORG_RELOCATED_REPO_ACCESS_IS_OK=no
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no
TIMESTAMP="$(date +"%Y-%m-%d-%H:%M:%S")-$(hostname)"

BORG_PRUNE() {
borg prune \
-v \
--stats \
--list \
--keep-yearly=-1 \
--keep-monthly=25 \
--keep-weekly=53 \
--keep-daily=90 \
${BORG_REPO}
}

BORG_CREATE() {
borg create \
--stats \
--progress \
--one-file-system \
--compression ${BORG_COMPRESSION} \
--exclude-caches \
--exclude '/mnt/x/paths/to/exclude/' \
${BORG_REPO}::${TIMESTAMP} \
${PATH_TO_BACKUP}
}


####################################################################
SEPARATOR="-------------------------------"
SEPARATOR_LONG="---------------------------------------------------------------------------------------------"

HEADING="#################################
# BORG BACKUP AUTOMATION SCRIPT #
#################################"

echo "${HEADING}"

# Check if directory to backup is not empty (then it is possibly not mounted)
if [ -n "$(ls -A ${PATH_TO_BACKUP} 2>/dev/null)" ]
then
    echo "All clear; directory to backup is not empty."
else
    echo "Directory to backup is empty. Exiting..."
    exit 0
fi

# Print version for logging
echo "Borg version: $(borg --version)"

echo "Starting pruning. This may take some moments ..."
printf "\tExecution string: \`$(declare -f BORG_PRUNE)\`\n\n"
BORG_PRUNE 
printf "Done with pruning.\n\n"

echo "Starting backup for \`${TIMESTAMP}\`. This may take some moments ..."
printf "\tExecution string: \`$(declare -f BORG_CREATE)\`\n\n"
BORG_CREATE
echo "Done with backup \`${TIMESTAMP}\`."

# Just be sure that something was still cached but not written to disk
sync

Systemd Unit File

#/etc/systemd/system/borg-backup.service

[Unit]
Description=Borg Backup
 
[Service]
# To avoid permission errors with the backup use the current user/group
User=yourUser
Group=users

Type=simple

# We do not want to interrupt other important tasks so we set the scheduling parameters accordingly
Nice=19
IOSchedulingClass=2
IOSchedulingPriority=7

# In case something went wrong during the last backup try to unlock the repository
ExecStartPre=/usr/bin/borg break-lock /mnt/test_2
ExecStart=/opt/borg-backup/borg-automation.sh

Systemd Timer

It suffice that .timer and .service components have the same name.

#/etc/systemd/system/borg-backup.timer

[Unit]
Description=Borg Backup Timer

[Timer]
WakeSystem=false
# Start every day at 04h
OnCalendar=*-*-* 04:00:00

[Install]
WantedBy=timers.target

Regarding OnCalendar syntax see here.

Then refresh and enable:

systemctl enable borg-backup.timer
systemctl start borg-backup.timer
systemctl daemon-reload
# Check timer
systemctl list-timers --all
# Check service execution (when timer was triggered at `OnCalendar`-time)
systemctl status borg-backup.service