Build resilience in Raspberry Pi-based compute infrastructure for makers. Start with setting up Pi to operate off an SSD instead of the conventional SD card. The build resilience by containerizing apps with Docker, automated backups of application data to Cloud and disk imaging.
This note covers automated backups of application data to Cloud with rclone and rsync.
Step | Instruction |
---|---|
YAML file for IOT stack | See Andreas Spiess and Graham Garner here. The application data is mapped to a volumes folder on host machine. |
Dropbox mount | Use rclone as shown here. |
Dropbox mount-point service | Use systemd as explained here and here and shown in Exhibit A. |
Backup command | Use rsync as explained here and shown in Exhibit B. |
Backup shell script | Use bash script for backup job as explained here and shown in Exhibit C. |
Backup automation on task scheduler | Use crontab as explained here and shown in Exhibit D. |
# /etc/systemd/system/rclone.service
[Unit]
Description=Dropbox (rclone)
AssertPathIsDirectory=/mnt/dropbox
[Service]
Type=notify
ExecStart=/usr/bin/rclone mount \
--config /home/pi/.config/rclone/rclone.conf \
--allow-other \
--vfs-cache-mode writes \
--vfs-cache-max-size 100M \
dropbox: /mnt/dropbox
ExecStop=/bin/fusermount -u /mnt/dropbox
- Create in Nano using arrow-keys to navigate, Ctrl-K to cut lines, Ctrl-U to paste, Ctrl-O to save and Ctrl-X to quit.
- The important specifications are:
--config
to point to therclone.conf
file and the mapping fromdropbox:
to the local/mnt/dropbox
. - Place the file in the location: /etc/systemd/system. Use sudo cp to write to this destination.
- Start with the following commands:
systemctl start rclone
systemctl enable rclone
- Use
ls -l
to verify that the Dropbox folders show up at the mount location. Note that the files may not show up in the File Explorer (GUI). - In case of any errors, proceed as follows:
- Grab the error logs with the following command:
systemctl status rclone.service > err.txt
. - Fix the file in the home directory using Nano, then
sudo cp
to/etc/systemd/system
as before. - Refresh:
systemctl daemon-reload
.
- Grab the error logs with the following command:
To backup, use the command as follows:
sudo rsync -avz --delete /home/pi/IOTstack/volumes /mnt/dropbox
To test, execute dry-run as follows:
sudo rsync -avz --delete --dry-run /home/pi/IOTstack/volumes /mnt/dropbox
Exhibit C: Write bash (shell) script 'backup2dropbox.sh' in /home/pi
for backup with rsync. Use Nano following guide here.
NOW="$(date)"
echo "Started backup at $NOW"
# sudo rsync -avz --delete --dry-run /home/pi/IOTstack/volumes /mnt/dropbox
sudo rsync -avz --delete /home/pi/IOTstack/volumes /mnt/dropbox
NOW="$(date)"
echo "Ended backup at $NOW"
- The flags are as follows:
-a
: save metadata (e.g. ownership) while taking back-up.-v
: verbose-z
: compress
- Use
--delete
to clean up files at the destination that have been removed at the source. - Make the shell script executable with chmod as follows:
chmod 755 /home/pi/backup2dropbox.sh
- Then test it as follows:
sh /home/pi/backup2dropbox.sh
sudo crontab -e
Select Nano as the editor from options presented by the interactive shell. Then add the following line to run the shell script at 4:30 am on the 1st and 15th day of every month and every Friday.
30 4 1,15 * 5 /home/pi/backup2dropbox.sh >> /home/pi/logs/cron.log 2>&1
Make sure to have a logs folder /home/pi/logs
. The output will be appended to the log file instead of being sent to standard output. Errors will be sent to standard output. To backup daily, modify as follows:
30 4 * * * /home/pi/backup2dropbox.sh >> /home/pi/logs/cron.log 2>&1