-
Notifications
You must be signed in to change notification settings - Fork 76
Setting up Offsite Backups
BOA already has a backup system that creates copies of files in
/data/disk/o1/backups/
But this is on the same disk as the main site. It will not maintain backups if the primary disk fails. As a solution to this, I have created a new persistent drive and mounted it to that folder. At least then, if there is a failure on the primary drive, the backups are safe.
But this does not protect against a hacked server where everything is deleted. Or if the server and all drives are lost. So I set up a synchronisation to copy the backups to Amazon S3.
The CRON file looks like this
#!/bin/bash
echo Started ``date`` >> /tmp/sync-backups.log
find /data/disk/o1/backups/ -type f -maxdepth 1 -exec bash "/root/tools/move-to-s3.sh" "{}" \;
echo Finished ``date`` >> /tmp/sync-backups.log
It finds all files in the backup folder and runs another script. Then appends a date to a file in /tmp so I can see when it ran.
The guts of the copy utility (move-to-s3.sh) are as follows
#!/bin/bash
# Copy Backup file to S3 Bucket
#
# Filename is in the format
#
# domain.com.au-20140520.133810.tar.gz
#
# And save to the format
#
# domain.com.au/2014/05/20/domain.com.au-20140520.133810.tar.gz
#
# Keeping the same file name (so it can be identified if downloaded later)
# but into a date based folder structure based on the domain name
FILENAME=`basename ""`
echo $FILENAME
# Target - where to save it
TARGET=`echo $FILENAME | sed -r -e 's/(.*)\-([0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])\.(.*)/\1\/\2\/\3\/\4/'`
echo "--> $TARGET"
s3cmd sync s3://your-bucket-name/$TARGET/$FILENAME
Essentially I break the name up so that all the backups are saved in a manageable folder structure on S3. I have a rule in the S3 bucket to archive the files to glacier after a short period of time and delete them completely after many months.
You can obviously run the script directly against a backup file or part of the CRON.