feat: Implement backup/restore CLI subcommands

[benbjohnson]

Overview

This pull request adds the influx backup and influx restore commands to the OSS 2.0 CLI. It combines the functionality of the 1.x commands of the same name but updates them to work with organizations & buckets.

Usage

Backup

# Backs up all data to the /mybackup directory.
$ influx backup /mybackup

# Backs up only data for organization "influx".
$ influx backup -o influx /mybackup

Restore

Full restore

A full restore completely deletes all data on the server and replaces it with the data in the backup. This includes all org/bucket meta data as well as other info like dashboards:

$ influx restore --full /mybackup

Partial restore

To restore only the bucket shard data, a partial restore can be done:

# Restores all data from the /mybackup directory to the local InfluxDB server.
$ influx restore /mybackup

# Restores only data for the "mybkt" bucket in the "influx" organization.
$ influx restore -o influx -b mybkt /mybackup

# Restores data from "mybkt" into a new bucket named "myotherbkt"
$ influx restore -o influx -b mybkt --new-bucket myotherbkt /mybackup

---

• CHANGELOG.md updated with a link to the PR (not the Issue)
• Well-formatted commit messages
• Rebased/mergeable
• Tests pass
• http/swagger.yml updated (if modified Go structs or API)
• Feature flagged (if modified API)
• Documentation updated or issue created (provide link to issue/pr)
• Signed CLA (if not already signed)

[benbjohnson]

@stuartcarnie I don't know the permissions we're using too well so I just left this as a TODO. Should this be OwnerPermissions()? Is there a better permission set for backup/restore?

[stuartcarnie]

This is looking really great, Ben! Thanks so much for pulling all this together.

I have experimented with the tool and the backups appear to be working after I added a minor tweak (see one of my comments). I could confirm the manifest, bolt and TSM files were in my backup path.

I was not able to restore the database, as the tool failed with the following errors.

Trying to restore into the existing bucket:

❯ bin/darwin/influx restore -b my-bucket ~/tmp/backup/
2020-10-30T23:33:01.195380Z	info	Resources opened	{"log_id": "0QAyZOLl000", "path": "/Users/stuartcarnie/tmp/backup/20201030T230351Z.bolt"}
2020-10-30T23:33:01.195762Z	info	Restoring organization	{"log_id": "0QAyZOLl000", "id": "93b0c30ea94cb097", "name": "my-org"}
2020-10-30T23:33:01.198169Z	info	Restoring bucket	{"log_id": "0QAyZOLl000", "id": "7425a44ac4116e88", "name": "my-bucket"}
Error: Cannot create bucket: bucket with name my-bucket already exists.
See 'influx restore -h' for help

Tried to restore into a new bucket:

❯ bin/darwin/influx restore -b foo ~/tmp/backup/
2020-10-30T23:33:10.379596Z	info	Resources opened	{"log_id": "0QAyZxDl000", "path": "/Users/stuartcarnie/tmp/backup/20201030T230351Z.bolt"}
2020-10-30T23:33:10.379950Z	info	Restoring organization	{"log_id": "0QAyZxDl000", "id": "93b0c30ea94cb097", "name": "my-org"}
Error: Bucket "foo" not found.
See 'influx restore -h' for help

[benbjohnson]

@stuartcarnie I added the ShardGroup.IsDeleted() check & returned ENotFound from BackupShard().

As for the restore usage, the -b flag is the filter for the bucket to restore and --new-bucket is the flag for the bucket to restore to. I agree the CLI flags are confusing but I tried to base it on the flags in the old 1.x version.

[stuartcarnie]

As for the restore usage, the -b flag is the filter for the bucket to restore and --new-bucket is the flag for the bucket to restore to. I agree the CLI flags are confusing but I tried to base it on the flags in the old 1.x version.

Ahh right – I didn't even catch that. Sorry for the user error 🤦

I'll take another pass over it shortly

[sanderson]

LGTM

All requests were addressed, clearing to unblock merge

[stuartcarnie]

Great work, @benbjohnson!