Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental backup and restore: vtctldclient usage #1546

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ vtctldclient EmergencyReparentShard <keyspace/shard>
-i, --ignore-replicas strings Comma-separated, repeated list of replica tablet aliases to ignore during the emergency reparent.
--new-primary string Alias of a tablet that should be the new primary. If not specified, the vtctld will select the best candidate to promote.
--prevent-cross-cell-promotion Only promotes a new primary from the same cell as the previous primary.
--wait-for-all-tablets Should ERS wait for all the tablets to respond. Useful when all the tablets are reachable.
--wait-replicas-timeout duration Time to wait for replicas to catch up in reparenting. (default 15s)
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ vtctldclient UpdateThrottlerConfig [--enable|--disable] [--threshold=<float64>]
--throttle-app string an app name to throttle
--throttle-app-duration duration duration after which throttled app rule expires (app specififed in --throttled-app) (default 1h0m0s)
--throttle-app-ratio float ratio to throttle app (app specififed in --throttled-app) (default 1)
--unthrottle-app string expire any throttling rule for the given app
--unthrottle-app string an app name to unthrottle
```

### Options inherited from parent commands
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Vitess supports restoring to a _timestamp_ or to a specific _position_. Either w
Starting with `v18`, it is possible to restore to a given timestamp. The restore process will apply all events up to, and excluding, the given timestamp, at 1 second granularity. That is, the restore will bring the database to a point in time which is _about_ 1 second before the specified timestamp. Example:

```shell
$ vtctlclient -- RestoreFromBackup --restore_to_timestamp "2023-06-15T09:49:50Z" zone1-0000000100
vtctldclient RestoreFromBackup --restore-to-timestamp "2023-06-15T09:49:50Z" zone1-0000000100
```

The timestamp must be in `RFC3339` format.
Expand All @@ -73,13 +73,27 @@ The timestamp must be in `RFC3339` format.
It is possible to restore onto a precise GTID position. Vitess will restore up to, and including, the exact requested position. This gives you the utmost granularity into the state of the restored database.

```shell
vtctlclient -- RestoreFromBackup --restore_to_pos <position> <tablet-alias>
vtctldclient RestoreFromBackup --restore-to-pos <position> <tablet-alias>
```

Example:

```shell
vtctlclient -- RestoreFromBackup --restore_to_pos "MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-60" zone1-0000000102
vtctldclient RestoreFromBackup --restore-to-pos "MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-60" zone1-0000000102
```

#### Dry run

It is possible to verify whether a restore-to-timestamp or restore-to-pos is possible without actually performing the restore. Run:


```shell
vtctldclient RestoreFromBackup --dry-run --restore-to-timestamp "2023-06-15T09:49:50Z" zone1-0000000100
```

or
```shell
vtctldclient RestoreFromBackup --dry-run --restore-to-pos "MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-60" zone1-0000000102
```

A dry run restore looks at existing backups and sees whether there is a path that restores up to given timestamp or pos, but then quits and does not interrupt any tablet's execution and without changing the tablet's type. If there's no valid path to restore, the process exits with error.
Original file line number Diff line number Diff line change
Expand Up @@ -101,19 +101,19 @@ vtctldclient --server=<vtctld_host>:<vtctld_port> BackupShard [--allow_primary=f

## Create an incremental backup with vtctl

An incremental backup requires additional information: the point from which to start the backup. An incremental backup is taken by supplying `--incremental_from_pos` to the `Backup` command. The argument may either indicate a valid position, or the value `auto`. Examples:
An incremental backup requires additional information: the point from which to start the backup. An incremental backup is taken by supplying `--incremental-from-pos` to the `Backup` command. The argument may either indicate a valid position, or the value `auto`. Examples:

```sh
vtctlclient -- Backup --incremental_from_pos="MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-53" zone1-0000000102
vtctldclient Backup --incremental-from-pos="MySQL56/0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-53" zone1-0000000102

vtctlclient -- Backup --incremental_from_pos="0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-53" zone1-0000000102
vtctldclient Backup --incremental-from-pos="0d7aaca6-1666-11ee-aeaf-0a43f95f28a3:1-53" zone1-0000000102

vtctlclient -- Backup --incremental_from_pos="auto" zone1-0000000102
vtctldclient Backup --incremental-from-pos="auto" zone1-0000000102
```

When indicating a position, you may choose to use or to omit the `MySQL56/` prefix (which you can find in the backup manifest's Position).

When `--incremental_from_pos="auto"`, Vitess chooses the position of the last successful backup as the starting point for the incremental backup. This is a convenient way to ensure a sequence of contiguous incremental backups.
When `--incremental-from-pos="auto"`, Vitess chooses the position of the last successful backup as the starting point for the incremental backup. This is a convenient way to ensure a sequence of contiguous incremental backups.

An incremental backup backs up one or more MySQL binary log files. These binary log files may begin with the requested position, or with an earlier position. They will necessarily include the requested position. When the incremental backup begins, Vitess rotates the MySQL binary logs on the tablet, so that it does not back up an active log file.

Expand All @@ -122,7 +122,7 @@ An incremental backup fails in these scenarios:
- It is unable to find binary log files that covers the requested position. This can happen if the binary logs are purged earlier than the incremental backup was taken. It essentially means there's a gap in the changelog events. **Note** that while on one tablet the binary logs may be missing, another tablet may still have binary logs that cover the requested position.
- There is no change to the database since the requested position, i.e. the GTID position has not changed since.

`v17` only supports `--incremental_from_pos` in the `Backup` command, not in `BackupShard`. Also, only `vtctlclient` supports the flag, where `vtctldclient` does not. `v18` is expected to support incremental backups for `BackupShard` and for `vtctldclient`.
`v17` only supports `--incremental-from-pos` in the `Backup` command, not in `BackupShard`. Also, only `vtctlclient` supports the flag, where `vtctldclient` does not. `v18` is expected to support incremental backups for `BackupShard` and for `vtctldclient`.

## Backing up Topology Server

Expand Down