Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update log backup user docs #2641

Open
wants to merge 43 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
12939f6
subcommands
RidRisR Oct 25, 2024
10092d5
stop
RidRisR Oct 25, 2024
1cb9ff6
crd
RidRisR Oct 25, 2024
e3d72d7
Update en/backup-to-aws-s3-using-br.md
RidRisR Nov 11, 2024
52450ed
Update en/backup-to-aws-s3-using-br.md
RidRisR Nov 11, 2024
8674d2c
fix
RidRisR Nov 11, 2024
99f9953
lint
RidRisR Nov 12, 2024
21e686b
lint
RidRisR Nov 12, 2024
1c333ce
lint
RidRisR Nov 12, 2024
8e6533e
Update zh/backup-to-aws-s3-using-br.md
RidRisR Nov 25, 2024
95d1c68
fix comment
RidRisR Nov 28, 2024
fd68f5b
modify more examples
RidRisR Nov 28, 2024
a7fc814
lint
RidRisR Nov 28, 2024
9252218
add version
RidRisR Nov 28, 2024
dc23a0d
Update zh/backup-restore-overview.md
RidRisR Nov 28, 2024
ef1a001
Update en/backup-to-aws-s3-using-br.md
RidRisR Nov 28, 2024
33b1b74
Update en/backup-to-aws-s3-using-br.md
RidRisR Nov 28, 2024
1bf5060
fix comment
RidRisR Nov 28, 2024
5132b8b
use -
RidRisR Nov 28, 2024
2770c85
lint
RidRisR Nov 28, 2024
461ed2a
fix
RidRisR Nov 28, 2024
be007a3
fix
RidRisR Nov 28, 2024
6882037
replace • with -
RidRisR Nov 29, 2024
ce5ac2d
lint
RidRisR Nov 29, 2024
7dda757
Discard changes to .markdownlint.yaml
Oreoxmt Dec 25, 2024
f869a85
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
1534f5e
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
ee452f7
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
168ed6b
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
015a4d2
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
1296659
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
760b637
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
f13a5cf
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
dbb04fd
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
05e8505
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
f29b5fa
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
734177c
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
55b2671
Update zh/backup-to-azblob-using-br.md
RidRisR Dec 26, 2024
041af46
Update zh/backup-to-gcs-using-br.md
RidRisR Dec 26, 2024
f80c664
Update zh/backup-to-gcs-using-br.md
RidRisR Dec 26, 2024
8f56e79
Update zh/backup-restore-cr.md
RidRisR Dec 26, 2024
0268bf1
Update zh/backup-restore-overview.md
RidRisR Dec 26, 2024
ec8d4f0
Update zh/backup-to-aws-s3-using-br.md
RidRisR Dec 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions en/backup-restore-cr.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ This section introduces the fields in the `Backup` CR.
* `volume-snapshot`: back up data by volume snapshots.
* `log`: back up log data in real time in the KV layer.

* `.spec.logSubcommand`: the subcommand for controlling the log backup status in the Backup CR. This field provides three options for managing a log backup task:
RidRisR marked this conversation as resolved.
Show resolved Hide resolved
* `log-start`: Initiates a new log backup task or resumes an existing task that has been paused. Use this option to start the log backup process or to continue it from a paused state.
* `log-pause`: Temporarily pauses an active log backup task, allowing it to be resumed later using the log-start command.
* `log-stop`: Permanently stops the log backup task. When this command is issued, the Backup CR enters a stopped state and cannot be restarted.

* `.spec.restoreMode`: the restore mode. The default value is `snapshot`, which means restoring data from snapshots in the KV layer. This field is valid only for restore and has three value options currently:
* `snapshot`: restore data from snapshots in the KV layer.
* `volume-snapshot`: restore data from volume snapshots.
Expand Down
2 changes: 2 additions & 0 deletions en/backup-restore-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ kubectl delete backupschedule ${name} -n ${namespace}

If you use TiDB Operator v1.1.2 or an earlier version, or if you use TiDB Operator v1.1.3 or a later version and set the value of `spec.cleanPolicy` to `Delete`, TiDB Operator cleans the backup data when it deletes the CR.

If you are using version 1.5 or later, when you delete a CR, TiDB Operator will attempt to automatically stop the running log backup task. The auto-stop feature will only stop log backup tasks that are running on track; it does not handle tasks that are in an error or failed state.
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

If you back up cluster data using AWS EBS volume snapshots and set the value of `spec.cleanPolicy` to `Delete`, TiDB Operator deletes the CR, and cleans up the backup file and the volume snapshots on AWS.

In such cases, if you need to delete the namespace, it is recommended that you first delete all the `Backup`/`BackupSchedule` CRs and then delete the namespace.
Expand Down
109 changes: 105 additions & 4 deletions en/backup-to-aws-s3-using-br.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,21 @@ demo1-full-backup-s3 full snapshot Complete s3://my-bucket/my-full-backu

You can use a `Backup` CR to describe the start and stop of a log backup task and manage the log backup data. Log backup grants permissions to remote storages in the same way as snapshot backup. In this section, the example shows log backup operations by taking a `Backup` CR named `demo1-log-backup-s3` as an example. Note that these operations assume that permissions to remote storages are granted using accessKey and secretKey. See the following detailed steps.

#### Log backup subcommands
csuzhangxc marked this conversation as resolved.
Show resolved Hide resolved

The logSubcommand field in the Backup CR allows you to control the status of a log backup task. There are three valid inputs for logSubcommand:

• log-start: This command initiates a new log backup task or resumes an existing task that has been paused. It can be used to start the log backup process or resume from the paused state.
• log-pause: This command temporarily pauses an active log backup task. The task can be resumed later using the log-start command.
• log-stop: This command permanently stops the log backup task. When this command is issued, the Backup CR enters a stopped state and cannot be restarted.
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

These commands allow fine-grained control over the lifecycle of log backup tasks, enabling start, pause, resume, and stop operations to manage log data retention in a Kubernetes environment.

<Tip>
In the v1.5 or earily TiDB Operator versions. You could use `logStop: true/false` to stop/start the task.
You are not recommanded to do so in the later versions.
</Tip>

#### Start log backup

1. In the `backup-test` namespace, create a `Backup` CR named `demo1-log-backup-s3`.
Expand All @@ -247,6 +262,7 @@ You can use a `Backup` CR to describe the start and stop of a log backup task an
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
Expand Down Expand Up @@ -308,15 +324,101 @@ Conditions:
Log Checkpoint Ts: 436569119308644661
```

#### Pause log backup

Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can stop the log backup by modifying the same `Backup` CR.
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

To pause the log backup task, you only need to change the `logSubcommand` from `log-start` to `log-pause`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-pause
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Running` to `Pause`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Pause ....
```

#### Resume log backup

If a log backup task is paused, you could set `logSubcommand: log-start` to resume it. Be aware that, you couldn't resume a task from `Fail` or `Stopped` state.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

To resume the log backup task, you only need to change the `logSubcommand` from `log-pause` to `log-start`. Then save and quit the editor. The modified content is as follows:

```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-start
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Running` to `Pause`:
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Running ....
```

#### Stop log backup

Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can stop the log backup by modifying the same `Backup` CR. The priority of all operations is: stop log backup > delete log backup data > start log backup.
Because you already created a `Backup` CR named `demo1-log-backup-s3` when you started log backup, you can stop the log backup by modifying the same `Backup` CR.

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

In the last line of the CR, append `spec.logStop: true`. Then save and quit the editor. The modified content is as follows:
In the last line of the CR, change the change the `logSubcommand` to `log-pause`. Then save and quit the editor. The modified content is as follows:
RidRisR marked this conversation as resolved.
Show resolved Hide resolved
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

```yaml
---
Expand All @@ -337,7 +439,6 @@ spec:
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
logStop: true
```

You can see the `STATUS` of the `Backup` CR named `demo1-log-backup-s3` change from `Running` to `Stopped`:
Expand All @@ -352,7 +453,7 @@ demo1-log-backup-s3 log Stopped ....
```

<Tip>
You can also stop log backup by taking the same steps as in [Start log backup](#start-log-backup). The existing `Backup` CR will be updated.
Stopped is the terminated state of a log backup CR, you couldn't change the state again, but you still could clean log backup data.
</Tip>

#### Clean log backup data
Expand Down
5 changes: 5 additions & 0 deletions zh/backup-restore-cr.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ summary: 介绍用于备份与恢复的 Custom Resource (CR) 资源的各字段
* `volume-snapshot`:基于卷快照的备份。
* `log`:从 KV 层备份实时数据变更日志数据。

* `.spec.logSubcommand`:指定日志备份任务的子命令,用于控制日志备份任务的状态。该字段提供以下三种选项:
* `log-start`:启动一个新的日志备份任务,或恢复一个已暂停的任务。可以用于开始日志备份流程或从暂停状态恢复。
* `log-pause`:暂时暂停正在运行的日志备份任务,可以稍后使用 log-start 命令恢复任务。
* `log-stop`:永久停止日志备份任务。当此命令发出后,Backup CR 进入停止状态,无法再次启动。

* `.spec.restoreMode`:指定 Restore 的模式,默认为 `snapshot`,即基于 KV 层的快照恢复。该字段仅在恢复时有效,目前支持以下三种类型:
* `snapshot`:基于 KV 层的快照恢复。
* `volume-snapshot`:基于卷快照的 TiDB 集群恢复。
Expand Down
2 changes: 2 additions & 0 deletions zh/backup-restore-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ kubectl delete backupschedule ${name} -n ${namespace}

如果你使用 v1.1.2 及以前版本,或使用 v1.1.3 及以后版本并将 `spec.cleanPolicy` 设置为 `Delete` 时,TiDB Operator 在删除 CR 时会同时清理备份文件。

如果你使用v1.5及以后的版本,当你删除CR时,TiDB Operator会尝试自动停止正在运行的日志备份任务。自动停止功能只会停止正常运行的日志备份任务,并不处理错误或者失败的任务。
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

如果你使用基于 AWS EBS 卷快照的备份,并将 `spec.cleanPolicy` 设置为 `Delete` 时,TiDB Operator 在删除 CR 以及清理备份文件的同时删除 AWS 上相关的所有卷快照。

在满足上述条件时,如果需要删除 namespace,建议首先删除所有的 Backup/BackupSchedule CR,再删除 namespace。
Expand Down
108 changes: 105 additions & 3 deletions zh/backup-to-aws-s3-using-br.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,21 @@ demo1-full-backup-s3 full snapshot Complete s3://my-bucket/my-full-backu

你可以使用一个 `Backup` CR 来描述日志备份任务的启动、停止以及清理日志备份数据等操作。日志备份对远程存储访问授权方式与快照备份一致。本节示例创建了名为 `demo1-log-backup-s3` 的 `Backup` CR,对远程存储访问授权方式仅以通过 accessKey 和 secretKey 的方式为例,具体操作如下所示。

#### 日志备份的`logSubcommand`
RidRisR marked this conversation as resolved.
Show resolved Hide resolved

Backup 自定义资源(CR)中的 logSubcommand 字段允许你控制日志备份任务的状态。logSubcommand 有三个有效输入:

• log-start:该命令用于启动新的日志备份任务,或恢复已暂停的任务。它可用于启动日志备份进程或从暂停状态恢复任务。
• log-pause:该命令用于暂时暂停活动中的日志备份任务。稍后可以通过 log-start 命令恢复任务。
• log-stop:该命令永久停止日志备份任务。当执行此命令时,Backup CR 将进入停止状态且无法重启。

这些命令使日志备份任务的生命周期控制更加精细,支持启动、暂停、恢复和停止操作,以管理 Kubernetes 环境中的日志数据保留。

<Tip>
在 TiDB Operator v1.5 或更早的版本中,你可以使用 logStop: true/false 来停止或启动任务。
然而,不建议在后续版本中继续使用该方式。
</Tip>

#### 启动日志备份

1. 在 `backup-test` 这个 namespace 中创建一个名为 `demo1-log-backup-s3` 的 `Backup` CR。
Expand Down Expand Up @@ -305,15 +320,102 @@ Conditions:
Log Checkpoint Ts: 436569119308644661
```

#### 暂停日志备份

由于在启动日志备份时已经创建了名为 demo1-log-backup-s3 的 Backup 自定义资源(CR),可以通过修改相同的 Backup CR 来暂停日志备份。

```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

要暂停日志备份任务,只需将 logSubcommand 从 log-start 更改为 log-pause。然后保存并退出编辑器。修改后的内容如下:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
要暂停日志备份任务,只需将 logSubcommandlog-start 更改为 log-pause然后保存并退出编辑器。修改后的内容如下:
要暂停日志备份任务,只需将 `logSubcommand` 的值从 `log-start` 修改为 `log-pause`然后保存并退出编辑器。修改后的内容如下:


```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-pause
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```


可以看到名为 `demo1-log-backup-s3` 的 `Backup` CR 的 `STATUS` 从 `Running` 变成了 `Pause`:

```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Pause ....
```

#### 恢复日志备份

如果日志备份任务已暂停,可以将 logSubcommand 设置为 log-start 来恢复它。请注意,你无法从 Fail 或 Stopped 状态恢复任务。
Copy link
Collaborator

@Oreoxmt Oreoxmt Dec 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
如果日志备份任务已暂停,可以将 logSubcommand 设置为 log-start 来恢复它。请注意,你无法从 Fail 或 Stopped 状态恢复任务
如果日志备份任务已暂停,你可以通过将 `logSubcommand` 字段设置为 `log-start` 来恢复该任务。下面以恢复[暂停日志备份](#暂停日志备份)中已暂停的 `demo1-log-backup-s3` CR 为例
> **Note:**
>
> 此操作仅适用于处于暂停状态 (`Pause`) 的任务,无法恢复状态为 `Fail``Stopped` 的任务。


```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

要恢复日志备份任务,只需将 logSubcommand 从 log-pause 更改为 log-start。然后保存并退出编辑器。修改后的内容如下:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
要恢复日志备份任务,只需将 logSubcommandlog-pause 更改为 log-start然后保存并退出编辑器。修改后的内容如下:
要恢复日志备份任务,只需将 `logSubcommand` 的值从 `log-pause` 更改为 `log-start`然后保存并退出编辑器。修改后的内容如下:


```yaml
---
apiVersion: pingcap.com/v1alpha1
kind: Backup
metadata:
name: demo1-log-backup-s3
namespace: backup-test
spec:
backupMode: log
logSubcommand: log-pause
RidRisR marked this conversation as resolved.
Show resolved Hide resolved
br:
cluster: demo1
clusterNamespace: test1
sendCredToTikv: true
s3:
provider: aws
secretName: s3-secret
region: us-west-1
bucket: my-bucket
prefix: my-log-backup-folder
```

可以看到名为 demo1-log-backup-s3 的 Backup CR 的 STATUS 从 Paused 状态变为 Running:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
可以看到名为 demo1-log-backup-s3 的 Backup CR 的 STATUS 从 Paused 状态变为 Running:
可以看到名为 `demo1-log-backup-s3` 的 Backup CR 的 `STATUS``Paused` 状态变为 `Running`


```shell
kubectl get backup -n backup-test
```

```
NAME MODE STATUS ....
demo1-log-backup-s3 log Running ....
```

#### 停止日志备份

由于你在开启日志备份的时候已经创建了名为 `demo1-log-backup-s3` 的 `Backup` CR,因此可以直接更新该 `Backup` CR 的配置,来激活停止日志备份的操作。操作激活优先级从高到低分别是停止日志备份任务、删除日志备份数据和开启日志备份任务
由于你在开启日志备份的时候已经创建了名为 `demo1-log-backup-s3` 的 `Backup` CR,因此可以直接更新该 `Backup` CR 的配置,停止日志备份
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
由于你在开启日志备份的时候已经创建了名为 `demo1-log-backup-s3``Backup` CR,因此可以直接更新该 `Backup` CR 的配置,停止日志备份
你可以通过将 Backup 自定义资源 (CR) 的 `logSubcommand` 字段设置为 `log-stop` 来停止日志备份。下面以停止[启动日志备份](#启动日志备份)中创建的名为 `demo1-log-backup-s3`CR 为例


```shell
kubectl edit backup demo1-log-backup-s3 -n backup-test
```

在最后新增一行字段 `spec.logStop: true`,保存并退出。更新后的内容如下
在 CR 的最后一行,将 logSubcommand 更改为 log-pause。然后保存并退出编辑器。修改后的内容如下

```yaml
---
Expand Down Expand Up @@ -349,7 +451,7 @@ demo1-log-backup-s3 log Stopped ....
```

<Tip>
你也可以采用和启动日志备份时相同的方法来停止日志备份,已经被创建过的 `Backup` CR 会因此被更新
Stopped 是日志备份的终止状态,此状态下无法再次更改状态,但你仍然可以清理日志备份的数据
</Tip>

#### 清理日志备份数据
Expand Down
Loading