v1.13.0-rc.2

v1.13

2024-01-10

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.13.0-rc.2

Container Image

velero/velero:v1.13.0-rc.2

Documentation

https://velero.io/docs/v1.13/

Upgrading

https://velero.io/docs/v1.13/upgrade-to-1.13/

Highlights

Resource Modifier Enhancement

Velero introduced the Resource Modifiers in v1.12.0. This feature allows users to specify a ConfigMap with a set of rules to modify the resources during restoration. However, only the JSON Patch is supported when creating the rules, and JSON Patch has some limitations, which cannot cover all use cases. In v1.13.0, Velero adds new support for JSON Merge Patch and Strategic Merge Patch, which provide more power and flexibility and allow users to use the same ConfigMap to apply patches on the resources. More design details can be found in Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers design. For instructions on how to use the feature, please refer to the Resource Modifiers doc.

Node-Agent Concurrency

Velero data movement activities from fs-backups and CSI snapshot data movements run in Velero node-agent, so may be hosted by every node in the cluster and consume resources (i.e. CPU, memory, network bandwidth) from there. With v1.13, users are allowed to configure how many data movement activities (a.k.a, loads) run in each node globally or by node, so that users can better leverage the performance of Velero data movement activities and the resource consumption in the cluster. For more information, check the Node-Agent Concurrency document.

Parallel Files Upload Options

Velero now supports configurable options for parallel files upload when using Kopia uploader to do fs-backups or CSI snapshot data movements which makes speed up backup possible.
For more information, please check Here.

Write Sparse Files Options

If using fs-restore or CSI snapshot data movements, it’s supported to write sparse files during restore. For more information, please check Here.

Backup Describe

In v1.13, the Backup Volume section is added to the velero backup describe command output. The backup Volumes section describes information for all the volumes included in the backup of various backup types, i.e. native snapshot, fs-backup, CSI snapshot, and CSI snapshot data movement. Particularly, the velero backup description now supports showing the information of CSI snapshot data movements, which is not supported in v1.12.

Additionally, backup describe command will not check EnableCSI feature gate from client side, so if a backup has volumes with CSI snapshot or CSI snapshot data movement, backup describe command always shows the corresponding information in its output.

Backup's new VolumeInfo metadata

Create a new metadata file in the backup repository's backup name sub-directory to store the backup-including PVC and PV information. The information includes the backing-up method of the PVC and PV data, snapshot information, and status. The VolumeInfo metadata file determines how the PV resource should be restored. The Velero downstream software can also use this metadata file to get a summary of the backup's volume data information.

Enhancement for CSI Snapshot Data Movements when Velero Pod Restart

When performing backup and restore operations, enhancements have been implemented for Velero server pods or node agents to ensure that the current backup or restore process is not stuck or interrupted after restart due to certain exceptional circumstances.

New status fields added to show hook execution details

Hook execution status is now included in the backup/restore CR status and displayed in the backup/restore describe command output. Specifically, it will show the number of hooks which attempted to execute under the HooksAttempted field and the number of hooks which failed to execute under the HooksFailed field.

AWS SDK Bump Up

Bump up AWS SDK for Go to version 2, which offers significant performance improvements in CPU and memory utilization over version 1.

Azure AD/Workload Identity Support

Azure AD/Workload Identity is the recommended approach to do the authentication with Azure services/AKS, Velero has introduced support for Azure AD/Workload Identity on the Velero Azure plugin side in previous releases, and in v1.13.0 Velero adds new support for Kopia operations(file system backup/data mover/etc.) with Azure AD/Workload Identity.

Runtime and dependencies

To fix CVEs and keep pace with Golang, Velero made changes as follows:

• Bump Golang runtime to v1.21.6.
• Bump several dependent libraries to new versions.
• Bump Kopia to v0.15.0.

Breaking changes

• Backup describe command: due to the backup describe output enhancement, some existing information (i.e. the output for native snapshot, CSI snapshot, and fs-backup) has been moved to the Backup Volumes section with some format changes.
• API type changes: changes the field DataMoverConfig in DataUploadSpec from *map[string][string]`` to map[string]string`
• Velero install command: due to the issue #7264, v1.13.0 introduces a break change that make the informer cache enabled by default to keep the actual behavior consistent with the helper message(the informer cache is disabled by default before the change).

Limitations/Known issues

• The backup's VolumeInfo metadata doesn't have the information updated in the async operations. This function could be supported in v1.14 release.

Note

• Velero introduces the informer cache which is enabled by default. The informer cache improves the restore performance but may cause higher memory consumption. Increase the memory limit of the Velero pod or disable the informer cache by specifying the --disable-informer-cache option when installing Velero if you get the OOM error.

Deprecation announcement

• The generated k8s clients, informers, and listers are deprecated in the Velero v1.13 release. They are put in the Velero repository's pkg/generated directory. According to the n+2 supporting policy, the deprecated are kept for two more releases. The pkg/generated directory should be deleted in the v1.15 release.
• After the backup VolumeInfo metadata file is added to the backup, Velero decides how to restore the PV resource according to the VolumeInfo content. To support the backup generated by the older version of Velero, the old logic is also kept. The support for the backup without the VolumeInfo metadata file will be kept for two releases. The support logic will be deleted in the v1.15 release.

All Changes

• Check resource Group Version and Kind is available in cluster before attempting restore to prevent being stuck (#7336, @kaovilai)
• Make "disable-informer-cache" option false(enabled) by default to keep it consistent with the help message (#7294, @ywk253100)
• Fix issue #6928, remove snapshot deletion timeout for PVB (#7282, @Lyndon-Li)
• Do not set "targetNamespace" to namespace items (#7274, @reasonerjt)
• Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7245, @Lyndon-Li)
• Adjust the newline output of resource list in restore describer (#7238, @allenxu404)
• Remove the redundant newline in backup describe output (#7229, @allenxu404)
• Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7201, @Lyndon-Li)
• Update CSIVolumeSnapshotsCompleted in backup's status and the metric
  during backup finalize stage according to async operations content. (#7184, @blackpiglet)
• Refactor DownloadRequest Stream function (#7175, @blackpiglet)
• Add --skip-immediately flag to schedule commands; --schedule-skip-immediately server and install (#7169, @kaovilai)
• Add node-agent concurrency doc and change the config name from dataPathConcurrency to loadCocurrency (#7161, @Lyndon-Li)
• Enhance hooks tracker by adding a returned error to record function (#7153, @allenxu404)
• Track the skipped PV when SnapshotVolumes set as false (#7152, @reasonerjt)
• Add more linters part 2. (#7151, @blackpiglet)
• Fix issue #7135, check pod status before checking node-agent pod status (#7150, @Lyndon-Li)
• Treat namespace as a regular restorable item (#7143, @reasonerjt)
• Allow sparse option for Kopia & Restic restore (#7141, @qiuming-best)
• Use VolumeInfo to help restore the PV. (#7138, @blackpiglet)
• Node agent restart enhancement (#7130, @qiuming-best)
• Fix issue #6695, add describe for data mover backups (#7125, @Lyndon-Li)
• Add hooks status to backup/restore CR (#7117, @allenxu404)
• Include plugin name in the error message by operations (#7115, @reasonerjt)
• Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7102, @Lyndon-Li)
• Generate VolumeInfo for backup. (#7100, @blackpiglet)
• Fix issue #7094, fallback to full backup if previous snapshot is not found (#7096, @Lyndon-Li)
• Fix issue #7068, due to an behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API i...

v1.13.0-rc.1

v1.13

2024-01-10

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.13.0-rc.1

Container Image

velero/velero:v1.13.0-rc.1

Documentation

https://velero.io/docs/v1.13/

Upgrading

https://velero.io/docs/v1.13/upgrade-to-1.13/

Highlights

Resource Modifier Enhancement

Velero introduced the Resource Modifiers in v1.12.0. This feature allows users to specify a ConfigMap with a set of rules to modify the resources during restoration. However, only the JSON Patch is supported when creating the rules, and JSON Patch has some limitations, which cannot cover all use cases. In v1.13.0, Velero adds new support for JSON Merge Patch and Strategic Merge Patch, which provide more power and flexibility and allow users to use the same ConfigMap to apply patches on the resources. More design details can be found in Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers design. For instructions on how to use the feature, please refer to the Resource Modifiers doc.

Node-Agent Concurrency

Velero data movement activities from fs-backups and CSI snapshot data movements run in Velero node-agent, so may be hosted by every node in the cluster and consume resources (i.e. CPU, memory, network bandwidth) from there. With v1.13, users are allowed to configure how many data movement activities (a.k.a, loads) run in each node globally or by node, so that users can better leverage the performance of Velero data movement activities and the resource consumption in the cluster. For more information, check the Node-Agent Concurrency document.

Parallel Files Upload Options

Velero now supports configurable options for parallel files upload when using Kopia uploader to do fs-backups or CSI snapshot data movements which makes speed up backup possible.
For more information, please check Here.

Write Sparse Files Options

If using fs-restore or CSI snapshot data movements, it’s supported to write sparse files during restore. For more information, please check Here.

Backup Describe

In v1.13, the Backup Volume section is added to the velero backup describe command output. The backup Volumes section describes information for all the volumes included in the backup of various backup types, i.e. native snapshot, fs-backup, CSI snapshot, and CSI snapshot data movement. Particularly, the velero backup description now supports showing the information of CSI snapshot data movements, which is not supported in v1.12.

Additionally, backup describe command will not check EnableCSI feature gate from client side, so if a backup has volumes with CSI snapshot or CSI snapshot data movement, backup describe command always shows the corresponding information in its output.

Backup's new VolumeInfo metadata

Create a new metadata file in the backup repository's backup name sub-directory to store the backup-including PVC and PV information. The information includes the backing-up method of the PVC and PV data, snapshot information, and status. The VolumeInfo metadata file determines how the PV resource should be restored. The Velero downstream software can also use this metadata file to get a summary of the backup's volume data information.

Enhancement for CSI Snapshot Data Movements when Velero Pod Restart

When performing backup and restore operations, enhancements have been implemented for Velero server pods or node agents to ensure that the current backup or restore process is not stuck or interrupted after restart due to certain exceptional circumstances.

New status fields added to show hook execution details

Hook execution status is now included in the backup/restore CR status and displayed in the backup/restore describe command output. Specifically, it will show the number of hooks which attempted to execute under the HooksAttempted field and the number of hooks which failed to execute under the HooksFailed field.

AWS SDK Bump Up

Bump up AWS SDK for Go to version 2, which offers significant performance improvements in CPU and memory utilization over version 1.

Azure AD/Workload Identity Support

Azure AD/Workload Identity is the recommended approach to do the authentication with Azure services/AKS, Velero has introduced support for Azure AD/Workload Identity on the Velero Azure plugin side in previous releases, and in v1.13.0 Velero adds new support for Kopia operations(file system backup/data mover/etc.) with Azure AD/Workload Identity.

Runtime and dependencies

To fix CVEs and keep pace with Golang, Velero made changes as follows:

• Bump Golang runtime to v1.21.6.
• Bump several dependent libraries to new versions.
• Bump Kopia to v0.15.0.

Breaking changes

• Backup describe command: due to the backup describe output enhancement, some existing information (i.e. the output for native snapshot, CSI snapshot, and fs-backup) has been moved to the Backup Volumes section with some format changes.
• API type changes: changes the field DataMoverConfig in DataUploadSpec from *map[string][string] to map[string]string
• Velero install command: due to the issue #7264, v1.13.0 introduces a break change that make the informer cache enabled by default to keep the actual behavior consistent with the helper message(the informer cache is disabled by default before the change).

Limitations/Known issues

• The backup's VolumeInfo metadata doesn't have the information updated in the async operations. This function could be supported in v1.14 release.

Deprecation announcement

• The generated k8s clients, informers, and listers are deprecated in the Velero v1.13 release. They are put in the Velero repository's pkg/generated directory. According to the n+2 supporting policy, the deprecated are kept for two more releases. The pkg/generated directory should be deleted in the v1.15 release.
• After the backup VolumeInfo metadata file is added to the backup, Velero decides how to restore the PV resource according to the VolumeInfo content. To support the backup generated by the older version of Velero, the old logic is also kept. The support for the backup without the VolumeInfo metadata file will be kept for two releases. The support logic will be deleted in the v1.15 release.

All Changes

• Make "disable-informer-cache" option false(enabled) by default to keep it consistent with the help message (#7294, @ywk253100)
• Fix issue #6928, remove snapshot deletion timeout for PVB (#7282, @Lyndon-Li)
• Do not set "targetNamespace" to namespace items (#7274, @reasonerjt)
• Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7245, @Lyndon-Li)
• Adjust the newline output of resource list in restore describer (#7238, @allenxu404)
• Remove the redundant newline in backup describe output (#7229, @allenxu404)
• Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7201, @Lyndon-Li)
• Update CSIVolumeSnapshotsCompleted in backup's status and the metric
  during backup finalize stage according to async operations content. (#7184, @blackpiglet)
• Refactor DownloadRequest Stream function (#7175, @blackpiglet)
• Add --skip-immediately flag to schedule commands; --schedule-skip-immediately server and install (#7169, @kaovilai)
• Add node-agent concurrency doc and change the config name from dataPathConcurrency to loadCocurrency (#7161, @Lyndon-Li)
• Enhance hooks tracker by adding a returned error to record function (#7153, @allenxu404)
• Track the skipped PV when SnapshotVolumes set as false (#7152, @reasonerjt)
• Add more linters part 2. (#7151, @blackpiglet)
• Fix issue #7135, check pod status before checking node-agent pod status (#7150, @Lyndon-Li)
• Treat namespace as a regular restorable item (#7143, @reasonerjt)
• Allow sparse option for Kopia & Restic restore (#7141, @qiuming-best)
• Use VolumeInfo to help restore the PV. (#7138, @blackpiglet)
• Node agent restart enhancement (#7130, @qiuming-best)
• Fix issue #6695, add describe for data mover backups (#7125, @Lyndon-Li)
• Add hooks status to backup/restore CR (#7117, @allenxu404)
• Include plugin name in the error message by operations (#7115, @reasonerjt)
• Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7102, @Lyndon-Li)
• Generate VolumeInfo for backup. (#7100, @blackpiglet)
• Fix issue #7094, fallback to full backup if previous snapshot is not found (#7096, @Lyndon-Li)
• Fix issue #7068, due to an behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7095, @Lyndon-Li)
• Skip syncing the backup which doesn't contain backup metadata (#7081, @ywk253100)
• Fix issue #6693, partially fail restore if CSI snapshot is involved but CSI feature is not ready, i.e., CSI feature gate is not enabled or CSI plugin is not installed. (#7077, @Lyndon-Li)
• Truncate the credential file to avoid the change of secret content messing it up (#7072, @ywk253100)
• Add VolumeInfo...

v1.12.3

v1.12.3

2024-01-09

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.3

Container Image

velero/velero:v1.12.3

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

All changes

• Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7247, @Lyndon-Li)
• Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7203, @Lyndon-Li)
• Update CSIVolumeSnapshotsCompleted in backup's status and the metric
  during backup finalize stage according to async operations content. (#7202, @blackpiglet)
• Node agent restart enhancement (#7130, @qiuming-best)
• Fix issue #6928, remove snapshot deletion timeout for PVB (#7283, @Lyndon-Li)

v1.12.3-rc.1

v1.12.3

2024-01-09

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.3-rc.1

Container Image

velero/velero:v1.12.3-rc.1

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

All changes

• Fix issue #7244. By the end of the upload, check the outstanding incomplete snapshots and delete them by calling ApplyRetentionPolicy (#7247, @Lyndon-Li)
• Fix issue #7189, data mover generic restore - don't assume the first volume as the restore volume (#7203, @Lyndon-Li)
• Update CSIVolumeSnapshotsCompleted in backup's status and the metric
  during backup finalize stage according to async operations content. (#7202, @blackpiglet)
• Node agent restart enhancement (#7130, @qiuming-best)
• Fix issue #6928, remove snapshot deletion timeout for PVB (#7283, @Lyndon-Li)

v1.12.2

v1.12.2

2023-11-27

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.2

Container Image

velero/velero:v1.12.2

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

All changes

• Resource Modifier support for Escaped bool, float, null (#7131, @anshulahuja98)
• Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7114, @Lyndon-Li)
• Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize (#7111, @kaovilai)
• Cherry-pick #6917 - Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#7049, @27149chen)
• Bump up Velero base image to latest patch release (#7110, @allenxu404)
• Fix the node-agent missing metrics-address defines. (#7098, @yanggangtony)
• Fix issue #7094, fallback to full backup if previous snapshot is not found (#7097, @Lyndon-Li)
• Add DataUpload Result and CSI VolumeSnapshot check for restore PV. (#7087, @blackpiglet)
• Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod (#7060, @Lyndon-Li)
• Truncate the credential file to avoid the change of secret content messing it up (#7058, @ywk253100)
• restore: Use warning when Create IsAlreadyExist and Get error (#7054, @kaovilai)
• Read information from the credential specified by BSL (#7033, @ywk253100)
• Fix issue 6913: Velero Built-in Datamover: Backup stucks in phase WaitingForPluginOperations when Node Agent pod gets restarted (#7025, @shubham-pampattiwar)
• Fix unified repository (kopia) s3 credentials profile selection (#6997, @kaovilai)

v1.12.2-rc.2

v1.12.2-rc.2

2023-11-24

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.2-rc.2

Container Image

velero/velero:v1.12.2-rc.2

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

All changes

• Resource Modifier support for Escaped bool, float, null (#7131, @anshulahuja98)
• Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7114, @Lyndon-Li)
• Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize (#7111, @kaovilai)
• Cherry-pick #6917 - Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#7049, @27149chen)
• Bump up Velero base image to latest patch release (#7110, @allenxu404)
• Fix the node-agent missing metrics-address defines. (#7098, @yanggangtony)
• Fix issue #7094, fallback to full backup if previous snapshot is not found (#7097, @Lyndon-Li)
• Add DataUpload Result and CSI VolumeSnapshot check for restore PV. (#7087, @blackpiglet)
• Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod (#7060, @Lyndon-Li)
• Truncate the credential file to avoid the change of secret content messing it up (#7058, @ywk253100)
• restore: Use warning when Create IsAlreadyExist and Get error (#7054, @kaovilai)
• Read information from the credential specified by BSL (#7033, @ywk253100)
• Fix issue 6913: Velero Built-in Datamover: Backup stucks in phase WaitingForPluginOperations when Node Agent pod gets restarted (#7025, @shubham-pampattiwar)
• Fix unified repository (kopia) s3 credentials profile selection (#6997, @kaovilai)

v1.12.2-rc.1

v1.12.2-rc.1

2023-11-20

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.2-rc.1

Container Image

velero/velero:v1.12.2-rc.1

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

All changes

• Fix issue #7068, due to a behavior of CSI external snapshotter, manipulations of VS and VSC may not be handled in the same order inside external snapshotter as the API is called. So add a protection finalizer to ensure the order (#7114, @Lyndon-Li)
• Update Backup.Status.CSIVolumeSnapshotsCompleted during finalize (#7111, @kaovilai)
• Cherry-pick #6917 - Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers (#7049, @27149chen)
• Bump up Velero base image to latest patch release (#7110, @allenxu404)
• Fix the node-agent missing metrics-address defines. (#7098, @yanggangtony)
• Fix issue #7094, fallback to full backup if previous snapshot is not found (#7097, @Lyndon-Li)
• Add DataUpload Result and CSI VolumeSnapshot check for restore PV. (#7087, @blackpiglet)
• Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod (#7060, @Lyndon-Li)
• Truncate the credential file to avoid the change of secret content messing it up (#7058, @ywk253100)
• restore: Use warning when Create IsAlreadyExist and Get error (#7054, @kaovilai)
• Read information from the credential specified by BSL (#7033, @ywk253100)
• Fix issue 6913: Velero Built-in Datamover: Backup stucks in phase WaitingForPluginOperations when Node Agent pod gets restarted (#7025, @shubham-pampattiwar)
• Fix unified repository (kopia) s3 credentials profile selection (#6997, @kaovilai)

v1.12.1

v1.12.1

2023-10-25

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.1

Container Image

velero/velero:v1.12.1

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

Highlights

Data Mover Adds Support for Block Mode Volumes

For PersistentVolumes with volumeMode set as Block, the volumes are mounted as raw block devices in pods, in 1.12.1, Velero CSI snapshot data movement supports to backup and restore this kind of volumes under linux based Kubernetes clusters.

New Parameter in Installation to Enable Data Mover

The velero install sub-command now includes a new parameter,--default-snapshot-move-data, which configures Velero server to move data by default for all snapshots supporting data movement. This feature is useful for users who will always want to use VBDM for backups instead of plain CSI , as they no longer need to specify the --snapshot-move-data flag for each individual backup.

Velero Base Image change

The base image previously used by Velero was distroless, which contains several CVEs cannot be addressed quickly. As a result, Velero will now use paketobuildpacks image starting from this new version.

Limitations/Known issues

• The data mover's support for block mode volumes is currently only applicable to Linux environments.

Note

• The support for linux/arm/v7 and linux/ppc64le platforms of the Velero server are removed as the base image doesn't support them

All changes

• Import auth provider plugins (#6970, @0x113)
• Perf improvements for existing resource restore (#6948, @sseago)
• Retry failed create when using generateName (#6943, @sseago)
• Fix issue #6647, add the --default-snapshot-move-data parameter to Velero install, so that users don't need to specify --snapshot-move-data per backup when they want to move snapshot data for all backups (#6940, @Lyndon-Li)
• Partially fix #6734, guide Kubernetes' scheduler to spread backup pods evenly across nodes as much as possible, so that data mover backup could achieve better parallelism (#6935, @Lyndon-Li)
• Replace the base image with paketobuildpacks image (#6934, @ywk253100)
• Add support for block volumes with Kopia (#6897, @dzaninovic)
• Set ParallelUploadAboveSize as MaxInt64 and flush repo after setting up policy so that policy is retrieved correctly by TreeForSource (#6886, @Lyndon-Li)
• Kubernetes 1.27 new job label batch.kubernetes.io/controller-uid are deleted during restore per kubernetes/kubernetes#114930 (#6713, @kaovilai)
• Add orLabelSelectors for backup, restore commands (#6881, @nilesh-akhade)
• Fix issue #6859, move plugin depending podvolume functions to util pkg, so as to remove the dependencies to unnecessary repository packages like kopia, azure, etc. (#6877, @Lyndon-Li)
• Fix issue #6786, always delete VSC regardless of the deletion policy (#6873, @Lyndon-Li)
• Fix #6988, always get region from BSL if it is not empty (#6991, @Lyndon-Li)
• Limit PVC block mode logic to non-Windows platform. (#6986, @blackpiglet)

v1.12.1-rc.1

v1.12.1

2023-10-20

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.1-rc.1

Container Image

velero/velero:v1.12.1-rc.1

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

Highlights

Data Mover Adds Support for Block Mode Volumes

For PersistentVolumes with volumeMode set as Block, the volumes are mounted as raw block devices in pods, in 1.12.1, Velero CSI snapshot data movement supports to backup and restore this kind of volumes under linux based Kubernetes clusters.

New Parameter in Installation to Enable Data Mover

The velero install sub-command now includes a new parameter,--default-snapshot-move-data, which configures Velero server to move data by default for all snapshots supporting data movement. This feature is useful for users who will always want to use VBDM for backups instead of plain CSI , as they no longer need to specify the --snapshot-move-data flag for each individual backup.

Velero Base Image change

The base image previously used by Velero was distroless, which contains several CVEs cannot be addressed quickly. As a result, Velero will now use paketobuildpacks image starting from this new version.

Limitations/Known issues

• The data mover's support for block mode volumes is currently only applicable to Linux environments.

All changes

• Import auth provider plugins (#6970, @0x113)
• Perf improvements for existing resource restore (#6948, @sseago)
• Retry failed create when using generateName (#6943, @sseago)
• Fix issue #6647, add the --default-snapshot-move-data parameter to Velero install, so that users don't need to specify --snapshot-move-data per backup when they want to move snapshot data for all backups (#6940, @Lyndon-Li)
• Partially fix #6734, guide Kubernetes' scheduler to spread backup pods evenly across nodes as much as possible, so that data mover backup could achieve better parallelism (#6935, @Lyndon-Li)
• Replace the base image with paketobuildpacks image (#6934, @ywk253100)
• Add support for block volumes with Kopia (#6897, @dzaninovic)
• Set ParallelUploadAboveSize as MaxInt64 and flush repo after setting up policy so that policy is retrieved correctly by TreeForSource (#6886, @Lyndon-Li)
• Kubernetes 1.27 new job label batch.kubernetes.io/controller-uid are deleted during restore per kubernetes/kubernetes#114930 (#6713, @kaovilai)
• Add orLabelSelectors for backup, restore commands (#6881, @nilesh-akhade)
• Fix issue #6859, move plugin depending podvolume functions to util pkg, so as to remove the dependencies to unnecessary repository packages like kopia, azure, etc. (#6877, @Lyndon-Li)
• Fix issue #6786, always delete VSC regardless of the deletion policy (#6873, @Lyndon-Li)
• Fix #6988, always get region from BSL if it is not empty (#6991, @Lyndon-Li)
• Add both non-Windows version and Windows version code for PVC block mode logic. (#6986, @blackpiglet)

v1.12.0

v1.12

2023-09-22

Download

https://github.com/vmware-tanzu/velero/releases/tag/v1.12.0

Container Image

velero/velero:v1.12.0

Documentation

https://velero.io/docs/v1.12/

Upgrading

https://velero.io/docs/v1.12/upgrade-to-1.12/

Highlights

CSI Snapshot Data Movement

CSI Snapshot Data Movement refers to back up CSI snapshot data from the volatile and limited production environment into durable, heterogeneous, and scalable backup storage in a consistent manner; and restore the data to volumes in the original or alternative environment.

CSI Snapshot Data Movement is useful in below scenarios:

• For on-premises users, the storage usually doesn't support durable snapshots, so it is impossible/less efficient/cost ineffective to keep volume snapshots by the storage This feature helps to move the snapshot data to a storage with lower cost and larger scale for long time preservation.
• For public cloud users, this feature helps users to fulfill the multiple cloud strategy. It allows users to back up volume snapshots from one cloud provider and preserve or restore the data to another cloud provider. Then users will be free to flow their business data across cloud providers based on Velero backup and restore

CSI Snapshot Data Movement is built according to the Volume Snapshot Data Movement design (Volume Snapshot Data Movement design). Additionally, guidance on how to use the feature can be found in the Volume Snapshot Data Movement doc(Volume Snapshot Data Movement doc).

Resource Modifiers

In many use cases, customers often need to substitute specific values in Kubernetes resources during the restoration process like changing the namespace, changing the storage class, etc.

To address this need, Resource Modifiers (also known as JSON Substitutions) offer a generic solution in the restore workflow. It allows the user to define filters for specific resources and then specify a JSON patch (operator, path, value) to apply to the resource. This feature simplifies the process of making substitutions without requiring the implementation of a new RestoreItemAction plugin. More design details can be found in Resource Modifiers design (Resource Modifiers design). For instructions on how to use the feature, please refer to Resource Modifiers doc(Resource Modifiers doc).

Multiple VolumeSnapshotClasses

Prior to version 1.12, the Velero CSI plugin would choose the VolumeSnapshotClass in the cluster based on matching driver names and the presence of the "velero.io/csi-volumesnapshot-class" label. However, this approach proved inadequate for many user scenarios.

With the introduction of version 1.12, Velero now offers support for multiple VolumeSnapshotClasses in the CSI Plugin, enabling users to select a specific class for a particular backup. More design details can be found in Multiple VolumeSnapshotClasses design (Multiple VolumeSnapshotClasses design). For instructions on how to use the feature, please refer to Multiple VolumeSnapshotClasses doc (Multiple VolumeSnapshotClasses doc).

Restore Finalizer

Before v1.12, the restore controller would only delete restore resources but wouldn’t delete restore data from the backup storage location when the command velero restore delete was executed. The only chance Velero deletes restores data from the backup storage location is when the associated backup is deleted.

In this version, Velero introduces a finalizer that ensures the cleanup of all associated data for restores when running the command velero restore delete.

Runtime and dependencies

To fix CVEs and keep pace with Golang, Velero made changes as follows:

• Bump Golang runtime to v1.20.7.
• Bump several dependent libraries to new versions.
• Bump Kopia to v0.13.

Breaking changes

• Prior to v1.12, the parameter uploader-type for Velero installation had a default value of "restic". However, starting from this version, the default value has been changed to "kopia". This means that Velero will now use Kopia as the default path for file system backup.
• The ways of setting CSI snapshot time have changed in v1.12. First, the sync waiting time for creating a snapshot handle in the CSI plugin is changed from the fixed 10 minutes into backup.Spec.CSISnapshotTimeout. The second, the async waiting time for VolumeSnapshot and VolumeSnapshotContent's status turning into ReadyToUse in operation uses the operation's timeout. The default value is 4 hours.
• As from Velero helm chart v4.0.0, it supports multiple BSL and VSL, and the BSL and VSL have changed from the map into a slice, and this breaking change is not backward compatible. So it would be best to change the BSL and VSL configuration into slices before the Upgrade.
• Prior to v1.12, deleting the Velero namespace would easily remove all the resources within it. However, with the introduction of finalizers attached to the Velero CR including restore, dataupload, and datadownload in this version, directly deleting Velero namespace may get stuck indefinitely because the pods responsible for handling the finalizers might be deleted before the resources attached to the finalizers. To avoid this issue, please use the command velero uninstall to delete all the Velero resources or ensure that you handle the finalizer appropriately before deleting the Velero namespace.

Limitations/Known issues

• The Azure plugin supports Azure AD Workload identity way, but it only works for Velero native snapshots. It cannot support filesystem backup and snapshot data mover scenarios.
• File System backup under Kopia path and CSI Snapshot Data Movement backup fail to back up files that are large the 2GiB due to issue #6668.

All Changes

• Fixes #6498. Get resource client again after restore actions in case resource's gv is changed. This is an improvement of pr #6499, to support group changes. A group change usually happens in a restore plugin which is used for resource conversion: convert a resource from a not supported gv to a supported gv (#6634, @27149chen)
• Add API support for volMode block, only error for now. (#6608, @shawn-hurley)
• Fix how the AWS credentials are obtained from configuration (#6598, @aws_creds)
• Add performance E2E test (#6569, @qiuming-best)
• Non default s3 credential profiles work on Unified Repository Provider (kopia) (#6558, @kaovilai)
• Fix issue #6571, fix the problem for restore item operation to set the errors correctly so that they can be recorded by Velero restore and then reflect the correct status for Velero restore. (#6594, @Lyndon-Li)
• Fix issue 6575, flush the repo after delete the snapshot, otherwise, the changes(deleting repo snapshot) cannot be committed to the repo. (#6587, @Lyndon-Li)
• Delete moved snapshots when the backup is deleted (#6547, @reasonerjt)
• check if restore crd exist before operating restores (#6544, @allenxu404)
• Remove PVC's selector in backup's PVC action. (#6481, @blackpiglet)
• Delete the expired deletebackuprequests that are stuck in "InProgress" (#6476, @reasonerjt)
• Fix issue #6534, reset PVB CR's StorageLocation to the latest one during backup sync as same as the backup CR. Also fix similar problem with DataUploadResult for data mover restore. (#6533, @Lyndon-Li)
• Fix issue #6519. Restrict the client manager of node-agent server to include only Velero resources from the server's namespace, otherwise, the controllers will try to reconcile CRs from all the installed Velero namespaces. (#6523, @Lyndon-Li)
• Track the skipped PVC and print the summary in backup log (#6496, @reasonerjt)
• Add restore finalizer to clean up external resources (#6479, @allenxu404)
• fix: Typos and add more spell checking rules to CI (#6415, @mateusoliveira43)
• Add missing CompletionTimestamp and metrics when restore moved into terminal phase in restoreOperationsReconciler (#6397, @Nutrymaco)
• Add support for resource Modifications in the restore flow. Also known as JSON Substitutions. (#6452, @anshulahuja98)
• Remove dependency of the legacy client code from pkg/cmd directory part 2 (#6497, @blackpiglet)
• Add data upload and download metrics (#6493, @allenxu404)
• Fix issue 6490, If a backup/restore has multiple async operations and one operation fails while others are still in-progress, when all the operations finish, the backup/restore will be set as Completed falsely (#6491, @Lyndon-Li)
• Velero Plugins no longer need kopia indirect dependency in their go.mod (#6484, @kaovilai)
• Remove dependency of the legacy client code from pkg/cmd directory (#6469, @blackpiglet)
• Add support for OpenStack CSI drivers topology keys (#6464, @openstack-csi-topology-keys)
• Add exit code log and possible memory shortage warning log for Restic command failure. (#6459, @blackpiglet)
• Modify DownloadRequest controller logic (#6433, @blackpiglet)
• Add data download controller for data mover (#6436, @qiuming-best)
• Fix hook filter display issue for backup describer (#6434, @allenxu404)
• Retrieve DataUpload into backup result ConfigMap during volume snapshot restore. (#6410, @blackpiglet)
• Design to add support for Multiple VolumeSnapshotClasses in CSI Plugin. (#5774, @anshulahuja98)
• Clarify the...