Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configurable Kopia Maintenance Interval #8581

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kaovilai
Copy link
Member

@kaovilai kaovilai commented Jan 6, 2025

Signed-off-by: Tiger Kaovilai tkaovila@redhat.com

Thank you for contributing to Velero!

Please add a summary of your change

Configurable Kopia Maintenance Interval. repo-maintenance-job-configmap adds an option for fullMaintenanceInterval where fastGC (12 hours), and eagerGC (6 hours).
This configmap is loaded in assembleRepoParam that returns RepoParam which now contains the maintenance interval referenced by several repository functions.
The manager functions then call provider functions which read repoParam.FullMaintenanceInterval and add it to repoOption.
Repooption is then passed to repoService, in this case kopiaRepoService.Init
kopiaRepoService.Init calls writeInitParameters which finally set kopia p.FullCycle.Interval

Does your change fix a particular issue?

Fixes #8364

Please indicate you've done the following:

@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch 2 times, most recently from 6e3d09f to 569c2ef Compare January 15, 2025 10:03
Copy link

codecov bot commented Jan 15, 2025

Codecov Report

Attention: Patch coverage is 40.00000% with 12 lines in your changes missing coverage. Please review.

Project coverage is 59.37%. Comparing base (3eaa739) to head (7d9b489).
Report is 58 commits behind head on main.

Files with missing lines Patch % Lines
pkg/repository/udmrepo/kopialib/lib_repo.go 7.69% 9 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8581      +/-   ##
==========================================
+ Coverage   59.18%   59.37%   +0.18%     
==========================================
  Files         370      370              
  Lines       39595    40004     +409     
==========================================
+ Hits        23434    23752     +318     
- Misses      14679    14756      +77     
- Partials     1482     1496      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch from 569c2ef to bcc6e39 Compare January 15, 2025 13:13
@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch 3 times, most recently from 014d288 to 40af53f Compare January 15, 2025 13:54
@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch from 40af53f to 14200ad Compare January 15, 2025 15:13
@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch 2 times, most recently from cc27593 to 334ac16 Compare January 16, 2025 10:29
Copy link
Member Author

@kaovilai kaovilai Feb 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

found https://velero.io/docs/main/backup-repository-configuration/ so will add info there and link from maintenance config page.

Copy link
Member Author

@kaovilai kaovilai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently double checking the flow of backup-repository-configmap from specifying a configmap to reading to storing into a repoOption.StorageOptions[key]

func getBackupRepositoryConfig(ctx context.Context, ctrlClient client.Client, configName, namespace, repoName, repoType string, log logrus.FieldLogger) (map[string]string, error) {
if configName == "" {
is called [here](
config, err := getBackupRepositoryConfig(ctx, r, r.backupRepoConfig, r.namespace, req.Name, req.Spec.RepositoryType, log)

which defines config := map[string]string with all the values set by user.
This map of string is then set to backupRepository.spec.RepositoryConfig and patched to cluster CR in following
func (r *BackupRepoReconciler) initializeRepo(ctx context.Context, req *velerov1api.BackupRepository, log logrus.FieldLogger) error {

This function is only called when backupRepo is in newly created as part of a backuprepo Reconcile()

This is used by

func (urp *unifiedRepoProvider) GetStoreOptions(param interface{}) (map[string]string, error) {

via
storeVar, err := funcTable.getStorageVariables(repoParam.BackupLocation, urp.repoBackend, repoParam.BackupRepo.Spec.VolumeNamespace, repoParam.BackupRepo.Spec.RepositoryConfig)

However, getStorageVariables() do not pass all the options from backupRepoConfig back.

if backupRepoConfig != nil {
if v, found := backupRepoConfig[udmrepo.StoreOptionCacheLimit]; found {
result[udmrepo.StoreOptionCacheLimit] = v
}
}

So either make getStorageVariables() pass everything, or make it add the maintenance interval to one of the things it adds.

@kaovilai
Copy link
Member Author

kaovilai commented Feb 4, 2025

@blackpiglet @Lyndon-Li any opinions on making getStorageVariables return everything from backupRepoConfig?

So either make getStorageVariables() pass everything, or make it add the maintenance interval to one of the things it adds.

@kaovilai kaovilai force-pushed the configKopiaMaintInterval branch 2 times, most recently from d96729e to 3186280 Compare February 5, 2025 15:06
Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

comment update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

comment

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
…ons.

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Configurable Kopia Maintenance Interval
4 participants