Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add repositoryServerUserHostname to the kopia repository based functions #2177

Merged
merged 26 commits into from
Jul 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
384a075
Add userHostname to the kopia repository based functions
r4rajat Jul 14, 2023
81b4ed3
refactored functions.rst - removed unnecessary line
r4rajat Jul 14, 2023
dd0ba4b
Refactor username - remove unwanted variable
r4rajat Jul 17, 2023
0a37b2b
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 17, 2023
97cc57b
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 18, 2023
2179439
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 18, 2023
a3babd4
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 18, 2023
e5d1d56
Change KopiaRepositoryServerUserHostname Argument Value
r4rajat Jul 18, 2023
716e5fd
Update docs with repositoryServerUserHostname argument
r4rajat Jul 18, 2023
5c397a6
Update docs with repositoryServerUserHostname argument's description
r4rajat Jul 18, 2023
9d3701b
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 19, 2023
8cdc03b
Update pkg/function/backup_data_using_kopia_server.go
r4rajat Jul 19, 2023
9d68115
Update comment
r4rajat Jul 19, 2023
b7e381a
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 19, 2023
c502748
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 20, 2023
6821550
remove hostName variable dependency
r4rajat Jul 20, 2023
345c4a5
Add comment
r4rajat Jul 20, 2023
79bd293
Update pkg/function/backup_data_using_kopia_server.go
r4rajat Jul 20, 2023
a57695c
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 24, 2023
ced0ba7
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 24, 2023
07f5c23
Rename Argument from hostname -> hostName
r4rajat Jul 24, 2023
31cbaef
Update Comments
r4rajat Jul 25, 2023
be2460d
Update argument hostname
r4rajat Jul 25, 2023
4bc81bd
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 25, 2023
5fb1bd0
Refactored variable userPassPhrase -> userPassphrase
r4rajat Jul 25, 2023
92344ff
Merge branch 'master' into enable_custom_user_hostname_in_kanister_fu…
r4rajat Jul 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1607,6 +1607,7 @@ Arguments:
`container`, Yes, `string`, name of the kanister sidecar container
`includePath`, Yes, `string`, path of the data to be backed up
`snapshotTags`, No, `string`, custom tags to be provided to the kopia snapshots
`repositoryServerUserHostname`, No, `string`, user's hostname to access the kopia repository server. Hostname would be available in the user access credential secret
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does second sentence mean, user should figure out the hostname by looking at the user access credential secret?
What does it mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes ! User could provide the hostname which he/she wants to use by looking at user access credentials secret. If it's not provided we'll get the last entry from the user access map which is present inside the secret by default.


Outputs:

Expand Down Expand Up @@ -1669,6 +1670,7 @@ function argument and restores data to the specified path.
`pod`, No, `string`, pod to which the volumes are attached
`volumes`, No, `map[string]string`, mapping of `pvcName` to `mountPath` under which the volume will be available
`podOverride`, No, `map[string]interface{}`, specs to override default pod specs with
`repositoryServerUserHostname`, No, `string`, user's hostname to access the kopia repository server. Hostname would be available in the user access credential secret

.. note::
The ``image`` argument requires the use of ``ghcr.io/kanisterio/kanister-tools``
Expand Down Expand Up @@ -1738,6 +1740,7 @@ function. It creates a new Pod that runs ``delete snapshot`` command.
`namespace`, Yes, `string`, namespace in which to execute the delete job
`backupID`, Yes, `string`, unique snapshot id generated during backup
`image`, Yes, `string`, image to be used for running delete job (should contain kopia binary)
`repositoryServerUserHostname`, No, `string`, user's hostname to access the kopia repository server. Hostname would be available in the user access credential secret

Example:

Expand Down
1 change: 1 addition & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,4 @@ Workflow
webhook
Kopia
kopia
hostname
58 changes: 42 additions & 16 deletions pkg/function/backup_data_using_kopia_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ const (
BackupDataUsingKopiaServerFuncName = "BackupDataUsingKopiaServer"
// BackupDataUsingKopiaServerSnapshotTagsArg is the key used for returning snapshot tags
BackupDataUsingKopiaServerSnapshotTagsArg = "snapshotTags"
// KopiaRepositoryServerUserHostname is the key used for returning the hostname of the user
KopiaRepositoryServerUserHostname = "repositoryServerUserHostname"
)

type backupDataUsingKopiaServerFunc struct{}
Expand Down Expand Up @@ -71,17 +73,19 @@ func (*backupDataUsingKopiaServerFunc) Arguments() []string {
BackupDataNamespaceArg,
BackupDataPodArg,
BackupDataUsingKopiaServerSnapshotTagsArg,
KopiaRepositoryServerUserHostname,
}
}

func (*backupDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.TemplateParams, args map[string]any) (map[string]any, error) {
var (
container string
err error
includePath string
namespace string
pod string
tagsStr string
container string
err error
includePath string
namespace string
pod string
tagsStr string
userHostname string
)
if err = Arg(args, BackupDataContainerArg, &container); err != nil {
return nil, err
Expand All @@ -98,6 +102,9 @@ func (*backupDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
if err = OptArg(args, BackupDataUsingKopiaServerSnapshotTagsArg, &tagsStr, ""); err != nil {
return nil, err
}
if err = OptArg(args, KopiaRepositoryServerUserHostname, &userHostname, ""); err != nil {
return nil, err
}

var tags []string = nil
if tagsStr != "" {
Expand All @@ -114,8 +121,7 @@ func (*backupDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
return nil, errors.Wrap(err, "Failed to fetch Kopia API Server Certificate Secret Data from Certificate")
}

username := tp.RepositoryServer.Username
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase)
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase, userHostname)
if err != nil {
return nil, errors.Wrap(err, "Failed to fetch Hostname/User Passphrase from Secret")
}
Expand All @@ -134,7 +140,7 @@ func (*backupDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
pod,
tp.RepositoryServer.Address,
fingerprint,
username,
tp.RepositoryServer.Username,
userAccessPassphrase,
tags,
)
Expand Down Expand Up @@ -223,24 +229,36 @@ func backupDataUsingKopiaServer(
return kopiacmd.ParseSnapshotCreateOutput(stdout, stderr)
}

func hostNameAndUserPassPhraseFromRepoServer(userCreds string) (string, string, error) {
func hostNameAndUserPassPhraseFromRepoServer(userCreds, hostname string) (string, string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func hostNameAndUserPassPhraseFromRepoServer(userCreds, hostname string) (string, string, error) {
func hostNameAndUserPassPhraseFromRepoServer(userCreds, hostName string) (string, string, error) {

should hostName be preferred?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see hostname is used as directly or as substring in other var name. I think we should change them to hostName or HostName. Whatever applicable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, seems good. Will make the changes !

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done !

var userAccessMap map[string]string
if err := json.Unmarshal([]byte(userCreds), &userAccessMap); err != nil {
return "", "", errors.Wrap(err, "Failed to unmarshal User Credentials Data")
}

var userPassPhrase string
var hostName string
// Check if hostname provided exists in the User Access Map
if hostname != "" {
err := checkHostnameExistsInUserAccessMap(userAccessMap, hostname)
if err != nil {
return "", "", errors.Wrap(err, "Failed to find hostname in the User Access Map")
}
}

// Set First Value of hostname and passphrase from the User Access Map
// Or if hostname provided by the user, set the hostname and password for hostname provided
var userPassphrase string
for key, val := range userAccessMap {
hostName = key
userPassPhrase = val
if hostname == "" || hostname == key {
hostname = key
userPassphrase = val
break
}
}

decodedUserPassPhrase, err := base64.StdEncoding.DecodeString(userPassPhrase)
decodedUserPassphrase, err := base64.StdEncoding.DecodeString(userPassphrase)
if err != nil {
return "", "", errors.Wrap(err, "Failed to Decode User Passphrase")
}
return hostName, string(decodedUserPassPhrase), nil
return hostname, string(decodedUserPassphrase), nil
}

func userCredentialsAndServerTLS(tp *param.TemplateParams) (string, string, error) {
Expand All @@ -254,3 +272,11 @@ func userCredentialsAndServerTLS(tp *param.TemplateParams) (string, string, erro
}
return string(userCredJSON), string(certJSON), nil
}

func checkHostnameExistsInUserAccessMap(userAccessMap map[string]string, hostname string) error {
// check if hostname that is provided by the user exists in the user access map
if _, ok := userAccessMap[hostname]; !ok {
return errors.New("hostname provided in the repository server CR does not exist in the user access map")
}
return nil
}
18 changes: 11 additions & 7 deletions pkg/function/delete_data_using_kopia_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,15 +61,17 @@ func (*deleteDataUsingKopiaServerFunc) Arguments() []string {
DeleteDataBackupIdentifierArg,
DeleteDataNamespaceArg,
RestoreDataImageArg,
KopiaRepositoryServerUserHostname,
}
}

func (*deleteDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.TemplateParams, args map[string]any) (map[string]any, error) {
var (
err error
image string
namespace string
snapID string
err error
image string
namespace string
snapID string
userHostname string
)
if err = Arg(args, DeleteDataBackupIdentifierArg, &snapID); err != nil {
return nil, err
Expand All @@ -80,6 +82,9 @@ func (*deleteDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
if err = Arg(args, RestoreDataImageArg, &image); err != nil {
return nil, err
}
if err = OptArg(args, KopiaRepositoryServerUserHostname, &userHostname, ""); err != nil {
return nil, err
}

userPassphrase, cert, err := userCredentialsAndServerTLS(&tp)
if err != nil {
Expand All @@ -91,8 +96,7 @@ func (*deleteDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
return nil, errors.Wrap(err, "Failed to fetch Kopia API Server Certificate Secret Data from Certificate")
}

username := tp.RepositoryServer.Username
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase)
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase, userHostname)
if err != nil {
return nil, errors.Wrap(err, "Failed to get hostname/user passphrase from Options")
}
Expand All @@ -112,7 +116,7 @@ func (*deleteDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templa
tp.RepositoryServer.Address,
fingerprint,
snapID,
username,
tp.RepositoryServer.Username,
userAccessPassphrase,
)
}
Expand Down
20 changes: 12 additions & 8 deletions pkg/function/restore_data_using_kopia_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,16 +68,18 @@ func (*restoreDataUsingKopiaServerFunc) Arguments() []string {
RestoreDataVolsArg,
RestoreDataPodOverrideArg,
RestoreDataImageArg,
KopiaRepositoryServerUserHostname,
}
}

func (*restoreDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.TemplateParams, args map[string]any) (map[string]any, error) {
var (
err error
image string
namespace string
restorePath string
snapID string
err error
image string
namespace string
restorePath string
snapID string
userHostname string
)
if err = Arg(args, RestoreDataBackupIdentifierArg, &snapID); err != nil {
return nil, err
Expand All @@ -91,6 +93,9 @@ func (*restoreDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templ
if err = Arg(args, RestoreDataImageArg, &image); err != nil {
return nil, err
}
if err = OptArg(args, KopiaRepositoryServerUserHostname, &userHostname, ""); err != nil {
return nil, err
}

userPassphrase, cert, err := userCredentialsAndServerTLS(&tp)
if err != nil {
Expand All @@ -115,8 +120,7 @@ func (*restoreDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templ
}
}

username := tp.RepositoryServer.Username
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase)
hostname, userAccessPassphrase, err := hostNameAndUserPassPhraseFromRepoServer(userPassphrase, userHostname)
if err != nil {
return nil, errors.Wrap(err, "Failed to get hostname/user passphrase from Options")
}
Expand All @@ -139,7 +143,7 @@ func (*restoreDataUsingKopiaServerFunc) Exec(ctx context.Context, tp param.Templ
tp.RepositoryServer.Address,
fingerprint,
snapID,
username,
tp.RepositoryServer.Username,
userAccessPassphrase,
sparseRestore,
vols,
Expand Down