-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement Kopia storage helpers #2655
Merged
Merged
Changes from 37 commits
Commits
Show all changes
39 commits
Select commit
Hold shift + click to select a range
680e782
Add safecli dependency
plar 72011e7
add new flag implementations based on the safecli package for the Kop…
plar 0f635ad
apply go fmt
plar d0a6dd1
Add common Kopia args and flags
plar ca853c1
Add Kopia storage core flags
plar 9850f4a
Fix Apply and test.Suit
plar 246e1c1
Remove variadic args for Common and Cache flags
plar 419652c
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar 24707e5
pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now
plar c029fc1
Merge branch 'master' into kopia-cli-core-infra
plar cff7220
Re-implement Kopia common args and opts; Sync with the master
plar 550d124
Add pkg/kopia/cli package
plar fc918a0
go mod tidy
plar 9530c91
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar 75fb036
Merge branch 'kopia-cli-common-flags' into kopia-cli-storage-core-flags
plar 9fdf94b
Add Kopia storage helpers
plar 2c6cb6e
Convert common flags from vars to funcs
plar 1a3ee2d
Add safecli dependency
plar 386e7e9
add new flag implementations based on the safecli package for the Kop…
plar b815f63
apply go fmt
plar bcafabb
Fix Apply and test.Suit
plar 6d61bb4
pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now
plar e6ddb8f
Add pkg/kopia/cli package
plar 4e7ffd7
go mod tidy
plar f16aea7
Update safecli to v0.0.5
plar dcd6425
Update safecli to v0.0.6
plar d3dfb31
Sync with upstream
plar d2ac987
Merge branch 'kopia-cli-common-flags' into kopia-cli-storage-core-flags
plar b093421
Fix tests
plar 6e71143
Merge branch 'kopia-cli-common-flags' into kopia-cli-storage-core-flags
plar de3a52b
Add Location.IsPointInTypeSupported
plar 722b279
Add tests for Location.IsPointInTypeSupported
plar 44c133e
Fix formatting
plar 1897a1e
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar 33b1342
organize imports
plar 83292af
Merge branch 'kopia-cli-common-flags' into kopia-cli-storage-core-flags
plar 516a14c
organize imports
plar cd03d1e
Merge branch 'master' into kopia-cli-storage-core-flags
plar 8cf36c7
Merge branch 'master' into kopia-cli-storage-core-flags
mergify[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package args | ||
|
||
// Cache provides the cache arguments for Kopia CLI. | ||
type Cache struct { | ||
CacheDirectory string // the directory where cache is stored. Default is "/tmp/kopia-cache". | ||
ContentCacheSizeLimitMB int // the maximum size of the content cache in MB. | ||
MetadataCacheSizeLimitMB int // the maximum size of the metadata cache in MB. | ||
|
||
// unused? | ||
ContentCacheSizeMB int // the size of the content cache in MB. | ||
MetadataCacheSizeMB int // the size of the metadata cache in MB. | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package args | ||
|
||
// Common provides the common arguments for Kopia CLI. | ||
type Common struct { | ||
ConfigFilePath string // the path to the config file. | ||
LogDirectory string // the directory where logs are stored. | ||
LogLevel string // the level of logging. Default is "error". | ||
RepoPassword string // the password for the repository. | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
// This package contains the implementation of the Kopia CLI using github.com/kanisterio/safecli. | ||
|
||
package cli | ||
|
||
import ( | ||
_ "github.com/kanisterio/safecli" | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package cli | ||
|
||
import ( | ||
"github.com/pkg/errors" | ||
) | ||
|
||
// Common errors | ||
var ( | ||
// ErrInvalidID is returned when the ID is empty. | ||
ErrInvalidID = errors.New("invalid ID") | ||
) | ||
|
||
// storage errors | ||
var ( | ||
// ErrUnsupportedStorage is returned when the storage is not supported. | ||
ErrUnsupportedStorage = errors.New("unsupported storage") | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package args | ||
|
||
import ( | ||
"github.com/kanisterio/safecli/command" | ||
|
||
"github.com/kanisterio/kanister/pkg/kopia/cli" | ||
) | ||
|
||
// ID creates a new ID argument. | ||
func ID(id string) command.Applier { | ||
if id == "" { | ||
return command.NewErrorArgument(cli.ErrInvalidID) | ||
} | ||
return command.NewArgument(id) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package args_test | ||
|
||
import ( | ||
"testing" | ||
|
||
"github.com/kanisterio/safecli/test" | ||
"gopkg.in/check.v1" | ||
|
||
"github.com/kanisterio/kanister/pkg/kopia/cli" | ||
"github.com/kanisterio/kanister/pkg/kopia/cli/internal/args" | ||
) | ||
|
||
func TestArgs(t *testing.T) { check.TestingT(t) } | ||
|
||
var _ = check.Suite(&test.ArgumentSuite{Cmd: "cmd", Arguments: []test.ArgumentTest{ | ||
{ | ||
Name: "Invalid ID", | ||
Argument: args.ID(""), | ||
ExpectedErr: cli.ErrInvalidID, | ||
}, | ||
{ | ||
Name: "ID", | ||
Argument: args.ID("id12345"), | ||
ExpectedCLI: []string{"cmd", "id12345"}, | ||
}, | ||
}}) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
// Copyright 2024 The Kanister Authors. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package internal | ||
|
||
import ( | ||
"strconv" | ||
"strings" | ||
|
||
rs "github.com/kanisterio/kanister/pkg/secrets/repositoryserver" | ||
) | ||
|
||
// Location is a map of key-value pairs that represent different storage properties. | ||
type Location map[string][]byte | ||
|
||
// Type returns the location type. | ||
func (l Location) Type() rs.LocType { | ||
return rs.LocType(string(l[rs.TypeKey])) | ||
} | ||
|
||
// Region returns the location region. | ||
func (l Location) Region() string { | ||
return string(l[rs.RegionKey]) | ||
} | ||
|
||
// BucketName returns the location bucket name. | ||
func (l Location) BucketName() string { | ||
return string(l[rs.BucketKey]) | ||
} | ||
|
||
// Endpoint returns the location endpoint. | ||
func (l Location) Endpoint() string { | ||
return string(l[rs.EndpointKey]) | ||
} | ||
|
||
// Prefix returns the location prefix. | ||
func (l Location) Prefix() string { | ||
return string(l[rs.PrefixKey]) | ||
} | ||
|
||
// IsInsecureEndpoint returns true if the location endpoint is insecure/http. | ||
func (l Location) IsInsecureEndpoint() bool { | ||
return strings.HasPrefix(l.Endpoint(), "http:") | ||
} | ||
|
||
// HasSkipSSLVerify returns true if the location has skip SSL verification. | ||
func (l Location) HasSkipSSLVerify() bool { | ||
v, _ := strconv.ParseBool(string(l[rs.SkipSSLVerifyKey])) | ||
return v | ||
} | ||
|
||
// IsPointInTimeSupported returns true if the location supports point-in-time recovery. | ||
// Currently, only S3 and Azure support point-in-time recovery. | ||
func (l Location) IsPointInTypeSupported() bool { | ||
switch l.Type() { | ||
case rs.LocTypeAzure, rs.LocTypeS3, rs.LocTypes3Compliant: | ||
return true | ||
default: | ||
return false | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we handle error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've implemented the same logic as it is in the current implementation:
strconv.ParseBool
returnsfalse
in the case of error.