Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Kopia common args and opts #2654

Merged
merged 29 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
680e782
Add safecli dependency
plar Feb 3, 2024
72011e7
add new flag implementations based on the safecli package for the Kop…
plar Feb 3, 2024
0f635ad
apply go fmt
plar Feb 4, 2024
d0a6dd1
Add common Kopia args and flags
plar Feb 6, 2024
9850f4a
Fix Apply and test.Suit
plar Feb 12, 2024
246e1c1
Remove variadic args for Common and Cache flags
plar Feb 12, 2024
419652c
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar Feb 12, 2024
24707e5
pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now
plar Feb 17, 2024
c029fc1
Merge branch 'master' into kopia-cli-core-infra
plar Feb 17, 2024
cff7220
Re-implement Kopia common args and opts; Sync with the master
plar Feb 17, 2024
550d124
Add pkg/kopia/cli package
plar Feb 17, 2024
fc918a0
go mod tidy
plar Feb 17, 2024
9530c91
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar Feb 17, 2024
2c6cb6e
Convert common flags from vars to funcs
plar Feb 26, 2024
1a3ee2d
Add safecli dependency
plar Feb 3, 2024
386e7e9
add new flag implementations based on the safecli package for the Kop…
plar Feb 3, 2024
b815f63
apply go fmt
plar Feb 4, 2024
bcafabb
Fix Apply and test.Suit
plar Feb 12, 2024
6d61bb4
pkg/kopia/cli/internal/flag is implemented in the safecli@v0.0.4 now
plar Feb 17, 2024
e6ddb8f
Add pkg/kopia/cli package
plar Feb 17, 2024
4e7ffd7
go mod tidy
plar Feb 17, 2024
f16aea7
Update safecli to v0.0.5
plar Feb 27, 2024
dcd6425
Update safecli to v0.0.6
plar Feb 28, 2024
d3dfb31
Sync with upstream
plar Feb 28, 2024
b093421
Fix tests
plar Feb 28, 2024
44c133e
Fix formatting
plar Feb 29, 2024
1897a1e
Merge branch 'kopia-cli-core-infra' into kopia-cli-common-flags
plar Feb 29, 2024
33b1342
organize imports
plar Feb 29, 2024
8baab6d
Merge branch 'master' into kopia-cli-common-flags
pavannd1 Mar 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,8 @@ require (
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
)

require github.com/kanisterio/safecli v0.0.6

require (
github.com/Azure/go-autorest/autorest v0.11.27 // indirect
github.com/Azure/go-autorest/autorest/adal v0.9.18 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,8 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/kanisterio/safecli v0.0.6 h1:Mq99jK7A/SBiHKZalUIlsk4qaSVZJNbBWb8rjaJv/Jk=
github.com/kanisterio/safecli v0.0.6/go.mod h1:KBraqj8mdv2cwAr9wecknGUb8jztTzUik0r7uE6yRA8=
github.com/kastenhq/check v0.0.0-20180626002341-0264cfcea734 h1:qulsCaCv+O2y9/sQ9nd5KChnAgFOWakTHQ9ZADjs6DQ=
github.com/kastenhq/check v0.0.0-20180626002341-0264cfcea734/go.mod h1:rdqSnvOJuKCPFW/h2rVLuXOAkRnHHdp9PZcKx4HCoDM=
github.com/kastenhq/stow v0.2.6-kasten.1.0.20231101232131-9321daa23aae h1:2cl4yuAJpdmLCx7G8eIsfNlQBLEfw0JDj6mTTyqc5qg=
Expand Down
26 changes: 26 additions & 0 deletions pkg/kopia/cli/args/cache_args.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package args

// Cache provides the cache arguments for Kopia CLI.
type Cache struct {
CacheDirectory string // the directory where cache is stored. Default is "/tmp/kopia-cache".
ContentCacheSizeLimitMB int // the maximum size of the content cache in MB.
MetadataCacheSizeLimitMB int // the maximum size of the metadata cache in MB.

// unused?
ContentCacheSizeMB int // the size of the content cache in MB.
MetadataCacheSizeMB int // the size of the metadata cache in MB.
Comment on lines +23 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think we had switched to ContentCacheSizeLimitMB and MetadataCacheSizeLimitMB for setting hard limits. @kale-amruta can confirm.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes these are not used, we are using hard limits when providing cache options in K10 and kanister.

Comment on lines +24 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@plar If these are not referenced anywhere, can you remove them please?

}
23 changes: 23 additions & 0 deletions pkg/kopia/cli/args/common_args.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package args

// Common provides the common arguments for Kopia CLI.
type Common struct {
ConfigFilePath string // the path to the config file.
LogDirectory string // the directory where logs are stored.
LogLevel string // the level of logging. Default is "error".
RepoPassword string // the password for the repository.
}
21 changes: 21 additions & 0 deletions pkg/kopia/cli/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package cli

// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import (
_ "github.com/kanisterio/safecli"
)

// This package contains the implementation of the Kopia CLI using github.com/kanisterio/safecli.
25 changes: 25 additions & 0 deletions pkg/kopia/cli/errors.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cli

import (
"github.com/pkg/errors"
)

// Common errors
var (
// ErrInvalidID is returned when the ID is empty.
ErrInvalidID = errors.New("invalid ID")
)
28 changes: 28 additions & 0 deletions pkg/kopia/cli/internal/args/args.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package args

import (
"github.com/kanisterio/kanister/pkg/kopia/cli"
"github.com/kanisterio/safecli/command"
)

// ID creates a new ID argument.
func ID(id string) command.Applier {
if id == "" {
return command.NewErrorArgument(cli.ErrInvalidID)
}
return command.NewArgument(id)
}
39 changes: 39 additions & 0 deletions pkg/kopia/cli/internal/args/args_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package args_test

import (
"testing"

"github.com/kanisterio/kanister/pkg/kopia/cli"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal/args"
"github.com/kanisterio/safecli/test"
"gopkg.in/check.v1"
plar marked this conversation as resolved.
Show resolved Hide resolved
)

func TestArgs(t *testing.T) { check.TestingT(t) }

var _ = check.Suite(&test.ArgumentSuite{Cmd: "cmd", Arguments: []test.ArgumentTest{
{
Name: "Invalid ID",
Argument: args.ID(""),
ExpectedErr: cli.ErrInvalidID,
},
{
Name: "ID",
Argument: args.ID("id12345"),
ExpectedCLI: []string{"cmd", "id12345"},
},
}})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not test to run defined? Am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? We have 2 tests here:

func TestArgs(t *testing.T) { check.TestingT(t) }
// Go test/bridge func for `go test` and delegate the actual testing work to the `check` framework.

// register `test.ArgumentSuite` with the `check` framework.
var _ = check.Suite(&test.ArgumentSuite{Cmd: "cmd", Arguments: []test.ArgumentTest{
	{
		Name:        "Invalid ID",
		Argument:    args.ID(""),
		ExpectedErr: cli.ErrInvalidID,
	},
	{
		Name:        "ID",
		Argument:    args.ID("id12345"),
		ExpectedCLI: []string{"cmd", "id12345"},
	},
}})

We're using github.com/kanisterio/safecli/test.ArgumentSuite to run table tests.

68 changes: 68 additions & 0 deletions pkg/kopia/cli/internal/opts/cache_opts.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package opts

import (
"strconv"

"github.com/kanisterio/kanister/pkg/kopia/cli/args"
"github.com/kanisterio/safecli/command"
plar marked this conversation as resolved.
Show resolved Hide resolved
)

const (
defaultCacheDirectory = "/tmp/kopia-cache"
)

// CacheDirectory creates a new cache directory option with a given directory.
// If the directory is empty, the default cache directory is used.
func CacheDirectory(dir string) command.Applier {
if dir == "" {
dir = defaultCacheDirectory
}
return command.NewOptionWithArgument("--cache-directory", dir)
plar marked this conversation as resolved.
Show resolved Hide resolved
plar marked this conversation as resolved.
Show resolved Hide resolved
}

// ContentCacheSizeLimitMB creates a new content cache size option with a given size.
func ContentCacheSizeLimitMB(size int) command.Applier {
val := strconv.Itoa(size)
return command.NewOptionWithArgument("--content-cache-size-limit-mb", val)
}

// MetadataCacheSizeLimitMB creates a new metadata cache size option with a given size.
func MetadataCacheSizeLimitMB(size int) command.Applier {
val := strconv.Itoa(size)
return command.NewOptionWithArgument("--metadata-cache-size-limit-mb", val)
}

// ContentCacheSizeMB creates a new content cache size option with a given size.
func ContentCacheSizeMB(size int) command.Applier {
val := strconv.Itoa(size)
return command.NewOptionWithArgument("--content-cache-size-mb", val)
}

// MetadataCacheSizeMB creates a new metadata cache size option with a given size.
func MetadataCacheSizeMB(size int) command.Applier {
val := strconv.Itoa(size)
return command.NewOptionWithArgument("--metadata-cache-size-mb", val)
}

// Cache maps the Cache arguments to the CLI command options.
func Cache(args args.Cache) command.Applier {
return command.NewArguments(
CacheDirectory(args.CacheDirectory),
ContentCacheSizeLimitMB(args.ContentCacheSizeLimitMB),
MetadataCacheSizeLimitMB(args.MetadataCacheSizeLimitMB),
)
}
54 changes: 54 additions & 0 deletions pkg/kopia/cli/internal/opts/cache_opts_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package opts_test

import (
"testing"

"github.com/kanisterio/kanister/pkg/kopia/cli/args"
"github.com/kanisterio/kanister/pkg/kopia/cli/internal/opts"
"github.com/kanisterio/safecli/command"
"github.com/kanisterio/safecli/test"
"gopkg.in/check.v1"
plar marked this conversation as resolved.
Show resolved Hide resolved
)

func TestCacheOptions(t *testing.T) { check.TestingT(t) }

var _ = check.Suite(&test.ArgumentSuite{Cmd: "cmd", Arguments: []test.ArgumentTest{
{
Name: "CacheDirectory",
Argument: command.NewArguments(opts.CacheDirectory(""), opts.CacheDirectory("/path/to/cache")),
ExpectedCLI: []string{"cmd", "--cache-directory=/tmp/kopia-cache", "--cache-directory=/path/to/cache"},
},
{
Name: "ContentCacheSizeLimitMB",
Argument: command.NewArguments(opts.ContentCacheSizeLimitMB(0), opts.ContentCacheSizeLimitMB(1024)),
ExpectedCLI: []string{"cmd", "--content-cache-size-limit-mb=0", "--content-cache-size-limit-mb=1024"},
},
{
Name: "MetadataCacheSizeLimitMB",
Argument: command.NewArguments(opts.MetadataCacheSizeLimitMB(0), opts.MetadataCacheSizeLimitMB(1024)),
ExpectedCLI: []string{"cmd", "--metadata-cache-size-limit-mb=0", "--metadata-cache-size-limit-mb=1024"},
},
{
Name: "Cache",
Argument: opts.Cache(args.Cache{
CacheDirectory: "/path/to/cache",
ContentCacheSizeLimitMB: 1024,
MetadataCacheSizeLimitMB: 2048,
}),
ExpectedCLI: []string{"cmd", "--cache-directory=/path/to/cache", "--content-cache-size-limit-mb=1024", "--metadata-cache-size-limit-mb=2048"},
},
}})
70 changes: 70 additions & 0 deletions pkg/kopia/cli/internal/opts/common_opts.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Copyright 2024 The Kanister Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package opts

import (
"github.com/kanisterio/kanister/pkg/kopia/cli/args"
"github.com/kanisterio/safecli/command"
plar marked this conversation as resolved.
Show resolved Hide resolved
)

const (
defaultLogLevel = "error"
)

// LogDirectory creates a new log directory option with a given directory.
// if the directory is empty, the log directory option is not set.
func LogDirectory(dir string) command.Applier {
if dir == "" {
return command.NewNoopArgument()
}
return command.NewOptionWithArgument("--log-dir", dir)
}

// LogLevel creates a new log level flag with a given level.
// If the level is empty, the default log level is used.
func LogLevel(level string) command.Applier {
if level == "" {
level = defaultLogLevel
}
return command.NewOptionWithArgument("--log-level", level)
}

// ConfigFilePath creates a new config file path option with a given path.
// If the path is empty, the config file path option is not set.
func ConfigFilePath(path string) command.Applier {
if path == "" {
return command.NewNoopArgument()
}
return command.NewOptionWithArgument("--config-file", path)
}

// RepoPassword creates a new repository password option with a given password.
// If the password is empty, the repository password option is not set.
func RepoPassword(password string) command.Applier {
if password == "" {
return command.NewNoopArgument()
}
return command.NewOptionWithRedactedArgument("--password", password)
}

// Common maps the common arguments to the CLI command options.
func Common(args args.Common) command.Applier {
return command.NewArguments(
ConfigFilePath(args.ConfigFilePath),
LogDirectory(args.LogDirectory),
LogLevel(args.LogLevel),
RepoPassword(args.RepoPassword),
)
}
Loading