From 60a848b461f403a9223b348bfdbae08467d305f0 Mon Sep 17 00:00:00 2001 From: Louis DeJardin Date: Wed, 15 Dec 2021 20:45:06 -0800 Subject: [PATCH] Oci support rebased (#2621) * design document template (#2563) * updated maintainers * update to the front page * installation paragraph * design doc template * additional process detail * slight updates * update kyaml and cli-utils to latest (#2523) * Do not error while adding merge comment (#2564) * Remove check if repo is checked into git for update (#2569) * Remove check if repo is checked into git for update * Update tests * Update should accept empty path with version (#2572) * Update roadmap for Q4 (#2571) * Update roadmap for Q4 * Suggested changes * Update roadmap (#2574) * Update roadmap * Suggested changes * Update with suggested format * Organize better * update installation doc and Homebrew to use 1.0.0-beta.8 (#2578) * Update kpt book with depends-on annotation (#2565) * Update kpt book with depends-on annotation * Addressed comments * Add snippet for resource dependencies in chapter overview * fix: use KptFileKind variable instead of KptFileName for GVR (#2580) When specifying the Kptfile resource kind, some code were using the KptFileName variable instead of KptFileKind. Both variables currently contain the same value: "Kptfile", but this will be an issue in the future should the KptFileName and KptFileKind variables have different values. * Disable unwrapping to read fnconfig (#2582) * Disable wrapping to read fnconfig * Update index annotation * Update licenses (#2583) - Use go install with explicit tag to avoid modifying go mod - Add licenses to yaml files (new addlicense feature) - Add updated license-check to make all * Revert "update installation doc and Homebrew to use 1.0.0-beta.8 (#2578)" (#2584) This reverts commit 98f029925dc2faaf43115c531e9a318ffc26b5f2. * add tests to exercise modify path annotation (#2586) * Update design doc review process (#2594) * Update beta 9 (#2597) * Add missing json annotation to KptFile (#2598) * Implement metrics annotation (#2588) * Implement metrics annotation * Changes * Suggested changes * Suggested changes * Attributor * Design proposal: kpt live -- Inventory to ResourceGroup (#2576) * DD for kpt live apply from STDIN * resolve comments * Update doc with rg fn interaction Co-authored-by: Phani Teja Marupaka * Add cnrm annotation to merge3 (#2605) * docs: Specify origin in git push command in 3.8 (#2608) Issues: GoogleContainerTools/kpt#2596 * upgrade to kyaml 0.13.0+ (#2603) * upgrade to kyaml 0.13.0+ * use the latest kyaml * --image flag typo fix (#2601) Fix typo in `kpt fn eval`'s --image flag description * Simplify and speed up "kpt live" e2e tests (#2613) * Simplify and speed up live e2e tests * Addressed comments * Adding OCI to Kptfile schema Adding OCI to error utils Adding OCI argument parsing to kpt pkg get Adding get/fetch validation Adding pull and untar to support kpt get Making change in md file go fmt and fixing tests * Adds `oci://` to disambiguate `kpt pkg get` argument Factoring upstream fetcher out of switch cases go fmt fixes Ignore scopelint in table tests Adding support for `kpt pkg update` Fix fmt and tests Path must be added to returned absPath Removing git repo error message check * this startup check probably doesn't make sense any more? the pkg update logic doesn't rely on the state of the local git repo to merge Adding OCI support to kpt pkg diff * Mainly involves using upstream.Fetcher methods Adding prototype kpt pkg pull command Based on design and latest conversations Adding kpt pkg push command Minor fixes * Fixing a direct reference to Git field * Adding output to pkg push command Rename upstream pkg to remote Refactoring remote names * Rename Fether to Upstream * Split Upstream and Origin implementations Removing redundant Origin suffix from names Adding compile-time interface assertion go fix formatting Adding missing err checks Fixing package reference case Fixing case sensitive package * Fixing go.sum after rebase * Adjusting merge of update * Adding support for directory in oci subpackage upstream * Fix formatting * Remove check if repo is checked into git for update (#2569) * Remove check if repo is checked into git for update * Update tests * Updating go.sum * Adding previous round of PR feedback * Updating scheme * Adding TODO(oci-support) marker to kpt site content Co-authored-by: Mike Borozdin Co-authored-by: Natasha Sarkar Co-authored-by: phani Co-authored-by: Mengqi Yu Co-authored-by: Morten Torkildsen Co-authored-by: Ramon Quitales Co-authored-by: Karl Isenberg Co-authored-by: Martin Maly Co-authored-by: Yuwen Ma Co-authored-by: Phani Teja Marupaka Co-authored-by: sdowell Co-authored-by: Ben Congdon Co-authored-by: Louis DeJardin --- commands/pkgcmd.go | 5 +- docs/design-docs/00-template.md | 1 + .../01-live_Inventory-to-ResourceGroup.md | 297 +++++++++++ .../live-apply/crd-and-cr/config.yaml | 1 + .../json-output/resources/fourth.yaml | 37 ++ .../json-output/resources/third.yaml | 36 ++ go.mod | 2 + go.sum | 205 +++++++- internal/cmddestroy/cmddestroy.go | 8 + internal/cmdget/cmdget.go | 22 +- internal/cmdget/cmdget_test.go | 1 + internal/cmdpull/cmdpull.go | 108 ++++ internal/cmdpush/cmdpush.go | 161 ++++++ internal/cmdrender/executor.go | 3 + internal/docs/generated/pkgdocs/docs.go | 160 +++++- internal/errors/errors.go | 3 + internal/testutil/setup_manager.go | 9 +- internal/util/diff/diff.go | 114 ++--- internal/util/fetch/fetch.go | 222 ++------- internal/util/get/example_test.go | 25 +- internal/util/get/get.go | 50 +- internal/util/get/get_test.go | 71 +-- internal/util/parse/parse.go | 123 ++++- internal/util/pull/pull.go | 109 +++++ internal/util/push/push.go | 159 ++++++ internal/util/remote/git.go | 402 +++++++++++++++ internal/util/remote/git_test.go | 76 +++ internal/util/remote/oci.go | 461 ++++++++++++++++++ internal/util/remote/origin.go | 63 +++ internal/util/remote/upstream.go | 79 +++ internal/util/remote/upstream_test.go | 134 +++++ internal/util/update/common.go | 2 +- internal/util/update/update.go | 89 ++-- pkg/api/kptfile/v1/types.go | 47 ++ pkg/kptfile/kptfileutil/util.go | 46 +- pkg/test/live/runner.go | 9 +- site/installation/README.md | 6 + site/reference/cli/pkg/get/README.md | 9 +- site/reference/cli/pkg/pull/README.md | 126 +++++ site/reference/cli/pkg/push/README.md | 126 +++++ site/reference/schema/kptfile/kptfile.json | 64 +++ site/reference/schema/kptfile/kptfile.yaml | 51 ++ site/sidebar.md | 3 + 43 files changed, 3301 insertions(+), 424 deletions(-) create mode 100644 docs/design-docs/01-live_Inventory-to-ResourceGroup.md create mode 100644 e2e/testdata/live-apply/json-output/resources/fourth.yaml create mode 100644 e2e/testdata/live-apply/json-output/resources/third.yaml create mode 100644 internal/cmdpull/cmdpull.go create mode 100644 internal/cmdpush/cmdpush.go create mode 100644 internal/util/pull/pull.go create mode 100644 internal/util/push/push.go create mode 100644 internal/util/remote/git.go create mode 100644 internal/util/remote/git_test.go create mode 100644 internal/util/remote/oci.go create mode 100644 internal/util/remote/origin.go create mode 100644 internal/util/remote/upstream.go create mode 100644 internal/util/remote/upstream_test.go create mode 100644 site/reference/cli/pkg/pull/README.md create mode 100644 site/reference/cli/pkg/push/README.md diff --git a/commands/pkgcmd.go b/commands/pkgcmd.go index eb6cd75b83..a2b851a2eb 100644 --- a/commands/pkgcmd.go +++ b/commands/pkgcmd.go @@ -20,6 +20,8 @@ import ( "github.com/GoogleContainerTools/kpt/internal/cmddiff" "github.com/GoogleContainerTools/kpt/internal/cmdget" "github.com/GoogleContainerTools/kpt/internal/cmdinit" + "github.com/GoogleContainerTools/kpt/internal/cmdpull" + "github.com/GoogleContainerTools/kpt/internal/cmdpush" "github.com/GoogleContainerTools/kpt/internal/cmdupdate" "github.com/GoogleContainerTools/kpt/internal/docs/generated/pkgdocs" "github.com/GoogleContainerTools/kpt/thirdparty/cmdconfig/commands/cmdtree" @@ -47,7 +49,8 @@ func GetPkgCommand(ctx context.Context, name string) *cobra.Command { pkg.AddCommand( cmdget.NewCommand(ctx, name), cmdinit.NewCommand(ctx, name), cmdupdate.NewCommand(ctx, name), cmddiff.NewCommand(ctx, name), - cmdtree.NewCommand(ctx, name), + cmdtree.NewCommand(ctx, name), cmdpull.NewCommand(ctx, name), + cmdpush.NewCommand(ctx, name), ) return pkg } diff --git a/docs/design-docs/00-template.md b/docs/design-docs/00-template.md index 86fca78778..3f1745378b 100644 --- a/docs/design-docs/00-template.md +++ b/docs/design-docs/00-template.md @@ -79,6 +79,7 @@ Please name your documents with the design document number. Pick n+1 from what' renaming might need to happen if multiple design docs get accepted simultaneously. Please keep the name of the design doc in lower case with dashes in between words. +======= Reviewers are auto-assigned to review the PR. Optionally, doc owner can mention any specific reviewers on the PR description. The turn around time for each review cycle on the PR from kpt maintainers is 1-2 days. After maintainers add comments to the PR, diff --git a/docs/design-docs/01-live_Inventory-to-ResourceGroup.md b/docs/design-docs/01-live_Inventory-to-ResourceGroup.md new file mode 100644 index 0000000000..ce2e50f86f --- /dev/null +++ b/docs/design-docs/01-live_Inventory-to-ResourceGroup.md @@ -0,0 +1,297 @@ +# Title + +* Author(s): Yuwen Ma +* Contributor(s): + - Sean Sullivan + - Sunil Arora + - Mortent Torkildsen + - Mengqi Yu + - Natasha Sarkar + +* Approver: Sunil Arora + +## Why + +This design is aimed at making `kpt live` be hydration-tool-neutral so that `kpt live` +applicable use cases can be expanded to `kustomize build`, `helm templates` and raw +kubernetes manifests. + +### Non-goal +- This doc is not aimed at designing a unified hydration between kpt and kustomize +- This doc is not aimed at changing the kpt in-place hydration models. + +## Design + +We propose to make `kpt live` support reading configurations from standard input, +detach the Inventory from Kptfile and simplify the Inventory-ResourceGroup mechanism +to use ResourceGroup only. + +### Read standard input resources + +We will change the existing `kpt live apply` from STDIN to not require the existence of Kptfile ( +to be more specific, the `inventory` file from Kptfile). As long as the STDIN contains +one and only one valid ResourceGroup, `kpt live apply` should be able to create/match the +ResourceGroup in cluster. + +### Initialize a `ResourceGroup` object + +`kpt live init` will create a ResourceGroup CR in resourcegroup.yaml. + +By default, the ResoureGroup will have name and namespace assigned as below. And users can +override via existing flags "--name" and "--namespace". + +- metadata.name: A client-provided valid RFC 1123 DNS subdomain name. Default value has prefix “inventory-” with + 8 digit numbers. E.g. "inventory-02014045" +- metadata.namespace: a valid namespace. default to value "default" + +If users want to reuse an existing inventory (from Kptfile) or ResourceGroup (which has been deployed to the cluster), +they shall provide the value of the inventory's inventory-id or the ResourceGroup's "metadata.labels[0].cli-utils.sigs.k8s.io/inventory-id" +via "--inventory-id" flag. + +### Convert Inventory to ResourceGroup + +The client-side Inventory is considered as the identifier of the cluster-side +ResourceGroup CR. Right now, the Inventory is stored in the Kptfile to guarantee +the ResourceGroup is unique. This requires Kptfile to exist to use `kpt live apply`. + +To split the Inventory from Kptfile to resourcegroup.yaml and convert the Inventory to +ResourceGroup, `kpt live migrate` should be extended to map the inventory.name, +inventory.namespace, inventory.inventoryID to ResourceGroup CR metadata.name, +metadata.namespace, metadata.labels.cli-utils.sigs.k8s.io/inventory-id correspondingly. + +Inventory in Kptfile +```yaml +apiVersion: kpt.dev/v1 +kind: Kptfile +inventory: + name: + namespace: + inventoryID: +``` +resoucegroup.yaml +```yaml +apiVersion: kpt.dev/v1alpha1 +kind: ResourceGroup +metadata + name: + namespace: + labels: + cli-utils.sigs.k8s.io/inventory-id: +``` + +### Simplify the Inventory + +Current inventory contains inventory-id which is required to match the label `cli-utils.sigs.k8s.io/inventory-id`. + +For new users, they should no longer need to be exposed to the inventory-id, but kpt will +build one composed by `name-namespace` on the fly. + +For existing users, the inventory-id is still +required in the standalone ResourceGroup file to guarantee the adoption matches, unless they use "--inventory-policy=adopt" +to override the label. This flag is only required as a one-off via `kpt live apply -` to override the label to `name-namespace`. + +### ResourceGroup as a singleton object + +`kpt live apply [--rg-file] -` from STDIN accepts and only accepts a single +ResourceGroup, including the ResourceGroup provided by the flag. It detects +1. If more than one ResourceGroup is found, raise errors and display all the ResourceGroup objects. +2. If no ResourceGroup is found in STDIN and and Kptfile inventory does not exists, raise errors and suggest users to + run kpt live init +3. If no ResourceGroup is found in STDIN and Kptfile inventory exists, raise errors and + suggest users to run kpt live migrate + +### New flags + +#### `--rg-file` + +- description: The file path to the ResourceGroup CR, default to `resourcegroup.yaml` +- short form `--rg` +- This flag will be added to `kpt live init`, `kpt live migrate` and `kpt live apply` + +#### `--name` for inventory + +- description: The name for the ResourceGroup +- This flag will continue to be used by `kpt live init`. Rather than overriding the + inventory.name in Kptfile, it will override the default metadata.name in the standalone ResourceGroup file. + +#### `--namespace` for inventory + +- description: The namespace for the ResourceGroup +- This flag will continue to be used by `kpt live init`, Rather than overriding the + inventory.namespace in Kptfile, it will override the default metadata.namespace in the standalone ResourceGroup file. + +#### `--inventory-id` for inventory + +- description: Inventory identifier. This is used to detect overlap between + two sets of ResourceGroup managed resources that might use the same name and namespace. +- This flag will continue to be accepted by `kpt live init` for backward compatibility reasons. + If given, ResourceGroup will store the inventory-id value in "metadata.labels[0].cli-utils.sigs.k8s.io/inventory-id" + of the ResourceGroup. + If not given, the ResourceGroup labels will be empty and the value of "-" will be + used as the "cli-utils.sigs.k8s.io/inventory-id" label in `kpt live apply` from STD. + +### `resoucegroup.yaml` interaction with `kpt fn` commands + +All `kpt fn` commands should treat `resoucegroup.yaml` as a special resource +and not modify it in any scenario. It should not be considered as a meta resource and +`include-meta-resources` flag should not include the `ResourceGroup` resource in +the `ResourceList`. In the future, we can add a new flag `--include-rg` if there are +valid use-cases to modify or include `ResourceGroup` resource in `ResourceList`. + +## User Guide + +### To hydrate via kustomize and deploy via kpt + +#### Day 1 + +##### For new users to start from scratch (no Kptfile) +User can run `kpt live init [--rg-file=resourcegroup.yaml]` to create a +ResourceGroup object and store it in a resourcegroup.yaml file. +Users can customize the file path with the flag “--rg-file”. + +##### For existing kpt users to migrate from Kptfile +Users run `kpt live migrate [--rg-file=resourcegroup.yaml]` to convert the +Inventory object from Kptfile to a standalone resourcegroup.yaml file. +Users can customize the file path with the flag “--rg-file”. + +##### [optional]: Add shareable ResourceGroup to kustomize resources +If the ResourceGroup is expected to be shared in the Gitops workflow, users can add +the resourcegroup.yaml file path to the .resources field in kustomization.yaml. +This simplifies the Day N deployment by omitting the “–rg-file“ flag. + +#### Day N + +- Users can configure the hydration rules in kustomization.yaml +- Users can run `kustomize build | kpt live apply -` to hydrate and deploy + the kustomize-managed configurations. If resourcegroup.yaml is not added to + kustomize .resources field, users should provide the “–rg-file“ flag. + `kustomize build | kpt live apply –rg-file -` + +### To hydrate via helm and deploy via kpt + +#### Day 1 + +##### For new users to start from scratch (no Kptfile) +User can run `kpt live init --rg-file=/resourcegroup.yaml` to create +a ResourceGroup object and store it in the helm template . + +##### For existing kpt users to migrate from Kptfile +Users run `kpt live migrate --rg-file=/resourcegroup.yaml` to convert +the Inventory object from Kptfile to a standalone resourcegroup.yaml file. + +#### Day N + +Users can run `helm templates | kpt live apply -` to hydrate and deploy the +helm-managed resources. + +See kpt issue #2399 for expected Inventory usage in package publisher/consumer. + +## FAQ + +### Will Inventory be deprecated from Kptfile? + +Kpt still supports inventory in Kptfile and it is not required to migrate to the +standalone rg-path. In fact, users who do not use STDIN in `kpt live apply` +will still have inventory read from Kptfile by default unless the –inventory-path flag +is given. + +### Why not kpt live apply -k? + +We originally propose to use a single command kpt live apply -k (similar idea as +kubectl apply -k ) to kustomize-hydrate and kpt-deploy the resources, storing the +inventory in the annotation fields of the kustomization.yaml (See below). +The inventory can be auto added to kustomization.yaml by kpt live init +```yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +metadata: + annotations: + kpt-inventory-id: + kpt-inventory-name: + kpt-inventory-namespace: +``` +#### Pros + +- Extreme simple user steps. Kustomize users only need to install kpt and run kpt live + init. Then they can start using kpt live apply to deploy their kustomize resources. + No manual changes to any configurations. +- Easy to understand. The command is similar to kubectl apply -k which makes it easier + to be accepted and remembered. + +There are two main reasons to abandon this approach: +- Making Kpt a hydration-neutral deployment tool helps it better hook up with different + hydration tools. +- kpt is not planning to treat kustomize differently and thus it does not make sense to + provide kustomize a shortcut. + +## Alternatives Considered + +### Use Inventory metadata to store ResourceGroup info + +Store the Kptfile inventory to a standalone Inventory object. Since the inventory is +the metadata of the cluster side ResourceGroup, hiding the ResourceGroup behind +inventory provides users simpler syntax and mitigates the probability of overriding +the ResourceGroup unintentionally (e.g. via kubectl apply). + +inventory.yaml +```yaml +apiVersion: kpt.dev/v1 +kind: Inventory +metadata: + name: + namespace: +inventoryID: +``` + +#### Pros + +- Simpler resource syntax +- Mitigate unintentional resourceGroup override. + +#### Cons +- The client-side inventory requires users to manage. + This new resource increases the overall complexity of the kpt actuation config. + We frequently receive user confusions about the difference between ResourceGroup + and Inventory and why they cannot configure inventory like other kubernetes resource + (adoption mismatch errors). + +### Use Inventory spec to store ResourceGroup info + + +inventory.yaml +```yaml +apiVersion: kpt.dev/v1 +kind: Inventory +spec: + resourceGroup: + name: + Namespace: + labels: + cli-utils.sigs.k8s.io/inventory-id: +``` +#### Pros + +- Obey the kubernetes convention and make it clearer that users are expected to provide the resourceGroup kind with name, namespace and labels. +- Mitigate unintentional resourceGroup override. + +#### Cons +More complex syntax than ResourceGroup or Inventory metadata solution + +### Add a controller for ResourceGroup pruning; No client-side Inventory + +Inventory is a [cli-utils object](https://github.com/kubernetes-sigs/cli-utils/blob/52b000849deb2b233f0a58e9be16ca2725a4c9cf/pkg/inventory/inventory.go#L30) used to store the [ResourceGroup metadata](https://docs.google.com/document/d/1x_02xGKZH6YGuc3K-KNeY4KmADPAwzrlQKkvTNXYekc/edit#heading=h.265kfx5ku27). +It is stored in Kptfile .Inventory and required by `kpt live apply`. In lieu of a user configuration, it is more of a data for kpt to identify the corresponding ResourceGroup for pruning. This adds non-trivial overheads for users to understand, maintain and retrieve the inventory. + +The alternative proposal is to remove the inventory but use a new controller to prune the resource. +The controller will be installed together with ResourceGroup CRD (in each `kpt live apply`). +It will find the previous ResourceGroup CR from current resources, three-way merging and deploying +the new resources and update the ResourceGroup. + +#### Pros +- Improve the user experience on using kpt live and minimize the confusions around inventory and ResourceGroup +- Eliminate the pain points on retrieving the missing Inventory (list all ResourceGroup, check the .spec.resources and try one by one until not encounter the “missing adoption” error). + +#### Cons +The controller needs multiple requests to find out the previous ResourceGroup CR. This causes heavy traffic in a production cluster. +The controller cannot 100% guarantee the resourceGroup reverse lookup are accurate. +Since the pruning happens on the cluster side, kpt can no longer provide real-time status updates. diff --git a/e2e/testdata/live-apply/crd-and-cr/config.yaml b/e2e/testdata/live-apply/crd-and-cr/config.yaml index a0e5ef4a3b..ba2d81de11 100644 --- a/e2e/testdata/live-apply/crd-and-cr/config.yaml +++ b/e2e/testdata/live-apply/crd-and-cr/config.yaml @@ -26,6 +26,7 @@ stdOut: | custom.kpt.dev/cr reconcile pending custom.kpt.dev/cr reconciled 2 resource(s) reconciled, 0 skipped, 0 failed to reconcile, 0 timed out + inventory: - group: apiextensions.k8s.io kind: CustomResourceDefinition diff --git a/e2e/testdata/live-apply/json-output/resources/fourth.yaml b/e2e/testdata/live-apply/json-output/resources/fourth.yaml new file mode 100644 index 0000000000..48e7567715 --- /dev/null +++ b/e2e/testdata/live-apply/json-output/resources/fourth.yaml @@ -0,0 +1,37 @@ +# Copyright 2021 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: fourth-nginx + namespace: json-output + annotations: + config.k8s.io/owning-inventory: json-output + config.kubernetes.io/depends-on: apps/namespaces/json-output/Deployment/third-nginx +spec: + replicas: 1 + selector: + matchLabels: + app: fourth-nginx + template: + metadata: + labels: + app: fourth-nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 \ No newline at end of file diff --git a/e2e/testdata/live-apply/json-output/resources/third.yaml b/e2e/testdata/live-apply/json-output/resources/third.yaml new file mode 100644 index 0000000000..123b05bf0f --- /dev/null +++ b/e2e/testdata/live-apply/json-output/resources/third.yaml @@ -0,0 +1,36 @@ +# Copyright 2021 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: third-nginx + namespace: json-output + annotations: + config.k8s.io/owning-inventory: json-output +spec: + replicas: 1 + selector: + matchLabels: + app: third-nginx + template: + metadata: + labels: + app: third-nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 \ No newline at end of file diff --git a/go.mod b/go.mod index 7fc2f95625..71938bb659 100644 --- a/go.mod +++ b/go.mod @@ -3,8 +3,10 @@ module github.com/GoogleContainerTools/kpt go 1.16 require ( + github.com/Masterminds/semver v1.5.0 github.com/cpuguy83/go-md2man/v2 v2.0.0 github.com/go-errors/errors v1.4.0 + github.com/google/go-containerregistry v0.6.0 github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 github.com/igorsobreira/titlecase v0.0.0-20140109233139-4156b5b858ac github.com/otiai10/copy v1.6.0 diff --git a/go.sum b/go.sum index c2f2506236..eb823ac528 100644 --- a/go.sum +++ b/go.sum @@ -1,3 +1,4 @@ +bazil.org/fuse v0.0.0-20160811212531-371fbbdaa898/go.mod h1:Xbm+BRKSBEpa4q4hTSxohYNQpsxXPbPry4JJWOB3LB8= cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= @@ -49,6 +50,7 @@ github.com/Azure/go-autorest/autorest/adal v0.9.13 h1:Mp5hbtOePIzM8pJVRa3YLrWWmZ github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M= github.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw= github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= +github.com/Azure/go-autorest/autorest/mocks v0.4.0/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= github.com/Azure/go-autorest/autorest/mocks v0.4.1 h1:K0laFcLE6VLTOwNgSxaGbUcLPuGXlNkbVvq4cW4nIHk= github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= github.com/Azure/go-autorest/logger v0.2.1 h1:IG7i4p/mDa2Ce4TRyAO8IHnVhAVF3RFU+ZtXWSmf4Tg= @@ -59,6 +61,26 @@ github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03 github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd h1:sjQovDkwrZp8u+gxLtPgKGjk5hCxuy2hrRejBTA9xFU= github.com/MakeNowJust/heredoc v0.0.0-20170808103936-bb23615498cd/go.mod h1:64YHyfSL2R96J44Nlwm39UHepQbyR5q10x7iYa1ks2E= +github.com/Masterminds/semver v1.5.0 h1:H65muMkzWKEuNDnfl9d70GUjFniHKHRbFPGBuZ3QEww= +github.com/Masterminds/semver v1.5.0/go.mod h1:MB6lktGJrhw8PrUyiEoblNEGEQ+RzHPF078ddwwvV3Y= +github.com/Microsoft/go-winio v0.4.11/go.mod h1:VhR8bwka0BXejwEJY73c50VrPtXAaKcyvVC4A4RozmA= +github.com/Microsoft/go-winio v0.4.14/go.mod h1:qXqCSQ3Xa7+6tgxaGTIe4Kpcdsi+P8jBhyzoq1bpyYA= +github.com/Microsoft/go-winio v0.4.15-0.20190919025122-fc70bd9a86b5/go.mod h1:tTuCMEN+UleMWgg9dVx4Hu52b1bJo+59jBh3ajtinzw= +github.com/Microsoft/go-winio v0.4.16-0.20201130162521-d1ffc52c7331/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0= +github.com/Microsoft/go-winio v0.4.16/go.mod h1:XB6nPKklQyQ7GC9LdcBEcBl8PF76WugXOPRXwdLnMv0= +github.com/Microsoft/go-winio v0.4.17-0.20210211115548-6eac466e5fa3/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84= +github.com/Microsoft/go-winio v0.4.17-0.20210324224401-5516f17a5958/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84= +github.com/Microsoft/go-winio v0.4.17/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84= +github.com/Microsoft/go-winio v0.5.0/go.mod h1:JPGBdM1cNvN/6ISo+n8V5iA4v8pBzdOpzfwIujj1a84= +github.com/Microsoft/hcsshim v0.8.6/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg= +github.com/Microsoft/hcsshim v0.8.7-0.20190325164909-8abdbb8205e4/go.mod h1:Op3hHsoHPAvb6lceZHDtd9OkTew38wNoXnJs8iY7rUg= +github.com/Microsoft/hcsshim v0.8.7/go.mod h1:OHd7sQqRFrYd3RmSgbgji+ctCwkbq2wbEYNSzOYtcBQ= +github.com/Microsoft/hcsshim v0.8.9/go.mod h1:5692vkUqntj1idxauYlpoINNKeqCiG6Sg38RRsjT5y8= +github.com/Microsoft/hcsshim v0.8.14/go.mod h1:NtVKoYxQuTLx6gEq0L96c9Ju4JbRJ4nY2ow3VK6a9Lg= +github.com/Microsoft/hcsshim v0.8.15/go.mod h1:x38A4YbHbdxJtc0sF6oIz+RG0npwSCAvn69iY6URG00= +github.com/Microsoft/hcsshim v0.8.16/go.mod h1:o5/SZqmR7x9JNKsW3pu+nqHm0MF8vbA+VxGOoXdC600= +github.com/Microsoft/hcsshim/test v0.0.0-20201218223536-d3e5debf77da/go.mod h1:5hlzMzRKMLyo42nCZ9oml8AdTlq/0cvIaBv6tK1RehU= +github.com/Microsoft/hcsshim/test v0.0.0-20210227013316-43a75bb4edd3/go.mod h1:mw7qgWloBUl75W/gVH3cQszUg1+gUITj7D6NY7ywVnY= github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= @@ -85,6 +107,7 @@ github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+Ce github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= +github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA= github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM= github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= @@ -97,6 +120,7 @@ github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+ github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5 h1:7aWHqerlJ41y6FOsEUvknqgXnGmJyJSbjhAWq5pO4F8= github.com/chai2010/gettext-go v0.0.0-20160711120539-c6fed771bfd5/go.mod h1:/iP1qXHoty45bqomnu2LM+VVyAEdWN+vtSHGlQgyxbw= +github.com/checkpoint-restore/go-criu/v4 v4.1.0/go.mod h1:xUQBLp4RLc5zJtWY++yjOoMoB5lihDt7fai+75m+rGw= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= @@ -110,6 +134,8 @@ github.com/cockroachdb/logtags v0.0.0-20190617123548-eb05cc24525f/go.mod h1:i/u9 github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= +github.com/coreos/go-iptables v0.4.5/go.mod h1:/mVI274lEDI2ns62jHCDnCyBF9Iwsmekav8Dbxlm1MU= +github.com/coreos/go-iptables v0.5.0/go.mod h1:/mVI274lEDI2ns62jHCDnCyBF9Iwsmekav8Dbxlm1MU= github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= @@ -121,12 +147,25 @@ github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsr github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.11 h1:07n33Z8lZxZ2qwegKbObQohDhXDQxiMMz1NOUGYlesw= github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/cyphar/filepath-securejoin v0.2.2/go.mod h1:FpkQEhXnPnOthhzymB7CGsFk2G9VLXONKD9G7QGMM+4= +github.com/d2g/dhcp4 v0.0.0-20170904100407-a1d1b6c41b1c/go.mod h1:Ct2BUK8SB0YC1SMSibvLzxjeJLnrYEVLULFNiHY9YfQ= +github.com/d2g/dhcp4client v1.0.0/go.mod h1:j0hNfjhrt2SxUOw55nL0ATM/z4Yt3t2Kd1mW34z5W5s= +github.com/d2g/dhcp4server v0.0.0-20181031114812-7d4a0a7f59a5/go.mod h1:Eo87+Kg/IX2hfWJfwxMzLyuSZyxSoAug2nGa1G2QAi8= +github.com/d2g/hardwareaddr v0.0.0-20190221164911-e7d9fbe030e4/go.mod h1:bMl4RjIciD2oAxI7DmWRx6gbeqrkoLqv3MV0vzNad+I= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/daviddengcn/go-colortext v0.0.0-20160507010035-511bcaf42ccd/go.mod h1:dv4zxwHi5C/8AeI+4gX4dCWOIvNi7I6JCSX0HvlKPgE= +github.com/denverdino/aliyungo v0.0.0-20190125010748-a747050bb1ba/go.mod h1:dV8lFg6daOBZbT6/BDGIz6Y3WFGn8juu6G+CQ6LHtl0= +github.com/dgrijalva/jwt-go v0.0.0-20170104182250-a601269ab70c/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= +github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E= +github.com/docker/cli v20.10.7+incompatible h1:pv/3NqibQKphWZiAskMzdz8w0PRbtTaEB+f6NwdU7Is= +github.com/docker/cli v20.10.7+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= +github.com/docker/distribution v0.0.0-20190905152932-14b96e55d84c/go.mod h1:0+TTO4EOBfRPhZXAeF1Vu+W3hHZ8eLp8PgKVZlcvtFY= +github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= +github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug= github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= @@ -165,6 +204,7 @@ github.com/go-errors/errors v1.4.0/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3Bop github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-ini/ini v1.25.4/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8= github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= @@ -226,6 +266,11 @@ github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaS github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +<<<<<<< HEAD +======= +github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA= +github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) github.com/golangplus/testing v0.0.0-20180327235837-af21d9c3145e/go.mod h1:0AA//k/eakGydO4jKRoRL2j92ZKSzTgj9tclaCrvXHk= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= @@ -244,6 +289,8 @@ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6 h1:BKbKCqvP6I+rmFHt06ZmyQtvB8xAkWdhFyr0ZUNZcxQ= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-containerregistry v0.6.0 h1:niQ+8XD//kKgArIFwDVBXsWVWbde16LPdHMyNwSC8h4= +github.com/google/go-containerregistry v0.6.0/go.mod h1:euCCtNbZ6tKqi1E72vwDj2xZcN5ttKpZLfa/wSo5iLw= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= @@ -309,6 +356,9 @@ github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1: github.com/igorsobreira/titlecase v0.0.0-20140109233139-4156b5b858ac h1:AfRcPFr4uK97K6YaYi9XmNY/cTPF+cLspaXocdqkdCQ= github.com/igorsobreira/titlecase v0.0.0-20140109233139-4156b5b858ac/go.mod h1:KOzUkqpWM2xArNm82cehGc5PBFYV1Qadzzt81aJi7F0= github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= +github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= +github.com/imdario/mergo v0.3.10/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= +github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= @@ -333,12 +383,19 @@ github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvW github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/klauspost/compress v1.11.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= +github.com/klauspost/compress v1.11.13/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= +github.com/klauspost/compress v1.12.3/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= +github.com/klauspost/compress v1.13.0 h1:2T7tUoQrQT+fQWdaY5rjWztFGAFwbGD04iPJg90ZiOs= +github.com/klauspost/compress v1.13.0/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= @@ -357,10 +414,13 @@ github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJ github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI= +github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= +github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= +github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4= github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= @@ -385,6 +445,8 @@ github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9 github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 h1:n6/2gBQ3RWajuToeY6ZtZTIKv2v7ThUy5KKusIT0yc0= github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= +github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc= +github.com/mrunalp/fileutils v0.5.0/go.mod h1:M1WthSahJixYnrXQl/DFQuteStB1weuxD2QJNHXfbSQ= github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= @@ -397,6 +459,7 @@ github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= github.com/olekukonko/tablewriter v0.0.4/go.mod h1:zq6QwlOf5SlnkVbMSr5EoBv3636FWnp+qbPhuoO21uA= +github.com/onsi/ginkgo v0.0.0-20151202141238-7f8ab55aaf3b/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= @@ -426,6 +489,7 @@ github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR github.com/philopon/go-toposort v0.0.0-20170620085441-9be86dbd762f h1:WyCn68lTiytVSkk7W1K9nBiSGTSRlUOdyTnSjwrIlok= github.com/philopon/go-toposort v0.0.0-20170620085441-9be86dbd762f/go.mod h1:/iRjX3DdSK956SzsUdV55J+wIsQ+2IBWmBrB4RvZfk4= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.8.1-0.20171018195549-f15c970de5b7/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= @@ -434,6 +498,7 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA= +github.com/prometheus/client_golang v0.0.0-20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= @@ -445,6 +510,7 @@ github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1: github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/common v0.0.0-20180110214958-89604d197083/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= @@ -453,6 +519,7 @@ github.com/prometheus/common v0.26.0 h1:iMAkS2TDoNWnKM+Kopnx/8tnEStIfpYA0ur0xQzz github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.0-20190522114515-bc1a522cf7b1/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4= @@ -463,20 +530,24 @@ github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6L github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/russross/blackfriday v1.5.2 h1:HyvC0ARfnZBqnXwABFeSZHpKvJHJJfPz81GNueLj0oo= github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= -github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= -github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= +github.com/sirupsen/logrus v1.0.4-0.20170822132746-89742aefa4b2/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= +github.com/sirupsen/logrus v1.0.6/go.mod h1:pMByvHTf9Beacp5x1UXfOR9xyW/9antXMhjMPG0dEzc= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= +github.com/smartystreets/goconvey v0.0.0-20190330032615-68dc04aab96a/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= @@ -501,11 +572,13 @@ github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5q github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= github.com/spyzhov/ajson v0.4.2 h1:JMByd/jZApPKDvNsmO90X2WWGbmT2ahDFp73QhZbg3s= github.com/spyzhov/ajson v0.4.2/go.mod h1:63V+CGM6f1Bu/p4nLIN8885ojBdt88TbLoSFzyqMuVA= +github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980/go.mod h1:AO3tvPzVZ/ayst6UlUKUv6rcPQInYe3IknH3jYhAKu8= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48= github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= +github.com/stretchr/testify v0.0.0-20180303142811-b89eecf5ca5d/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= @@ -526,8 +599,19 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +<<<<<<< HEAD go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= +======= +github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43/go.mod h1:aX5oPXxHm3bOH+xeAttToC8pqch2ScQN/JoXYupl6xs= +github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPSUX/bi6SeDMUh6brw0nXpxHnc96TguQh0+r/ssA= +github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg= +go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= +go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= go.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= go.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ= @@ -535,6 +619,10 @@ go.etcd.io/etcd/client/v3 v3.5.0/go.mod h1:AIKXXVX/DQXtfTEqBryiLTUXwON+GuvO6Z7lL go.etcd.io/etcd/pkg/v3 v3.5.0/go.mod h1:UzJGatBQ1lXChBkQF0AuAtkRQMYnHubxAEYIrC3MSsE= go.etcd.io/etcd/raft/v3 v3.5.0/go.mod h1:UFOHSIvO/nKwd4lhkwabrTD3cqW5yVyYYf/KlD00Szc= go.etcd.io/etcd/server/v3 v3.5.0/go.mod h1:3Ah5ruV+M+7RZr0+Y/5mNLwC+eQlni+mQmOVdCRJoS4= +<<<<<<< HEAD +======= +go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= @@ -567,17 +655,29 @@ go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= go.uber.org/zap v1.19.0 h1:mZQZefskPPCMIBCSEH0v2/iUqqLrYtaeqwD6FUGUnFE= go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= +<<<<<<< HEAD +======= +golang.org/x/crypto v0.0.0-20171113213409-9f005a07e0d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20181009213950-7c1a557ab941/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +<<<<<<< HEAD +======= +golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83 h1:/ZScEX8SfEmUGRHs0gxpqteO5nfNW6axyZbBdw9A12g= golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= +golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 h1:It14KIkyBFYkHkwZ7k45minvA9aorojkyjGk9KJ5B/w= +golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -613,11 +713,18 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +<<<<<<< HEAD golang.org/x/mod v0.5.1 h1:OJxoQ/rynoF0dcCdI7cLPktw/hR2cueqYfjm43oqK38= golang.org/x/mod v0.5.1/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +======= +golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181011144130-49bb7cea24b1/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -631,10 +738,12 @@ golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190619014844-b5b0513f8c1b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= @@ -650,18 +759,33 @@ golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/ golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +<<<<<<< HEAD +======= +golang.org/x/net v0.0.0-20201006153459-a7d1128ccaa0/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +<<<<<<< HEAD +======= +golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= +<<<<<<< HEAD golang.org/x/net v0.0.0-20210520170846-37e1c6afe023 h1:ADo5wSpq2gqaCGQWzk7S5vd//0iyyLeAratkEoG5dLE= golang.org/x/net v0.0.0-20210520170846-37e1c6afe023/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +======= +golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210520170846-37e1c6afe023/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210525063256-abc453219eb5 h1:wjuX4b5yYQnEQHzd+CBcrcC6OVR2J1CN6mUy0oSxIPo= +golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -673,8 +797,14 @@ golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +<<<<<<< HEAD golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602 h1:0Ja1LBD+yisY6RWM/BH7TJVXWsSjs2VwBSmvSX4HdBc= golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +======= +golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c h1:pkQiBZBvdos9qq4wBAHqlzuZHEXo07pqV06ef90u1WI= +golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -685,6 +815,10 @@ golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +<<<<<<< HEAD +======= +golang.org/x/sync v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -699,22 +833,45 @@ golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190514135907-3a4b5fb9f71f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190522044717-8097e1b27ff5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190602015325-4c4f7f33c9ed/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +<<<<<<< HEAD golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +======= +golang.org/x/sys v0.0.0-20190606203320-7fc4e5ec1444/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190812073006-9eafafc0a87e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191002063906-3421d5a6bb1c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191115151921-52ab43148777/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +<<<<<<< HEAD +======= +golang.org/x/sys v0.0.0-20191210023423-ac6580df4449/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200120151820-655fe14d7479/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -725,14 +882,32 @@ golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200622214017-ed371f2e16b4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +<<<<<<< HEAD +golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +======= +golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200817155316-9781c653f443/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200909081042-eff7692f9009/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200916030750-2334cc1a136f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200922070232-aee5d888a860/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201112073958-5cba982894dd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201117170446-d9b008d0a637/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +<<<<<<< HEAD +======= +golang.org/x/sys v0.0.0-20201202213521-69691e467435/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -741,11 +916,21 @@ golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +<<<<<<< HEAD +======= +golang.org/x/sys v0.0.0-20210324051608-47abb6519492/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +<<<<<<< HEAD +golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +======= +golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210817190340-bfb29a6856f2 h1:c8PlLMqBbOHoqtjteWm5/kbe6rNY2pbRfbIMVnepueo= golang.org/x/sys v0.0.0-20210817190340-bfb29a6856f2/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= @@ -765,6 +950,10 @@ golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +<<<<<<< HEAD +======= +golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs= golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -781,6 +970,10 @@ golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBn golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +<<<<<<< HEAD +======= +golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= @@ -862,6 +1055,7 @@ google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCID google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/cloud v0.0.0-20151119220103-975617b05ea8/go.mod h1:0H1ncTHf11KCFhTc/+EFRbzSCOZx+VUbRMk55Yv5MYk= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= @@ -875,6 +1069,7 @@ google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvx google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200117163144-32f20d992d24/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= @@ -940,8 +1135,10 @@ google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlba google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +gopkg.in/airbrake/gobrake.v2 v2.0.9/go.mod h1:/h5ZAUhDkGaJfjzjKLSjv6zCL6O0LLBxU4K+aSYdM/U= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20141024133853-64131543e789/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= @@ -955,6 +1152,8 @@ gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= +gopkg.in/square/go-jose.v2 v2.3.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= +gopkg.in/square/go-jose.v2 v2.5.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= diff --git a/internal/cmddestroy/cmddestroy.go b/internal/cmddestroy/cmddestroy.go index 5d71c70758..5bca9d9216 100644 --- a/internal/cmddestroy/cmddestroy.go +++ b/internal/cmddestroy/cmddestroy.go @@ -166,6 +166,14 @@ func runDestroy(r *Runner, inv inventory.InventoryInfo, dryRunStrategy common.Dr if err != nil { return err } +<<<<<<< HEAD +======= + options := apply.DestroyerOptions{ + InventoryPolicy: r.inventoryPolicy, + DryRunStrategy: dryRunStrategy, + } + ch := destroyer.Run(context.Background(), inv, options) +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) destroyer, err := apply.NewDestroyer(r.factory, invClient) if err != nil { diff --git a/internal/cmdget/cmdget.go b/internal/cmdget/cmdget.go index 8d9a47fa89..9cb9e14aba 100644 --- a/internal/cmdget/cmdget.go +++ b/internal/cmdget/cmdget.go @@ -28,6 +28,7 @@ import ( "github.com/GoogleContainerTools/kpt/internal/util/cmdutil" "github.com/GoogleContainerTools/kpt/internal/util/get" "github.com/GoogleContainerTools/kpt/internal/util/parse" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" "github.com/spf13/cobra" ) @@ -38,7 +39,7 @@ func NewRunner(ctx context.Context, parent string) *Runner { ctx: ctx, } c := &cobra.Command{ - Use: "get REPO_URI[.git]/PKG_PATH[@VERSION] [LOCAL_DEST_DIRECTORY]", + Use: "get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY]", Args: cobra.MinimumNArgs(1), Short: docs.GetShort, Long: docs.GetShort + "\n" + docs.GetLong, @@ -84,15 +85,24 @@ func (r *Runner) preRunE(_ *cobra.Command, args []string) error { args[1] = resolvedPath } } - t, err := parse.GitParseArgs(r.ctx, args) + destination, err := parse.ParseArgs(r.ctx, args, parse.Options{ + SetGit: func(git *kptfilev1.Git) error { + r.Get.Git = git + r.Get.Upstream = remote.NewGitUpstream(git) + return nil + }, + SetOci: func(oci *kptfilev1.Oci) error { + r.Get.Upstream = remote.NewOciUpstream(oci) + return nil + }, + }) if err != nil { - return errors.E(op, err) + return err } - r.Get.Git = &t.Git - p, err := pkg.New(t.Destination) + p, err := pkg.New(destination) if err != nil { - return errors.E(op, types.UniquePath(t.Destination), err) + return errors.E(op, types.UniquePath(destination), err) } r.Get.Destination = string(p.UniquePath) diff --git a/internal/cmdget/cmdget_test.go b/internal/cmdget/cmdget_test.go index feb58cd914..b43b01eff2 100644 --- a/internal/cmdget/cmdget_test.go +++ b/internal/cmdget/cmdget_test.go @@ -345,6 +345,7 @@ func TestCmd_Execute_flagAndArgParsing(t *testing.T) { validations: func(repo, dir string, r *cmdget.Runner, err error) { assert.Error(t, err) assert.Contains(t, err.Error(), "specify '.git'") + assert.Contains(t, err.Error(), "specify 'oci://'") }, }, "valid strategy provided": { diff --git a/internal/cmdpull/cmdpull.go b/internal/cmdpull/cmdpull.go new file mode 100644 index 0000000000..28947012dc --- /dev/null +++ b/internal/cmdpull/cmdpull.go @@ -0,0 +1,108 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package cmdpull + +import ( + "context" + "os" + + docs "github.com/GoogleContainerTools/kpt/internal/docs/generated/pkgdocs" + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/types" + "github.com/GoogleContainerTools/kpt/internal/util/argutil" + "github.com/GoogleContainerTools/kpt/internal/util/cmdutil" + "github.com/GoogleContainerTools/kpt/internal/util/parse" + "github.com/GoogleContainerTools/kpt/internal/util/pull" + "github.com/GoogleContainerTools/kpt/internal/util/remote" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + "github.com/spf13/cobra" +) + +// NewRunner returns a command runner +func NewRunner(ctx context.Context, parent string) *Runner { + r := &Runner{ + ctx: ctx, + } + c := &cobra.Command{ + Use: "pull {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY]", + Args: cobra.MinimumNArgs(1), + Short: docs.PullShort, + Long: docs.PullShort + "\n" + docs.PullLong, + Example: docs.PullExamples, + RunE: r.runE, + PreRunE: r.preRunE, + } + cmdutil.FixDocs("kpt", parent, c) + r.Command = c + return r +} + +func NewCommand(ctx context.Context, parent string) *cobra.Command { + return NewRunner(ctx, parent).Command +} + +// Runner contains the run function +type Runner struct { + ctx context.Context + Pull pull.Command + Command *cobra.Command +} + +func (r *Runner) preRunE(_ *cobra.Command, args []string) error { + const op errors.Op = "cmdpull.preRunE" + if len(args) == 1 { + args = append(args, pkg.CurDir) + } else { + _, err := os.Lstat(args[1]) + if err == nil || os.IsExist(err) { + resolvedPath, err := argutil.ResolveSymlink(r.ctx, args[1]) + if err != nil { + return errors.E(op, err) + } + args[1] = resolvedPath + } + } + destination, err := parse.ParseArgs(r.ctx, args, parse.Options{ + SetGit: func(git *kptfilev1.Git) error { + r.Pull.Origin = remote.NewGitOrigin(git) + return nil + }, + SetOci: func(oci *kptfilev1.Oci) error { + r.Pull.Origin = remote.NewOciOrigin(oci) + return nil + }, + }) + if err != nil { + return err + } + + p, err := pkg.New(destination) + if err != nil { + return errors.E(op, types.UniquePath(destination), err) + } + r.Pull.Destination = string(p.UniquePath) + + return nil +} + +func (r *Runner) runE(c *cobra.Command, _ []string) error { + const op errors.Op = "cmdpull.runE" + if err := r.Pull.Run(r.ctx); err != nil { + return errors.E(op, types.UniquePath(r.Pull.Destination), err) + } + + return nil +} diff --git a/internal/cmdpush/cmdpush.go b/internal/cmdpush/cmdpush.go new file mode 100644 index 0000000000..33e9c8a49a --- /dev/null +++ b/internal/cmdpush/cmdpush.go @@ -0,0 +1,161 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package cmdpush + +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" + + docs "github.com/GoogleContainerTools/kpt/internal/docs/generated/pkgdocs" + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/types" + "github.com/GoogleContainerTools/kpt/internal/util/argutil" + "github.com/GoogleContainerTools/kpt/internal/util/cmdutil" + "github.com/GoogleContainerTools/kpt/internal/util/parse" + "github.com/GoogleContainerTools/kpt/internal/util/push" + "github.com/GoogleContainerTools/kpt/internal/util/remote" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + "github.com/spf13/cobra" +) + +// NewRunner returns a command runner +func NewRunner(ctx context.Context, parent string) *Runner { + r := &Runner{ + ctx: ctx, + } + c := &cobra.Command{ + Use: "push [DIR@VERSION] [flags]", + Args: cobra.MaximumNArgs(1), + Short: docs.PushShort, + Long: docs.PushShort + "\n" + docs.PushLong, + Example: docs.PushExamples, + RunE: r.runE, + PreRunE: r.preRunE, + } + + c.Flags().StringVar(&r.Origin, "origin", "", + "assigns or changes the location where the package should be pushed. Default is to push it to "+ + "the origin from which the package was pulled.") + c.Flags().BoolVar(&r.Increment, "increment", false, + "increment the version of the package when pushed. This will increment the DIR@VERSION if provided, "+ + "otherwise it will increment the origin's version when pulled. The version must be semver or integer, and "+ + "may have an optional leading 'v'") + cmdutil.FixDocs("kpt", parent, c) + r.Command = c + return r +} + +func NewCommand(ctx context.Context, parent string) *cobra.Command { + return NewRunner(ctx, parent).Command +} + +// Runner contains the run function +type Runner struct { + ctx context.Context + Push push.Command + Command *cobra.Command + Origin string + Increment bool +} + +func (r *Runner) preRunE(_ *cobra.Command, args []string) error { + const op errors.Op = "cmdpush.preRunE" + + var path string + var ref string + + if len(args) >= 1 { + parts := strings.Split(args[0], "@") + if len(parts) > 2 { + return errors.E(op, errors.InvalidParam, fmt.Errorf("at most 1 version permitted")) + } + + path = parts[0] + if len(parts) == 2 { + ref = parts[1] + } + } + + if path == "" { + // default to current directory + path = "." + } + + resolvedPath, err := argutil.ResolveSymlink(r.ctx, path) + if err != nil { + return err + } + + if ref != "" { + r.Push.Ref = ref + } + + r.Push.Increment = r.Increment + + r.Push.Pkg, err = pkg.New(resolvedPath) + if err != nil { + return errors.E(op, err) + } + relPath, err := resolveRelPath(r.Push.Pkg.UniquePath) + if err != nil { + return errors.E(op, r.Push.Pkg.UniquePath, err) + } + if strings.HasPrefix(relPath, pkg.ParentDir) { + return errors.E(op, r.Push.Pkg.UniquePath, fmt.Errorf("package path must be under current working directory")) + } + + if r.Origin != "" { + _, err := parse.ParseArgs(r.ctx, args, parse.Options{ + SetOci: func(oci *kptfilev1.Oci) error { + r.Push.Origin = remote.NewOciOrigin(oci) + return nil + }, + }) + if err != nil { + return err + } + } + + return nil +} + +func (r *Runner) runE(c *cobra.Command, _ []string) error { + const op errors.Op = "cmdpush.runE" + if err := r.Push.Run(r.ctx); err != nil { + return errors.E(op, r.Push.Pkg.UniquePath, err) + } + + return nil +} + +func resolveRelPath(path types.UniquePath) (string, error) { + const op errors.Op = "cmdpush.resolveRelPath" + cwd, err := os.Getwd() + if err != nil { + return "", errors.E(op, errors.IO, + fmt.Errorf("error looking up current working directory: %w", err)) + } + + relPath, err := filepath.Rel(cwd, path.String()) + if err != nil { + return "", errors.E(op, errors.IO, + fmt.Errorf("error resolving the relative path: %w", err)) + } + return relPath, nil +} diff --git a/internal/cmdrender/executor.go b/internal/cmdrender/executor.go index 8489e73db5..aeb1c460c9 100644 --- a/internal/cmdrender/executor.go +++ b/internal/cmdrender/executor.go @@ -28,7 +28,10 @@ import ( "github.com/GoogleContainerTools/kpt/internal/printer" "github.com/GoogleContainerTools/kpt/internal/types" "github.com/GoogleContainerTools/kpt/internal/util/attribution" +<<<<<<< HEAD "github.com/GoogleContainerTools/kpt/internal/util/cmdutil" +======= +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) "github.com/GoogleContainerTools/kpt/internal/util/printerutil" fnresult "github.com/GoogleContainerTools/kpt/pkg/api/fnresult/v1" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" diff --git a/internal/docs/generated/pkgdocs/docs.go b/internal/docs/generated/pkgdocs/docs.go index bb1970edca..588f10614a 100644 --- a/internal/docs/generated/pkgdocs/docs.go +++ b/internal/docs/generated/pkgdocs/docs.go @@ -97,7 +97,7 @@ var DiffExamples = ` var GetShort = `Fetch a package from a git repo.` var GetLong = ` - kpt pkg get REPO_URI[.git]/PKG_PATH[@VERSION] [LOCAL_DEST_DIRECTORY] [flags] + kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] Args: @@ -119,6 +119,13 @@ Args: A git tag, branch, ref or commit for the remote version of the package to fetch. Defaults to the default branch of the repository. + IMAGE: + Reference to an OCI image containing a package in the root directory. + + TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + LOCAL_DEST_DIRECTORY: The local directory to write the package to. Defaults to a subdirectory of the current working directory named after the upstream package. @@ -193,6 +200,157 @@ var InitExamples = ` $ kpt pkg init ` +var PullShort = `TODO(oci-support) rewrite: Fetch a package from a git repo.` +var PullLong = ` + kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] + +Args: + + REPO_URI: + URI of a git repository containing 1 or more packages as subdirectories. + In most cases the .git suffix should be specified to delimit the REPO_URI + from the PKG_PATH, but this is not required for widely recognized repo + prefixes. If get cannot parse the repo for the directory and version, + then it will print an error asking for '.git' to be specified as part of + the argument. + + PKG_PATH: + Path to remote subdirectory containing Kubernetes resource configuration + files or directories. Defaults to the root directory. + Uses '/' as the path separator (regardless of OS). + e.g. staging/cockroachdb + + VERSION: + A git tag, branch, ref or commit for the remote version of the package + to fetch. Defaults to the default branch of the repository. + + IMAGE: + Reference to an OCI image containing a package in the root directory. + + TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + + LOCAL_DEST_DIRECTORY: + The local directory to write the package to. Defaults to a subdirectory of the + current working directory named after the upstream package. + +Flags: + + --strategy: + Defines which strategy should be used to update the package. It defaults to + 'resource-merge'. + + * resource-merge: Perform a structural comparison of the original / + updated resources, and merge the changes into the local package. + * fast-forward: Fail without updating if the local package was modified + since it was fetched. + * force-delete-replace: Wipe all the local changes to the package and replace + it with the remote version. + +Env Vars: + + KPT_CACHE_DIR: + Controls where to cache remote packages when fetching them. + Defaults to /.kpt/repos/ + On macOS and Linux is determined by the $HOME env variable, while on + Windows it is given by the %USERPROFILE% env variable. +` +var PullExamples = ` + + # Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb + # This creates a new subdirectory 'cockroachdb' for the downloaded package. + $ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master + + + # Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb + # This will create a new directory 'my-package' for the downloaded package if it + # doesn't already exist. + $ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master ./my-package/ + + + # Fetch package examples from github.com/kubernetes/examples at the specified + # git hash. + # This will create a new directory 'examples' for the package. + $ kpt pkg get https://github.com/kubernetes/examples.git/@6fe2792 +` + +var PushShort = `TODO(oci-support) rewrite: Fetch a package from a git repo.` +var PushLong = ` +TODO(oci-support) rewrite: + kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] + +Args: + + REPO_URI: + URI of a git repository containing 1 or more packages as subdirectories. + In most cases the .git suffix should be specified to delimit the REPO_URI + from the PKG_PATH, but this is not required for widely recognized repo + prefixes. If get cannot parse the repo for the directory and version, + then it will print an error asking for '.git' to be specified as part of + the argument. + + PKG_PATH: + Path to remote subdirectory containing Kubernetes resource configuration + files or directories. Defaults to the root directory. + Uses '/' as the path separator (regardless of OS). + e.g. staging/cockroachdb + + VERSION: + A git tag, branch, ref or commit for the remote version of the package + to fetch. Defaults to the default branch of the repository. + + IMAGE: + Reference to an OCI image containing a package in the root directory. + + TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + + LOCAL_DEST_DIRECTORY: + The local directory to write the package to. Defaults to a subdirectory of the + current working directory named after the upstream package. + +Flags: + + --strategy: + Defines which strategy should be used to update the package. It defaults to + 'resource-merge'. + + * resource-merge: Perform a structural comparison of the original / + updated resources, and merge the changes into the local package. + * fast-forward: Fail without updating if the local package was modified + since it was fetched. + * force-delete-replace: Wipe all the local changes to the package and replace + it with the remote version. + +Env Vars: + + KPT_CACHE_DIR: + Controls where to cache remote packages when fetching them. + Defaults to /.kpt/repos/ + On macOS and Linux is determined by the $HOME env variable, while on + Windows it is given by the %USERPROFILE% env variable. +` +var PushExamples = ` + + # Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb + # This creates a new subdirectory 'cockroachdb' for the downloaded package. + $ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master + + + # Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb + # This will create a new directory 'my-package' for the downloaded package if it + # doesn't already exist. + $ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master ./my-package/ + + + # Fetch package examples from github.com/kubernetes/examples at the specified + # git hash. + # This will create a new directory 'examples' for the package. + $ kpt pkg get https://github.com/kubernetes/examples.git/@6fe2792 +` + var TreeShort = `Display resources, files and packages in a tree structure.` var TreeLong = ` kpt pkg tree [DIR] diff --git a/internal/errors/errors.go b/internal/errors/errors.go index 4800ffbfa0..4c34e89a8f 100644 --- a/internal/errors/errors.go +++ b/internal/errors/errors.go @@ -142,6 +142,7 @@ const ( InvalidParam // Value is not valid. MissingParam // Required value is missing or empty. Git // Errors from Git + OCI // Errors from OCI IO // Error doing IO operations YAML // yaml document can't be parsed ) @@ -160,6 +161,8 @@ func (c Class) String() string { return "missing parameter value" case Git: return "git error" + case OCI: + return "OCI error" case IO: return "IO error" case YAML: diff --git a/internal/testutil/setup_manager.go b/internal/testutil/setup_manager.go index 8b5ab5c057..1be9e9a458 100644 --- a/internal/testutil/setup_manager.go +++ b/internal/testutil/setup_manager.go @@ -25,6 +25,7 @@ import ( "github.com/GoogleContainerTools/kpt/internal/printer/fake" "github.com/GoogleContainerTools/kpt/internal/testutil/pkgbuilder" "github.com/GoogleContainerTools/kpt/internal/util/get" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" "github.com/stretchr/testify/assert" "sigs.k8s.io/kustomize/kyaml/yaml" @@ -106,11 +107,11 @@ func (g *TestSetupManager) Init() bool { // Get the content from the upstream repo into the local workspace. if !assert.NoError(g.T, get.Command{ Destination: filepath.Join(g.LocalWorkspace.WorkspaceDirectory, g.targetDir), - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.Repos[Upstream].RepoDirectory, Ref: g.GetRef, Directory: g.GetSubDirectory, - }}.Run(fake.CtxWithDefaultPrinter())) { + })}.Run(fake.CtxWithDefaultPrinter())) { return false } localGit, err := gitutil.NewLocalGitRunner(g.LocalWorkspace.WorkspaceDirectory) @@ -202,11 +203,11 @@ func UpdateGitDir(t *testing.T, name string, gitDir GitDirectory, changes []Cont func (g *TestSetupManager) GetSubPkg(dest, repo, upstreamDir string) { assert.NoError(g.T, get.Command{ Destination: path.Join(g.LocalWorkspace.FullPackagePath(), dest), - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.Repos[repo].RepoDirectory, Ref: g.GetRef, Directory: path.Join(g.GetSubDirectory, upstreamDir), - }}.Run(fake.CtxWithDefaultPrinter())) + })}.Run(fake.CtxWithDefaultPrinter())) localGit, err := gitutil.NewLocalGitRunner(g.LocalWorkspace.WorkspaceDirectory) assert.NoError(g.T, err) diff --git a/internal/util/diff/diff.go b/internal/util/diff/diff.go index e2f3a20eb7..a24925a9db 100644 --- a/internal/util/diff/diff.go +++ b/internal/util/diff/diff.go @@ -28,10 +28,9 @@ import ( "github.com/GoogleContainerTools/kpt/internal/gitutil" "github.com/GoogleContainerTools/kpt/internal/pkg" "github.com/GoogleContainerTools/kpt/internal/util/addmergecomment" - "github.com/GoogleContainerTools/kpt/internal/util/fetch" "github.com/GoogleContainerTools/kpt/internal/util/pkgutil" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" - "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" "sigs.k8s.io/kustomize/kyaml/errors" ) @@ -111,8 +110,8 @@ type Command struct { // PkgDiffer specifies package differ PkgDiffer PkgDiffer - // PkgGetter specifies packaging sourcing adapter - PkgGetter PkgGetter + // Contains information about the upstream package to fetch + Upstream remote.Upstream } func (c *Command) Run(ctx context.Context) error { @@ -123,6 +122,15 @@ func (c *Command) Run(ctx context.Context) error { return errors.Errorf("package missing Kptfile at '%s': %v", c.Path, err) } + c.Upstream, err = remote.NewUpstream(kptFile) + if err != nil { + return errors.Errorf("upstream required: %v", err) + } + upstreamRef, err := c.Upstream.Ref() + if err != nil { + return errors.Errorf("upstream ref required: %v", err) + } + // Create a staging directory to store all compared packages stagingDirectory, err := ioutil.TempDir("", "kpt-") if err != nil { @@ -138,7 +146,7 @@ func (c *Command) Run(ctx context.Context) error { // Stage current package // This prevents prepareForDiff from modifying the local package localPkgName := NameStagingDirectory(LocalPackageSource, - kptFile.Upstream.Git.Ref) + upstreamRef) currPkg, err := stageDirectory(stagingDirectory, localPkgName) if err != nil { return errors.Errorf("failed to create stage dir for current package: %v", err) @@ -150,14 +158,11 @@ func (c *Command) Run(ctx context.Context) error { } // get the upstreamPkg at current version - upstreamPkgName := NameStagingDirectory(RemotePackageSource, - kptFile.Upstream.Git.Ref) - upstreamPkg, err := c.PkgGetter.GetPkg(ctx, - stagingDirectory, - upstreamPkgName, - kptFile.Upstream.Git.Repo, - kptFile.Upstream.Git.Directory, - kptFile.Upstream.Git.Ref) + upstreamPkgName, err := stageDirectory(stagingDirectory, NameStagingDirectory(RemotePackageSource, upstreamRef)) + if err != nil { + return err + } + upstreamPkg, _, err := c.Upstream.FetchUpstream(ctx, upstreamPkgName) if err != nil { return err } @@ -165,13 +170,18 @@ func (c *Command) Run(ctx context.Context) error { var upstreamTargetPkg string if c.Ref == "" { - gur, err := gitutil.NewGitUpstreamRepo(ctx, kptFile.UpstreamLock.Git.Repo) - if err != nil { - return err - } - c.Ref, err = gur.GetDefaultBranch(ctx) - if err != nil { - return err + switch kptFile.UpstreamLock.Type { + case kptfilev1.GitOrigin: + gur, err := gitutil.NewGitUpstreamRepo(ctx, kptFile.UpstreamLock.Git.Repo) + if err != nil { + return err + } + c.Ref, err = gur.GetDefaultBranch(ctx) + if err != nil { + return err + } + case kptfilev1.OciOrigin: + c.Ref = "latest" } } @@ -179,13 +189,14 @@ func (c *Command) Run(ctx context.Context) error { c.DiffType == DiffTypeCombined || c.DiffType == DiffType3Way { // get the upstream pkg at the target version - upstreamTargetPkgName := NameStagingDirectory(TargetRemotePackageSource, - c.Ref) - upstreamTargetPkg, err = c.PkgGetter.GetPkg(ctx, stagingDirectory, - upstreamTargetPkgName, - kptFile.Upstream.Git.Repo, - kptFile.Upstream.Git.Directory, - c.Ref) + upstreamTargetPkgName, err := stageDirectory(stagingDirectory, NameStagingDirectory(TargetRemotePackageSource, c.Ref)) + if err != nil { + return err + } + if err := c.Upstream.SetRef(c.Ref); err != nil { + return err + } + upstreamTargetPkg, _, err = c.Upstream.FetchUpstream(ctx, upstreamTargetPkgName) if err != nil { return err } @@ -231,9 +242,6 @@ func (c *Command) DefaultValues() { if c.Output == nil { c.Output = os.Stdout } - if c.PkgGetter == nil { - c.PkgGetter = defaultPkgGetter{} - } if c.PkgDiffer == nil { c.PkgDiffer = &defaultPkgDiffer{ DiffType: c.DiffType, @@ -322,52 +330,6 @@ func (d *defaultPkgDiffer) prepareForDiff(dir string) error { return nil } -// PkgGetter knows how to fetch a package given a git repo, path and ref. -type PkgGetter interface { - GetPkg(ctx context.Context, stagingDir, targetDir, repo, path, ref string) (dir string, err error) -} - -// defaultPkgGetter uses fetch.Command abstraction to implement PkgGetter. -type defaultPkgGetter struct{} - -// GetPkg checks out a repository into a temporary directory for diffing -// and returns the directory containing the checked out package or an error. -// repo is the git repository the package was cloned from. e.g. https:// -// path is the sub directory of the git repository that the package was cloned from -// ref is the git ref the package was cloned from -func (pg defaultPkgGetter) GetPkg(ctx context.Context, stagingDir, targetDir, repo, path, ref string) (string, error) { - dir, err := stageDirectory(stagingDir, targetDir) - if err != nil { - return dir, err - } - - name := filepath.Base(dir) - kf := kptfileutil.DefaultKptfile(name) - kf.Upstream = &kptfilev1.Upstream{ - Type: kptfilev1.GitOrigin, - Git: &kptfilev1.Git{ - Repo: repo, - Directory: path, - Ref: ref, - }, - } - err = kptfileutil.WriteFile(dir, kf) - if err != nil { - return dir, err - } - - p, err := pkg.New(dir) - if err != nil { - return dir, err - } - - cmdGet := &fetch.Command{ - Pkg: p, - } - err = cmdGet.Run(ctx) - return dir, err -} - // stageDirectory creates a subdirectory of the provided path for temporary operations // path is the parent staged directory and should already exist // subpath is the subdirectory that should be created inside path diff --git a/internal/util/fetch/fetch.go b/internal/util/fetch/fetch.go index 1d384a0f5e..b7ca9a5f21 100644 --- a/internal/util/fetch/fetch.go +++ b/internal/util/fetch/fetch.go @@ -17,23 +17,11 @@ package fetch import ( "context" "fmt" - "io/ioutil" - "os" - "path" - "path/filepath" - "strings" - - "github.com/otiai10/copy" "github.com/GoogleContainerTools/kpt/internal/errors" - "github.com/GoogleContainerTools/kpt/internal/gitutil" "github.com/GoogleContainerTools/kpt/internal/pkg" - "github.com/GoogleContainerTools/kpt/internal/printer" - "github.com/GoogleContainerTools/kpt/internal/types" - "github.com/GoogleContainerTools/kpt/internal/util/git" - "github.com/GoogleContainerTools/kpt/internal/util/pkgutil" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" - "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" ) // Command takes the upstream information in the Kptfile at the path for the @@ -55,16 +43,15 @@ func (c Command) Run(ctx context.Context) error { return errors.E(op, c.Pkg.UniquePath, err) } - g := kf.Upstream.Git - repoSpec := &git.RepoSpec{ - OrgRepo: g.Repo, - Path: g.Directory, - Ref: g.Ref, - } - err = cloneAndCopy(ctx, repoSpec, c.Pkg.UniquePath.String()) + ups, err := remote.NewUpstream(kf) if err != nil { return errors.E(op, c.Pkg.UniquePath, err) } + + if err := ups.CloneUpstream(ctx, c.Pkg.UniquePath.String()); err != nil { + return errors.E(op, c.Pkg.UniquePath, err) + } + return nil } @@ -75,183 +62,34 @@ func (c Command) validate(kf *kptfilev1.KptFile) error { if kf.Upstream == nil { return errors.E(op, errors.MissingParam, fmt.Errorf("kptfile doesn't contain upstream information")) } - - if kf.Upstream.Git == nil { - return errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream doesn't have git information")) - } - - g := kf.Upstream.Git - if len(g.Repo) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify repo")) - } - if len(g.Ref) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify ref")) - } - if len(g.Directory) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify directory")) - } - return nil -} - -// cloneAndCopy fetches the provided repo and copies the content into the -// directory specified by dest. The provided name is set as `metadata.name` -// of the Kptfile of the package. -func cloneAndCopy(ctx context.Context, r *git.RepoSpec, dest string) error { - const op errors.Op = "fetch.cloneAndCopy" - pr := printer.FromContextOrDie(ctx) - - err := ClonerUsingGitExec(ctx, r) - if err != nil { - return errors.E(op, errors.Git, types.UniquePath(dest), err) - } - defer os.RemoveAll(r.Dir) - - sourcePath := filepath.Join(r.Dir, r.Path) - pr.Printf("Adding package %q.\n", strings.TrimPrefix(r.Path, "/")) - if err := pkgutil.CopyPackage(sourcePath, dest, true, pkg.All); err != nil { - return errors.E(op, types.UniquePath(dest), err) - } - - if err := kptfileutil.UpdateKptfileWithoutOrigin(dest, sourcePath, false); err != nil { - return errors.E(op, types.UniquePath(dest), err) - } - - if err := kptfileutil.UpdateUpstreamLockFromGit(dest, r); err != nil { - return errors.E(op, errors.Git, types.UniquePath(dest), err) - } - return nil -} - -// ClonerUsingGitExec uses a local git install, as opposed -// to say, some remote API, to obtain a local clone of -// a remote repo. It looks for tags with the directory as a prefix to allow -// for versioning multiple kpt packages in a single repo independently. It -// relies on the private clonerUsingGitExec function to try fetching different -// refs. -func ClonerUsingGitExec(ctx context.Context, repoSpec *git.RepoSpec) error { - const op errors.Op = "fetch.ClonerUsingGitExec" - - // Create a local representation of the upstream repo. This will initialize - // the cache for the specified repo uri if it isn't already there. It also - // fetches and caches all tag and branch refs from the upstream repo. - upstreamRepo, err := gitutil.NewGitUpstreamRepo(ctx, repoSpec.CloneSpec()) - if err != nil { - return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) - } - - // Check if we have a ref in the upstream that matches the package-specific - // reference. If we do, we use that reference. - ps := strings.Split(repoSpec.Path, "/") - for len(ps) != 0 { - p := path.Join(ps...) - packageRef := path.Join(strings.TrimLeft(p, "/"), repoSpec.Ref) - if _, found := upstreamRepo.ResolveTag(packageRef); found { - repoSpec.Ref = packageRef - break + switch kf.Upstream.Type { + case kptfilev1.GitOrigin: + if kf.Upstream.Git == nil { + return errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream doesn't have git information")) } - ps = ps[:len(ps)-1] - } - - // Pull the required ref into the repo git cache. - dir, err := upstreamRepo.GetRepo(ctx, []string{repoSpec.Ref}) - if err != nil { - return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) - } - - gitRunner, err := gitutil.NewLocalGitRunner(dir) - if err != nil { - return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) - } - - // Find the commit SHA for the ref that was just fetched. We need the SHA - // rather than the ref to be able to do a hard reset of the cache repo. - commit, found := upstreamRepo.ResolveRef(repoSpec.Ref) - if !found { - commit = repoSpec.Ref - } - - // Reset the local repo to the commit we need. Doing a hard reset instead of - // a checkout means we don't create any local branches so we don't need to - // worry about fast-forwarding them with changes from upstream. It also makes - // sure that any changes in the local worktree are cleaned out. - _, err = gitRunner.Run(ctx, "reset", "--hard", commit) - if err != nil { - gitutil.AmendGitExecError(err, func(e *gitutil.GitExecError) { - e.Repo = repoSpec.CloneSpec() - e.Ref = commit - }) - return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) - } - - // We need to create a temp directory where we can copy the content of the repo. - // During update, we need to checkout multiple versions of the same repo, so - // we can't do merges directly from the cache. - repoSpec.Dir, err = ioutil.TempDir("", "kpt-get-") - if err != nil { - return errors.E(op, errors.Internal, fmt.Errorf("error creating temp directory: %w", err)) - } - repoSpec.Commit = commit - - pkgPath := filepath.Join(dir, repoSpec.Path) - // Verify that the requested path exists in the repo. - _, err = os.Stat(pkgPath) - if os.IsNotExist(err) { - return errors.E(op, - errors.Internal, - err, - fmt.Errorf("path %q does not exist in repo %q", repoSpec.Path, repoSpec.OrgRepo)) - } - - // Copy the content of the pkg into the temp directory. - // Note that we skip the content outside the package directory. - err = copyDir(ctx, pkgPath, repoSpec.AbsPath()) - if err != nil { - return errors.E(op, errors.Internal, fmt.Errorf("error copying package: %w", err)) - } - // Verify that if a Kptfile exists in the package, it contains the correct - // version of the Kptfile. - _, err = pkg.ReadKptfile(pkgPath) - if err != nil { - // A Kptfile isn't required, so it is fine if there is no Kptfile. - if errors.Is(err, os.ErrNotExist) { - return nil + g := kf.Upstream.Git + if len(g.Repo) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify repo")) + } + if len(g.Ref) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify ref")) + } + if len(g.Directory) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify directory")) } - // If the error is of type KptfileError, we replace it with a - // RemoteKptfileError. This allows us to provide information about the - // git source of the Kptfile instead of the path to some random - // temporary directory. - var kfError *pkg.KptfileError - if errors.As(err, &kfError) { - return &pkg.RemoteKptfileError{ - RepoSpec: repoSpec, - Err: kfError.Err, - } + case kptfilev1.OciOrigin: + if kf.Upstream.Oci == nil { + return errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream doesn't have oci information")) + } + if len(kf.Upstream.Oci.Image) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify image")) } - } - return nil -} -// copyDir copies a src directory to a dst directory. -// copyDir skips copying the .git directory from the src and ignores symlinks. -func copyDir(ctx context.Context, srcDir string, dstDir string) error { - pr := printer.FromContextOrDie(ctx) - opts := copy.Options{ - Skip: func(src string) (bool, error) { - return strings.HasSuffix(src, ".git"), nil - }, - OnSymlink: func(src string) copy.SymlinkAction { - // try to print relative path of symlink - // if we can, else absolute path which is not - // pretty because it contains path to temporary repo dir - displayPath, err := filepath.Rel(srcDir, src) - if err != nil { - displayPath = src - } - pr.Printf("[Warn] Ignoring symlink %q \n", displayPath) - return copy.Skip - }, + default: + return errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream type must be one of: %s,%s", kptfilev1.GitOrigin, kptfilev1.OciOrigin)) } - return copy.Copy(srcDir, dstDir, opts) + + return nil } diff --git a/internal/util/get/example_test.go b/internal/util/get/example_test.go index 6d0669e3b7..8ab1765ac1 100644 --- a/internal/util/get/example_test.go +++ b/internal/util/get/example_test.go @@ -19,44 +19,45 @@ import ( "github.com/GoogleContainerTools/kpt/internal/printer/fake" "github.com/GoogleContainerTools/kpt/internal/util/get" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" ) func ExampleCommand() { - err := get.Command{Git: &kptfilev1.Git{ + err := get.Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "v1.0", - }}.Run(fake.CtxWithDefaultPrinter()) + })}.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error } } func ExampleCommand_branch() { - err := get.Command{Git: &kptfilev1.Git{ + err := get.Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "refs/heads/v1.0", - }}.Run(fake.CtxWithDefaultPrinter()) + })}.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error } } func ExampleCommand_tag() { - err := get.Command{Git: &kptfilev1.Git{ + err := get.Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "refs/tags/v1.0", - }}.Run(fake.CtxWithDefaultPrinter()) + })}.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error } } func ExampleCommand_commit() { - err := get.Command{Git: &kptfilev1.Git{ + err := get.Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "8186bef8e5c0621bf80fa8106bd595aae8b62884", - }}.Run(fake.CtxWithDefaultPrinter()) + })}.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error } @@ -64,11 +65,11 @@ func ExampleCommand_commit() { func ExampleCommand_subdir() { err := get.Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "v1.0", Directory: filepath.Join("path", "to", "package"), - }, + }), }.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error @@ -77,10 +78,10 @@ func ExampleCommand_subdir() { func ExampleCommand_destination() { err := get.Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "https://github.com/example-org/example-repo", Ref: "v1.0", - }, + }), Destination: "destination-dir"}.Run(fake.CtxWithDefaultPrinter()) if err != nil { // handle error diff --git a/internal/util/get/get.go b/internal/util/get/get.go index 70b1d6587b..a65c00292b 100644 --- a/internal/util/get/get.go +++ b/internal/util/get/get.go @@ -20,9 +20,7 @@ import ( goerrors "errors" "fmt" "os" - "path" "path/filepath" - "strings" "github.com/GoogleContainerTools/kpt/internal/errors" "github.com/GoogleContainerTools/kpt/internal/pkg" @@ -31,6 +29,7 @@ import ( "github.com/GoogleContainerTools/kpt/internal/util/addmergecomment" "github.com/GoogleContainerTools/kpt/internal/util/attribution" "github.com/GoogleContainerTools/kpt/internal/util/fetch" + "github.com/GoogleContainerTools/kpt/internal/util/remote" "github.com/GoogleContainerTools/kpt/internal/util/stack" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" @@ -43,6 +42,9 @@ type Command struct { // Git contains information about the git repo to fetch Git *kptfilev1.Git + // Contains information about the upstraem package to fetch + Upstream remote.Upstream + // Destination is the output directory to clone the package to. Defaults to the name of the package -- // either the base repo name, or the base subdirectory name. Destination string @@ -72,19 +74,10 @@ func (c Command) Run(ctx context.Context) error { return errors.E(op, errors.IO, types.UniquePath(c.Destination), err) } - // normalize path to a filepath - repoDir := c.Git.Directory - if !strings.HasSuffix(repoDir, "file://") { - repoDir = filepath.Join(path.Split(repoDir)) - } - c.Git.Directory = repoDir - kf := kptfileutil.DefaultKptfile(c.Name) - kf.Upstream = &kptfilev1.Upstream{ - Type: kptfilev1.GitOrigin, - Git: c.Git, - UpdateStrategy: c.UpdateStrategy, - } + + kf.Upstream = c.Upstream.BuildUpstream() + kf.Upstream.UpdateStrategy = c.UpdateStrategy err = kptfileutil.WriteFile(c.Destination, kf) if err != nil { @@ -135,8 +128,14 @@ func (c Command) fetchPackages(ctx context.Context, rootPkg *pkg.Pkg) error { if kf.Upstream != nil && kf.UpstreamLock == nil { packageCount += 1 pr.PrintPackage(p, !(p == rootPkg)) - pr.Printf("Fetching %s@%s\n", kf.Upstream.Git.Repo, kf.Upstream.Git.Ref) - err := (&fetch.Command{ + + upstream, err := remote.NewUpstream(kf) + if err != nil { + return errors.E(op, p.UniquePath, err) + } + pr.Printf("Fetching %s\n", upstream.String()) + + err = (&fetch.Command{ Pkg: p, }).Run(ctx) if err != nil { @@ -159,23 +158,12 @@ func (c Command) fetchPackages(ctx context.Context, rootPkg *pkg.Pkg) error { // DefaultValues sets values to the default values if they were unspecified func (c *Command) DefaultValues() error { const op errors.Op = "get.DefaultValues" - if c.Git == nil { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify git repo information")) - } - g := c.Git - if len(g.Repo) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify repo")) - } - if len(g.Ref) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify ref")) + if c.Upstream == nil { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify git repo or image reference information")) } - if len(c.Destination) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify destination")) - } - if len(g.Directory) == 0 { - return errors.E(op, errors.MissingParam, fmt.Errorf("must specify directory")) + if err := c.Upstream.Validate(); err != nil { + return errors.E(op, err) } - if !filepath.IsAbs(c.Destination) { return errors.E(op, errors.InvalidParam, fmt.Errorf("destination must be an absolute path")) } diff --git a/internal/util/get/get_test.go b/internal/util/get/get_test.go index 96c0ca0198..393d16277c 100644 --- a/internal/util/get/get_test.go +++ b/internal/util/get/get_test.go @@ -24,6 +24,7 @@ import ( "github.com/GoogleContainerTools/kpt/internal/testutil" "github.com/GoogleContainerTools/kpt/internal/testutil/pkgbuilder" . "github.com/GoogleContainerTools/kpt/internal/util/get" + "github.com/GoogleContainerTools/kpt/internal/util/remote" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" "github.com/stretchr/testify/assert" "sigs.k8s.io/kustomize/kyaml/kio" @@ -48,7 +49,7 @@ func TestCommand_Run_failEmptyRepo(t *testing.T) { if !assert.Error(t, err) { t.FailNow() } - assert.Contains(t, err.Error(), "must specify git repo information") + assert.Contains(t, err.Error(), "must specify git repo or image reference information") } // TestCommand_Run_failEmptyRepo verifies that Command fail if not repo is provided. @@ -57,9 +58,9 @@ func TestCommand_Run_failNoRevision(t *testing.T) { defer clean() err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "foo", - }, + }), Destination: w.WorkspaceDirectory, }.Run(fake.CtxWithDefaultPrinter()) if !assert.Error(t, err) { @@ -82,11 +83,11 @@ func TestCommand_Run(t *testing.T) { defer testutil.Chdir(t, w.WorkspaceDirectory)() absPath := filepath.Join(w.WorkspaceDirectory, g.RepoName) - err := Command{Git: &kptfilev1.Git{ + err := Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "file://" + g.RepoDirectory, Ref: "master", Directory: "/", - }, + }), Destination: absPath}.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -143,8 +144,8 @@ func TestCommand_Run_subdir(t *testing.T) { defer testutil.Chdir(t, w.WorkspaceDirectory)() absPath := filepath.Join(w.WorkspaceDirectory, subdir) - err := Command{Git: &kptfilev1.Git{ - Repo: g.RepoDirectory, Ref: "refs/heads/master", Directory: subdir}, + err := Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ + Repo: g.RepoDirectory, Ref: "refs/heads/master", Directory: subdir}), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -207,8 +208,8 @@ func TestCommand_Run_subdir_symlinks(t *testing.T) { cliOutput := &bytes.Buffer{} absPath := filepath.Join(w.WorkspaceDirectory, subdir) - err := Command{Git: &kptfilev1.Git{ - Repo: g.RepoDirectory, Ref: "refs/heads/master", Directory: subdir}, + err := Command{Upstream: remote.NewGitUpstream(&kptfilev1.Git{ + Repo: g.RepoDirectory, Ref: "refs/heads/master", Directory: subdir}), Destination: absPath, }.Run(fake.CtxWithPrinter(cliOutput, cliOutput)) assert.NoError(t, err) @@ -272,11 +273,11 @@ func TestCommand_Run_destination(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, dest) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "master", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -336,11 +337,11 @@ func TestCommand_Run_subdirAndDestination(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, dest) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "master", Directory: subdir, - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -416,11 +417,11 @@ func TestCommand_Run_branch(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, g.RepoName) err = Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "refs/heads/exp", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -499,11 +500,11 @@ func TestCommand_Run_tag(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, g.RepoName) err = Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "refs/tags/v2", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -640,11 +641,11 @@ func TestCommand_Run_ref(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, repos[testutil.Upstream].RepoName) err = Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: repos[testutil.Upstream].RepoDirectory, Ref: ref, Directory: tc.directory, - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -669,11 +670,11 @@ func TestCommand_Run_failExistingDir(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, g.RepoName) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "master", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -723,11 +724,11 @@ func TestCommand_Run_failExistingDir(t *testing.T) { // try to clone and expect a failure err = Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "master", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) if !assert.Error(t, err) { @@ -780,11 +781,11 @@ func TestCommand_Run_nonexistingParentDir(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, "more", "dirs", g.RepoName) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Ref: "master", Directory: "/", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) assert.NoError(t, err) @@ -800,11 +801,11 @@ func TestCommand_Run_failInvalidRepo(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, "foo") err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: "foo", Directory: "/", Ref: "refs/heads/master", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) if !assert.Error(t, err) { @@ -830,11 +831,11 @@ func TestCommand_Run_failInvalidBranch(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, g.RepoDirectory) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Directory: "/", Ref: "refs/heads/foo", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) if !assert.Error(t, err) { @@ -863,11 +864,11 @@ func TestCommand_Run_failInvalidTag(t *testing.T) { absPath := filepath.Join(w.WorkspaceDirectory, g.RepoDirectory) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: g.RepoDirectory, Directory: "/", Ref: "refs/tags/foo", - }, + }), Destination: absPath, }.Run(fake.CtxWithDefaultPrinter()) if !assert.Error(t, err) { @@ -1388,11 +1389,11 @@ func TestCommand_Run_subpackages(t *testing.T) { destinationDir := filepath.Join(w.WorkspaceDirectory, targetDir) err = Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: upstreamRepo.RepoDirectory, Directory: tc.directory, Ref: tc.ref, - }, + }), Destination: destinationDir, UpdateStrategy: tc.updateStrategy, }.Run(fake.CtxWithDefaultPrinter()) @@ -1459,11 +1460,11 @@ func TestCommand_Run_symlinks(t *testing.T) { destinationDir := filepath.Join(w.WorkspaceDirectory, upstreamRepo.RepoName) err := Command{ - Git: &kptfilev1.Git{ + Upstream: remote.NewGitUpstream(&kptfilev1.Git{ Repo: upstreamRepo.RepoDirectory, Directory: "/", Ref: "master", - }, + }), Destination: destinationDir, }.Run(fake.CtxWithDefaultPrinter()) if !assert.NoError(t, err) { diff --git a/internal/util/parse/parse.go b/internal/util/parse/parse.go index 7bedafa6bc..1d894d592c 100644 --- a/internal/util/parse/parse.go +++ b/internal/util/parse/parse.go @@ -22,18 +22,113 @@ import ( "path/filepath" "strings" + kpterrors "github.com/GoogleContainerTools/kpt/internal/errors" "github.com/GoogleContainerTools/kpt/internal/gitutil" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + "github.com/google/go-containerregistry/pkg/name" "sigs.k8s.io/kustomize/kyaml/errors" ) -type Target struct { +type Options struct { + SetGit func(git *kptfilev1.Git) error + SetOci func(oci *kptfilev1.Oci) error +} + +func ParseArgs(ctx context.Context, args []string, opts Options) (string, error) { + const op kpterrors.Op = "parse.ParseArgs" + + tGit, errGit := GitParseArgs(ctx, args) + if errGit == nil { + if opts.SetGit == nil { + return "", kpterrors.E(op, fmt.Errorf("git locations not supported: %v", errGit)) + } + if err := opts.SetGit(&tGit.Git); err != nil { + return "", err + } + return tGit.Destination, nil + } + + tOci, errOci := OciParseArgs(ctx, args) + if errOci == nil { + if opts.SetOci == nil { + return "", kpterrors.E(op, fmt.Errorf("oci locations not supported: %v", errOci)) + } + if err := opts.SetOci(&tOci.Oci); err != nil { + return "", err + } + return tOci.Destination, nil + } + + // TODO(oci-support) combining error messages like this is suboptimal in several ways + return "", kpterrors.E(op, fmt.Errorf("%v %v", errGit, errOci)) +} + +type OciTarget struct { + kptfilev1.Oci + Destination string +} + +func OciParseArgs(ctx context.Context, args []string) (OciTarget, error) { + oci := OciTarget{} + if args[0] == "-" { + return oci, nil + } + + // The prefix must occur, and must not have other characters before it + arg0parts := strings.SplitN(args[0], "oci://", 2) + if len(arg0parts) != 2 || len(arg0parts[0]) != 0 { + return oci, errors.Errorf("ambiguous image:tag specify 'oci://' before argument: %s", args[0]) + } + + return targetFromImageReference(arg0parts[1], args[1]) +} + +func targetFromImageReference(image, dest string) (OciTarget, error) { + ref, err := name.ParseReference(image) + if err != nil { + return OciTarget{}, err + } + + registry := ref.Context().RegistryStr() + repository := ref.Context().RepositoryStr() + destination, err := getDest(dest, registry, repository) + if err != nil { + return OciTarget{}, err + } + + directory := "" + parts := strings.SplitN(ref.Context().Name(), "//", 2) + if len(parts) == 2 { + directory = "/" + parts[1] + repo, err := name.NewRepository(parts[0]) + if err != nil { + return OciTarget{}, err + } + + switch r := ref.(type) { + case name.Tag: + ref = repo.Tag(r.TagStr()) + case name.Digest: + ref = repo.Tag(r.DigestStr()) + } + } + + return OciTarget{ + Oci: kptfilev1.Oci{ + Image: ref.Name(), + Directory: directory, + }, + Destination: destination, + }, nil +} + +type GitTarget struct { kptfilev1.Git Destination string } -func GitParseArgs(ctx context.Context, args []string) (Target, error) { - g := Target{} +func GitParseArgs(ctx context.Context, args []string) (GitTarget, error) { + g := GitTarget{} if args[0] == "-" { return g, nil } @@ -72,20 +167,23 @@ func GitParseArgs(ctx context.Context, args []string) (Target, error) { version = defaultRef } - destination, err := getDest(args[1], repo, remoteDir) - if err != nil { - return g, err - } g.Ref = version g.Directory = path.Clean(remoteDir) g.Repo = repo - g.Destination = filepath.Clean(destination) + + if len(args) >= 2 { + destination, err := getDest(args[1], repo, remoteDir) + if err != nil { + return g, err + } + g.Destination = filepath.Clean(destination) + } return g, nil } // targetFromPkgURL parses a pkg url and destination into kptfile git info and local destination Target -func targetFromPkgURL(ctx context.Context, pkgURL, dest string) (Target, error) { - g := Target{} +func targetFromPkgURL(ctx context.Context, pkgURL, dest string) (GitTarget, error) { + g := GitTarget{} var repo, dir, version string parts := strings.Split(pkgURL, ".git") repo = strings.TrimSuffix(parts[0], "/") @@ -244,6 +342,11 @@ func getRepoAndPkg(v string) (string, string, error) { } func getDest(v, repo, subdir string) (string, error) { + // v is "" for commands that do not require an output path + if v == "" { + return "", nil + } + v = filepath.Clean(v) f, err := os.Stat(v) diff --git a/internal/util/pull/pull.go b/internal/util/pull/pull.go new file mode 100644 index 0000000000..3b68afb35c --- /dev/null +++ b/internal/util/pull/pull.go @@ -0,0 +1,109 @@ +// Copyright 2019 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Package get contains libraries for fetching packages. +package pull + +import ( + "context" + goerrors "errors" + "fmt" + "os" + "path/filepath" + + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/printer" + "github.com/GoogleContainerTools/kpt/internal/types" + "github.com/GoogleContainerTools/kpt/internal/util/remote" + "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" +) + +// Command fetches a package from a git repository, copies it to a local +// directory, and expands any remote subpackages. +type Command struct { + // Contains information about the upstraem package to fetch + Origin remote.Origin + + // Destination is the output directory to clone the package to. Defaults to the name of the package -- + // either the base repo name, or the base subdirectory name. + Destination string +} + +// Run runs the Command. +func (c Command) Run(ctx context.Context) error { + const op errors.Op = "pull.Run" + pr := printer.FromContextOrDie(ctx) + + if err := (&c).DefaultValues(); err != nil { + return errors.E(op, err) + } + + if _, err := os.Stat(c.Destination); !goerrors.Is(err, os.ErrNotExist) { + return errors.E(op, errors.Exist, types.UniquePath(c.Destination), fmt.Errorf("destination directory already exists")) + } + + err := os.MkdirAll(c.Destination, 0700) + if err != nil { + return errors.E(op, errors.IO, types.UniquePath(c.Destination), err) + } + + pr.Printf("Pulling origin %s\n", c.Origin.String()) + + // TODO(oci-support) need to understand abs path for kpt pkg pull from git + _, digest, err := c.Origin.Fetch(ctx, c.Destination) + if err != nil { + return errors.E(op, types.UniquePath(c.Destination), err) + } + + pr.Printf("Pulled digest %s\n", digest) + + kf, err := pkg.ReadKptfile(c.Destination) + if err != nil { + return errors.E(op, types.UniquePath(c.Destination), err) + } + + kf.Origin = c.Origin.Build(digest) + err = kptfileutil.WriteFile(c.Destination, kf) + if err != nil { + return cleanUpDirAndError(c.Destination, err) + } + + return nil +} + +// DefaultValues sets values to the default values if they were unspecified +func (c *Command) DefaultValues() error { + const op errors.Op = "pull.DefaultValues" + if c.Origin == nil { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify git repo or image reference information")) + } + if err := c.Origin.Validate(); err != nil { + return errors.E(op, err) + } + if !filepath.IsAbs(c.Destination) { + return errors.E(op, errors.InvalidParam, fmt.Errorf("destination must be an absolute path")) + } + + return nil +} + +func cleanUpDirAndError(destination string, err error) error { + const op errors.Op = "pull.Run" + rmErr := os.RemoveAll(destination) + if rmErr != nil { + return errors.E(op, types.UniquePath(destination), err, rmErr) + } + return errors.E(op, types.UniquePath(destination), err) +} diff --git a/internal/util/push/push.go b/internal/util/push/push.go new file mode 100644 index 0000000000..fad663c292 --- /dev/null +++ b/internal/util/push/push.go @@ -0,0 +1,159 @@ +// Copyright 2019 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Package get contains libraries for fetching packages. +package push + +import ( + "bytes" + "context" + "fmt" + "strings" + + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/printer" + "github.com/GoogleContainerTools/kpt/internal/util/remote" + "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" + "github.com/Masterminds/semver" +) + +// Command fetches a package from a git repository, copies it to a local +// directory, and expands any remote subpackages. +type Command struct { + // Pkg captures information about the package that should be push. + Pkg *pkg.Pkg + + // Ref is the version to push to origin + Ref string + + // Contains information about the package origin + Origin remote.Origin + + // Increment determines is the version portion of the reference should be increased + Increment bool + + // Origin assigns remote location for push. Ref and Increment will alter parts of this value. + Path string +} + +// Run runs the Command. +func (c Command) Run(ctx context.Context) error { + const op errors.Op = "push.Run" + pr := printer.FromContextOrDie(ctx) + + if err := (&c).DefaultValues(); err != nil { + return errors.E(op, err) + } + + path := c.Pkg.UniquePath.String() + kf, err := c.Pkg.Kptfile() + if err != nil { + return errors.E(op, c.Pkg.UniquePath, err) + } + + if c.Origin == nil { + c.Origin, err = remote.NewOrigin(kf) + if err != nil { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("package must have an origin reference: %v", err)) + } + } + + if c.Ref != "" { + if err := c.Origin.SetRef(c.Ref); err != nil { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("error updating ref: %v", err)) + } + } + + if c.Increment { + // TODO(oci-support) move this logic into a util with test coverage + ref, err := c.Origin.Ref() + if err != nil { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("missing origin version information: %v", err)) + } + + prefix := "" + if ref != "" && ref[:1] == "v" { + prefix = "v" + } + + dotParts := len(strings.SplitN(ref, ".", 3)) + if dotParts > 3 { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("origin version '%s' has more than three dotted parts", ref)) + } + + v, err := semver.NewVersion(ref) + if err != nil { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("unable to increment '%s': %v", ref, err)) + } + + var buf bytes.Buffer + switch dotParts { + case 1: + fmt.Fprintf(&buf, "%s%d", prefix, v.Major()+1) + case 2: + fmt.Fprintf(&buf, "%s%d.%d", prefix, v.Major(), v.Minor()+1) + case 3: + fmt.Fprintf(&buf, "%s%d.%d.%d", prefix, v.Major(), v.Minor(), v.Patch()+1) + } + if v.Prerelease() != "" { + fmt.Fprintf(&buf, "-%s", v.Prerelease()) + } + if v.Metadata() != "" { + fmt.Fprintf(&buf, "+%s", v.Metadata()) + } + + pr.Printf("Incrementing %s to %s\n", ref, buf.String()) + + if err := c.Origin.SetRef(buf.String()); err != nil { + return errors.E(op, c.Pkg.UniquePath, fmt.Errorf("error updating ref: %v", err)) + } + } + + // the kptfile pushed in the package does not have origin data + // this is because the digest will be incorrect. Also, if it is + // pulled from a different location or via different branch, the + // correct origin will be added as part of the pull operation. + kf.Origin = nil + + pr.Printf("Pushing origin %s\n", c.Origin.String()) + + digest, err := c.Origin.Push(ctx, path, kf) + if err != nil { + return errors.E(op, c.Pkg.UniquePath, err) + } + + pr.Printf("Pushed digest %s\n", digest) + + kf.Origin = c.Origin.Build(digest) + err = kptfileutil.WriteFile(path, kf) + if err != nil { + return errors.E(op, c.Pkg.UniquePath, err) + } + + return nil +} + +// DefaultValues sets values to the default values if they were unspecified +func (c *Command) DefaultValues() error { + // const op errors.Op = "pull.DefaultValues" + // if c.Origin == nil { + // return errors.E(op, errors.MissingParam, fmt.Errorf("must specify git repo or image reference information")) + // } + // if err := c.Origin.Validate(); err != nil { + // return errors.E(op, err) + // } + + return nil +} diff --git a/internal/util/remote/git.go b/internal/util/remote/git.go new file mode 100644 index 0000000000..6d99daa479 --- /dev/null +++ b/internal/util/remote/git.go @@ -0,0 +1,402 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "context" + "fmt" + "io/ioutil" + "os" + "path" + "path/filepath" + "strings" + + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/gitutil" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/printer" + "github.com/GoogleContainerTools/kpt/internal/types" + "github.com/GoogleContainerTools/kpt/internal/util/git" + "github.com/GoogleContainerTools/kpt/internal/util/pkgutil" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" + "github.com/otiai10/copy" +) + +type gitUpstream struct { + git *kptfilev1.Git + gitLock *kptfilev1.GitLock +} + +var _ Upstream = &gitUpstream{} + +type gitOrigin struct { + git *kptfilev1.GitLock +} + +var _ Origin = &gitOrigin{} + +func NewGitUpstream(git *kptfilev1.Git) Upstream { + return &gitUpstream{ + git: git, + } +} + +func NewGitOrigin(git *kptfilev1.Git) Origin { + return &gitOrigin{ + git: &kptfilev1.GitLock{ + Repo: git.Repo, + Directory: git.Directory, + Ref: git.Ref, + }, + } +} + +func (u *gitUpstream) String() string { + return fmt.Sprintf("%s@%s", u.git.Repo, u.git.Ref) +} + +func (u *gitUpstream) LockedString() string { + return fmt.Sprintf("%s@%s", u.gitLock.Repo, u.gitLock.Ref) +} + +func (u *gitOrigin) String() string { + return fmt.Sprintf("%s@%s", u.git.Repo, u.git.Ref) +} + +func (u *gitOrigin) LockedString() string { + return fmt.Sprintf("%s@%s", u.git.Repo, u.git.Ref) +} + +func (u *gitUpstream) Validate() error { + const op errors.Op = "remote.Validate" + g := u.git + if g != nil { + if len(g.Repo) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify repo")) + } + if len(g.Ref) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify ref")) + } + if len(g.Directory) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify directory")) + } + } + return nil +} + +func (u *gitOrigin) Validate() error { + const op errors.Op = "remote.Validate" + g := u.git + if g != nil { + if len(g.Repo) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify repo")) + } + if len(g.Ref) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify ref")) + } + if len(g.Directory) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify directory")) + } + } + return nil +} + +func (u *gitUpstream) BuildUpstream() *kptfilev1.Upstream { + repoDir := u.git.Directory + if !strings.HasSuffix(repoDir, "file://") { + repoDir = filepath.Join(path.Split(repoDir)) + } + u.git.Directory = repoDir + + return &kptfilev1.Upstream{ + Type: kptfilev1.GitOrigin, + Git: u.git, + } +} + +func (u *gitUpstream) BuildUpstreamLock(digest string) *kptfilev1.UpstreamLock { + u.gitLock = &kptfilev1.GitLock{ + Repo: u.git.Repo, + Directory: u.git.Directory, + Ref: u.git.Ref, + Commit: digest, + } + return &kptfilev1.UpstreamLock{ + Type: kptfilev1.GitOrigin, + Git: u.gitLock, + } +} + +func (u *gitOrigin) Build(digest string) *kptfilev1.Origin { + return &kptfilev1.Origin{ + Type: kptfilev1.GitOrigin, + Git: &kptfilev1.GitLock{ + Repo: u.git.Repo, + Directory: u.git.Directory, + Ref: u.git.Ref, + Commit: digest, + }, + } +} + +func (u *gitUpstream) FetchUpstream(ctx context.Context, dest string) (string, string, error) { + repoSpec := &git.RepoSpec{ + OrgRepo: u.git.Repo, + Path: u.git.Directory, + Ref: u.git.Ref, + Dir: dest, + } + if err := ClonerUsingGitExec(ctx, repoSpec); err != nil { + return "", "", err + } + return path.Join(repoSpec.Dir, repoSpec.Path), repoSpec.Commit, nil +} + +func (u *gitUpstream) FetchUpstreamLock(ctx context.Context, dest string) (string, error) { + repoSpec := &git.RepoSpec{ + OrgRepo: u.gitLock.Repo, + Path: u.gitLock.Directory, + Ref: u.gitLock.Commit, + Dir: dest, + } + if err := ClonerUsingGitExec(ctx, repoSpec); err != nil { + return "", err + } + return path.Join(repoSpec.Dir, repoSpec.Path), nil +} + +func (u *gitOrigin) Fetch(ctx context.Context, dest string) (string, string, error) { + repoSpec := &git.RepoSpec{ + OrgRepo: u.git.Repo, + Path: u.git.Directory, + Ref: u.git.Ref, + } + if err := ClonerUsingGitExec(ctx, repoSpec); err != nil { + return "", "", err + } + defer os.RemoveAll(repoSpec.Dir) + if err := pkgutil.CopyPackage(repoSpec.AbsPath(), dest, true, pkg.All); err != nil { + return "", "", err + } + + return dest, repoSpec.Commit, nil +} + +func (u *gitUpstream) CloneUpstream(ctx context.Context, dest string) error { + repoSpec := &git.RepoSpec{ + OrgRepo: u.git.Repo, + Path: u.git.Directory, + Ref: u.git.Ref, + } + return cloneAndCopy(ctx, repoSpec, dest) +} + +func (u *gitOrigin) Push(ctx context.Context, dest string, kptfile *kptfilev1.KptFile) (digest string, err error) { + return "", fmt.Errorf("git push not implemented") +} + +func (u *gitUpstream) Ref() (string, error) { + return u.git.Ref, nil +} + +func (u *gitUpstream) SetRef(ref string) error { + u.git.Ref = ref + return nil +} + +func (u *gitOrigin) Ref() (string, error) { + return u.git.Ref, nil +} + +func (u *gitOrigin) SetRef(ref string) error { + u.git.Ref = ref + return nil +} + +// shouldUpdateSubPkgRef checks if subpkg ref should be updated. +// This is true if pkg has the same upstream repo, upstream directory is within or equal to root pkg directory and original root pkg ref matches the subpkg ref. +func (u *gitUpstream) ShouldUpdateSubPkgRef(rootUpstream Upstream, originalRootKfRef string) bool { + root, ok := rootUpstream.(*gitUpstream) + return ok && + u.git.Repo == root.git.Repo && + u.git.Ref == originalRootKfRef && + strings.HasPrefix(path.Clean(u.git.Directory), path.Clean(root.git.Directory)) +} + +// cloneAndCopy fetches the provided repo and copies the content into the +// directory specified by dest. The provided name is set as `metadata.name` +// of the Kptfile of the package. +func cloneAndCopy(ctx context.Context, r *git.RepoSpec, dest string) error { + const op errors.Op = "fetch.cloneAndCopy" + pr := printer.FromContextOrDie(ctx) + + err := ClonerUsingGitExec(ctx, r) + if err != nil { + return errors.E(op, errors.Git, types.UniquePath(dest), err) + } + defer os.RemoveAll(r.Dir) + + sourcePath := filepath.Join(r.Dir, r.Path) + pr.Printf("Adding package %q.\n", strings.TrimPrefix(r.Path, "/")) + if err := pkgutil.CopyPackage(sourcePath, dest, true, pkg.All); err != nil { + return errors.E(op, types.UniquePath(dest), err) + } + + if err := kptfileutil.UpdateKptfileWithoutOrigin(dest, sourcePath, false); err != nil { + return errors.E(op, types.UniquePath(dest), err) + } + + if err := kptfileutil.UpdateUpstreamLockFromGit(dest, r); err != nil { + return errors.E(op, errors.Git, types.UniquePath(dest), err) + } + return nil +} + +// ClonerUsingGitExec uses a local git install, as opposed +// to say, some remote API, to obtain a local clone of +// a remote repo. It looks for tags with the directory as a prefix to allow +// for versioning multiple kpt packages in a single repo independently. It +// relies on the private clonerUsingGitExec function to try fetching different +// refs. +func ClonerUsingGitExec(ctx context.Context, repoSpec *git.RepoSpec) error { + const op errors.Op = "fetch.ClonerUsingGitExec" + + // Create a local representation of the upstream repo. This will initialize + // the cache for the specified repo uri if it isn't already there. It also + // fetches and caches all tag and branch refs from the upstream repo. + upstreamRepo, err := gitutil.NewGitUpstreamRepo(ctx, repoSpec.CloneSpec()) + if err != nil { + return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) + } + + // Check if we have a ref in the upstream that matches the package-specific + // reference. If we do, we use that reference. + ps := strings.Split(repoSpec.Path, "/") + for len(ps) != 0 { + p := path.Join(ps...) + packageRef := path.Join(strings.TrimLeft(p, "/"), repoSpec.Ref) + if _, found := upstreamRepo.ResolveTag(packageRef); found { + repoSpec.Ref = packageRef + break + } + ps = ps[:len(ps)-1] + } + + // Pull the required ref into the repo git cache. + dir, err := upstreamRepo.GetRepo(ctx, []string{repoSpec.Ref}) + if err != nil { + return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) + } + + gitRunner, err := gitutil.NewLocalGitRunner(dir) + if err != nil { + return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) + } + + // Find the commit SHA for the ref that was just fetched. We need the SHA + // rather than the ref to be able to do a hard reset of the cache repo. + commit, found := upstreamRepo.ResolveRef(repoSpec.Ref) + if !found { + commit = repoSpec.Ref + } + + // Reset the local repo to the commit we need. Doing a hard reset instead of + // a checkout means we don't create any local branches so we don't need to + // worry about fast-forwarding them with changes from upstream. It also makes + // sure that any changes in the local worktree are cleaned out. + _, err = gitRunner.Run(ctx, "reset", "--hard", commit) + if err != nil { + gitutil.AmendGitExecError(err, func(e *gitutil.GitExecError) { + e.Repo = repoSpec.CloneSpec() + e.Ref = commit + }) + return errors.E(op, errors.Git, errors.Repo(repoSpec.CloneSpec()), err) + } + + if repoSpec.Dir == "" { + // We need to create a temp directory where we can copy the content of the repo. + // During update, we need to checkout multiple versions of the same repo, so + // we can't do merges directly from the cache. + repoSpec.Dir, err = ioutil.TempDir("", "kpt-get-") + if err != nil { + return errors.E(op, errors.Internal, fmt.Errorf("error creating temp directory: %w", err)) + } + } + repoSpec.Commit = commit + + pkgPath := filepath.Join(dir, repoSpec.Path) + // Verify that the requested path exists in the repo. + _, err = os.Stat(pkgPath) + if os.IsNotExist(err) { + return errors.E(op, + errors.Internal, + err, + fmt.Errorf("path %q does not exist in repo %q", repoSpec.Path, repoSpec.OrgRepo)) + } + + // Copy the content of the pkg into the temp directory. + // Note that we skip the content outside the package directory. + err = copyDir(ctx, pkgPath, repoSpec.AbsPath()) + if err != nil { + return errors.E(op, errors.Internal, fmt.Errorf("error copying package: %w", err)) + } + + // Verify that if a Kptfile exists in the package, it contains the correct + // version of the Kptfile. + _, err = pkg.ReadKptfile(pkgPath) + if err != nil { + // A Kptfile isn't required, so it is fine if there is no Kptfile. + if errors.Is(err, os.ErrNotExist) { + return nil + } + + // If the error is of type KptfileError, we replace it with a + // RemoteKptfileError. This allows us to provide information about the + // git source of the Kptfile instead of the path to some random + // temporary directory. + var kfError *pkg.KptfileError + if errors.As(err, &kfError) { + return &pkg.RemoteKptfileError{ + RepoSpec: repoSpec, + Err: kfError.Err, + } + } + } + return nil +} + +// copyDir copies a src directory to a dst directory. +// copyDir skips copying the .git directory from the src and ignores symlinks. +func copyDir(ctx context.Context, srcDir string, dstDir string) error { + pr := printer.FromContextOrDie(ctx) + opts := copy.Options{ + Skip: func(src string) (bool, error) { + return strings.HasSuffix(src, ".git"), nil + }, + OnSymlink: func(src string) copy.SymlinkAction { + // try to print relative path of symlink + // if we can, else absolute path which is not + // pretty because it contains path to temporary repo dir + displayPath, err := filepath.Rel(srcDir, src) + if err != nil { + displayPath = src + } + pr.Printf("[Warn] Ignoring symlink %q \n", displayPath) + return copy.Skip + }, + } + return copy.Copy(srcDir, dstDir, opts) +} diff --git a/internal/util/remote/git_test.go b/internal/util/remote/git_test.go new file mode 100644 index 0000000000..4a5cd6970d --- /dev/null +++ b/internal/util/remote/git_test.go @@ -0,0 +1,76 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "reflect" + "testing" + + v1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" +) + +//nolint:scopelint +func TestNewGitUpstream(t *testing.T) { + type args struct { + git *v1.Git + } + tests := []struct { + name string + args args + want Upstream + }{ + // TODO: Add test cases. + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := NewGitUpstream(tt.args.git); !reflect.DeepEqual(got, tt.want) { + t.Errorf("NewGitUpstream() = %v, want %v", got, tt.want) + } + }) + } +} + +//nolint:scopelint +func Test_gitUpstream_String(t *testing.T) { + type fields struct { + git *v1.Git + } + tests := []struct { + name string + fields fields + want string + }{ + { + name: "Returns repo ref string", + fields: fields{ + git: &v1.Git{ + Repo: "https://hostname/repo.git", + Ref: "main", + }, + }, + want: "https://hostname/repo.git@main", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + u := &gitUpstream{ + git: tt.fields.git, + } + if got := u.String(); got != tt.want { + t.Errorf("gitUpstream.String() = %v, want %v", got, tt.want) + } + }) + } +} diff --git a/internal/util/remote/oci.go b/internal/util/remote/oci.go new file mode 100644 index 0000000000..7363ce4568 --- /dev/null +++ b/internal/util/remote/oci.go @@ -0,0 +1,461 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "archive/tar" + "bytes" + "compress/gzip" + "context" + "fmt" + "io" + "io/fs" + "io/ioutil" + "os" + "path" + "path/filepath" + "strings" + + "github.com/GoogleContainerTools/kpt/internal/errors" + "github.com/GoogleContainerTools/kpt/internal/pkg" + "github.com/GoogleContainerTools/kpt/internal/types" + "github.com/GoogleContainerTools/kpt/internal/util/pkgutil" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" + "github.com/google/go-containerregistry/pkg/crane" + "github.com/google/go-containerregistry/pkg/gcrane" + "github.com/google/go-containerregistry/pkg/name" + "github.com/google/go-containerregistry/pkg/v1/empty" + "github.com/google/go-containerregistry/pkg/v1/mutate" + "github.com/google/go-containerregistry/pkg/v1/remote" +) + +type ociUpstream struct { + oci *kptfilev1.Oci + ociLock *kptfilev1.OciLock +} + +var _ Upstream = &ociUpstream{} + +type ociOrigin struct { + oci *kptfilev1.OciLock +} + +var _ Origin = &ociOrigin{} + +func NewOciUpstream(oci *kptfilev1.Oci) Upstream { + return &ociUpstream{ + oci: oci, + } +} + +func NewOciOrigin(oci *kptfilev1.Oci) Origin { + return &ociOrigin{ + oci: &kptfilev1.OciLock{ + Image: oci.Image, + }, + } +} + +func (u *ociUpstream) String() string { + return u.oci.Image +} + +func (u *ociUpstream) LockedString() string { + return u.ociLock.Digest +} + +func (u *ociOrigin) String() string { + return u.oci.Image +} + +func (u *ociOrigin) LockedString() string { + return u.oci.Digest +} + +func (u *ociUpstream) BuildUpstream() *kptfilev1.Upstream { + return &kptfilev1.Upstream{ + Type: kptfilev1.OciOrigin, + Oci: u.oci, + } +} + +func (u *ociUpstream) BuildUpstreamLock(digest string) *kptfilev1.UpstreamLock { + u.ociLock.Image = u.oci.Image + u.ociLock.Directory = u.oci.Directory + u.ociLock.Digest = digest + + return &kptfilev1.UpstreamLock{ + Type: kptfilev1.OciOrigin, + Oci: u.ociLock, + } +} + +func (u *ociOrigin) Build(digest string) *kptfilev1.Origin { + return &kptfilev1.Origin{ + Type: kptfilev1.OciOrigin, + Oci: &kptfilev1.OciLock{ + Image: u.oci.Image, + Digest: digest, + }, + } +} + +func (u *ociUpstream) Validate() error { + const op errors.Op = "remote.Validate" + if u.oci != nil { + if len(u.oci.Image) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify image")) + } + } + return nil +} + +func (u *ociOrigin) Validate() error { + const op errors.Op = "remote.Validate" + if u.oci != nil { + if len(u.oci.Image) == 0 { + return errors.E(op, errors.MissingParam, fmt.Errorf("must specify image")) + } + } + return nil +} + +func (u *ociUpstream) FetchUpstream(ctx context.Context, dest string) (string, string, error) { + const op errors.Op = "remote.FetchUpstream" + imageDigest, err := pullAndExtract(u.oci.Image, dest, remote.WithContext(ctx), remote.WithAuthFromKeychain(gcrane.Keychain)) + if err != nil { + return "", "", errors.E(op, errors.OCI, types.UniquePath(dest), err) + } + return path.Join(dest, u.oci.Directory), imageDigest.Name(), nil +} + +func (u *ociUpstream) FetchUpstreamLock(ctx context.Context, dest string) (string, error) { + const op errors.Op = "remote.FetchUpstreamLock" + _, err := pullAndExtract(u.ociLock.Digest, dest, remote.WithContext(ctx), remote.WithAuthFromKeychain(gcrane.Keychain)) + if err != nil { + return "", errors.E(op, errors.OCI, types.UniquePath(dest), err) + } + return path.Join(dest, u.ociLock.Directory), nil +} + +func (u *ociOrigin) Fetch(ctx context.Context, dest string) (string, string, error) { + const op errors.Op = "remote.Fetch" + imageDigest, err := pullAndExtract(u.oci.Image, dest, remote.WithContext(ctx), remote.WithAuthFromKeychain(gcrane.Keychain)) + if err != nil { + return "", "", errors.E(op, errors.OCI, types.UniquePath(dest), err) + } + return path.Join(dest, u.oci.Directory), imageDigest.Name(), nil +} + +func (u *ociUpstream) CloneUpstream(ctx context.Context, dest string) error { + const op errors.Op = "remote.FetchUpstreamClone" + // pr := printer.FromContextOrDie(ctx) + + // We need to create a temp directory where we can copy the content of the repo. + // During update, we need to checkout multiple versions of the same repo, so + // we can't do merges directly from the cache. + dir, err := ioutil.TempDir("", "kpt-get-") + if err != nil { + return errors.E(op, errors.Internal, fmt.Errorf("error creating temp directory: %w", err)) + } + defer os.RemoveAll(dir) + + imageDigest, err := pullAndExtract(u.oci.Image, dir, remote.WithContext(ctx), remote.WithAuthFromKeychain(gcrane.Keychain)) + if err != nil { + return errors.E(op, errors.OCI, types.UniquePath(dest), err) + } + + sourcePath := path.Join(dir, u.oci.Directory) + if err := pkgutil.CopyPackage(sourcePath, dest, true, pkg.All); err != nil { + return errors.E(op, types.UniquePath(dest), err) + } + + if err := kptfileutil.UpdateKptfileWithoutOrigin(dest, sourcePath, false); err != nil { + return errors.E(op, types.UniquePath(dest), err) + } + + if err := kptfileutil.UpdateUpstreamLock(dest, u.BuildUpstreamLock(imageDigest.String())); err != nil { + return errors.E(op, errors.OCI, types.UniquePath(dest), err) + } + + return nil +} + +func (u *ociOrigin) Push(ctx context.Context, source string, kptfile *kptfilev1.KptFile) (digest string, err error) { + const op errors.Op = "remote.Push" + + imageDigest, err := archiveAndPush(u.oci.Image, source, kptfile, remote.WithContext(ctx), remote.WithAuthFromKeychain(gcrane.Keychain)) + if err != nil { + return "", errors.E(op, errors.OCI, types.UniquePath(source), err) + } + + return imageDigest.String(), nil +} + +func (u *ociUpstream) Ref() (string, error) { + const op errors.Op = "remote.Ref" + r, err := name.ParseReference(u.oci.Image) + if err != nil { + return "", errors.E(op, errors.Internal, fmt.Errorf("error parsing reference: %s %w", u.oci.Image, err)) + } + return r.Identifier(), nil +} + +func (u *ociUpstream) SetRef(ref string) error { + const op errors.Op = "remote.SetRef" + r, err := name.ParseReference(u.oci.Image) + if err != nil { + return errors.E(op, errors.Internal, fmt.Errorf("error parsing reference: %s %w", u.oci.Image, err)) + } + + if len(strings.SplitN(ref, "sha256:", 2)[0]) == 0 { + u.oci.Image = r.Context().Digest(ref).Name() + } else { + u.oci.Image = r.Context().Tag(ref).Name() + } + + return nil +} + +func (u *ociOrigin) Ref() (string, error) { + const op errors.Op = "remote.Ref" + r, err := name.ParseReference(u.oci.Image) + if err != nil { + return "", errors.E(op, errors.Internal, fmt.Errorf("error parsing reference: %s %w", u.oci.Image, err)) + } + return r.Identifier(), nil +} + +func (u *ociOrigin) SetRef(ref string) error { + const op errors.Op = "remote.SetRef" + r, err := name.ParseReference(u.oci.Image) + if err != nil { + return errors.E(op, errors.Internal, fmt.Errorf("error parsing reference: %s %w", u.oci.Image, err)) + } + + if len(strings.SplitN(ref, "sha256:", 2)[0]) == 0 { + u.oci.Image = r.Context().Digest(ref).Name() + } else { + u.oci.Image = r.Context().Tag(ref).Name() + } + + return nil +} + +// shouldUpdateSubPkgRef checks if subpkg ref should be updated. +// This is true if pkg has the same upstream repo, upstream directory is within or equal to root pkg directory and original root pkg ref matches the subpkg ref. +func (u *ociUpstream) ShouldUpdateSubPkgRef(rootUpstream Upstream, originalRootKfRef string) bool { + root, ok := rootUpstream.(*ociUpstream) + if !ok { + return false + } + subName, err := name.ParseReference(u.oci.Image) + if err != nil { + return false + } + rootName, err := name.ParseReference(root.oci.Image) + if err != nil { + return false + } + return subName.Context().String() == rootName.Context().String() && + subName.Identifier() == originalRootKfRef +} + +// pullAndExtract uses current credentials (gcloud auth) to pull and +// extract (untar) image files to target directory. The desired version or digest must +// be in the imageName, and the resolved image sha256 digest is returned. +func pullAndExtract(imageName string, dir string, options ...remote.Option) (name.Reference, error) { + const op errors.Op = "remote.pullAndExtract" + + ref, err := name.ParseReference(imageName) + if err != nil { + return nil, fmt.Errorf("parsing reference %q: %v", imageName, err) + } + + // Pull image from source using provided options for auth credentials + image, err := remote.Image(ref, options...) + if err != nil { + return nil, fmt.Errorf("pulling image %s: %v", imageName, err) + } + + // Stream image files as if single tar (merged layers) + ioReader := mutate.Extract(image) + defer ioReader.Close() + + // Write contents to target dir + // TODO look for a more robust example of an untar loop + tarReader := tar.NewReader(ioReader) + for { + hdr, err := tarReader.Next() + if err == io.EOF { + break + } + if err != nil { + return nil, err + } + path := filepath.Join(dir, hdr.Name) + switch { + case hdr.FileInfo().IsDir(): + if err := os.MkdirAll(path, hdr.FileInfo().Mode()); err != nil { + return nil, err + } + case hdr.Linkname != "": + if err := os.Symlink(hdr.Linkname, path); err != nil { + // just warn for now + fmt.Fprintln(os.Stderr, err) + // return err + } + default: + file, err := os.OpenFile(path, + os.O_WRONLY|os.O_CREATE|os.O_TRUNC, + os.FileMode(hdr.Mode), + ) + if err != nil { + return nil, err + } + defer file.Close() + + _, err = io.Copy(file, tarReader) + if err != nil { + return nil, err + } + } + } + + // Determine the digest of the image that was extracted + imageDigestHash, err := image.Digest() + if err != nil { + return nil, errors.E(op, fmt.Errorf("error calculating image digest: %w", err)) + } + imageDigest := ref.Context().Digest("sha256:" + imageDigestHash.Hex) + + // Return the image with digest when successful, needed for upstreamLock + return imageDigest, nil +} + +// archiveAndPush uses current credentials (gcloud auth) to tar and +// extract (untar) image files to target directory. The desired version or digest must +// be in the imageName, and the resolved image sha256 digest is returned. +func archiveAndPush(imageName string, dir string, kptfile *kptfilev1.KptFile, options ...remote.Option) (name.Reference, error) { + const op errors.Op = "remote.archiveAndPush" + + ref, err := name.ParseReference(imageName) + if err != nil { + return nil, fmt.Errorf("parsing reference %q: %v", imageName, err) + } + + // Make new layer + tarFile, err := ioutil.TempFile("", "tar") + if err != nil { + return nil, err + } + defer os.Remove(tarFile.Name()) + + if err := func() error { + defer tarFile.Close() + + gw := gzip.NewWriter(tarFile) + defer gw.Close() + + tw := tar.NewWriter(gw) + defer tw.Close() + + if err := filepath.Walk(dir, func(path string, info fs.FileInfo, err error) error { + if err != nil { + return err + } + + relative, err := filepath.Rel(dir, path) + if err != nil { + return err + } + if info.IsDir() && relative == "." { + return nil + } + + // TODO(oci-support) if info is symlink also read link target + link := "" + + // generate tar header + header, err := tar.FileInfoHeader(info, link) + if err != nil { + return err + } + + // must provide real name + // (see https://golang.org/src/archive/tar/common.go?#L626) + header.Name = filepath.ToSlash(relative) + + var buf *bytes.Buffer + if strings.EqualFold(header.Name, "Kptfile") { + buf = &bytes.Buffer{} + if err := kptfileutil.Write(buf, kptfile); err != nil { + return err + } + header.Size = int64(buf.Len()) + } + + // write header + if err := tw.WriteHeader(header); err != nil { + return err + } + // if not a dir, write file content + if !info.IsDir() { + data, err := os.Open(path) + if err != nil { + return err + } + if buf != nil { + if _, err := io.Copy(tw, buf); err != nil { + return err + } + } else { + if _, err := io.Copy(tw, data); err != nil { + return err + } + } + } + return nil + }); err != nil { + return err + } + + return nil + }(); err != nil { + return nil, err + } + + // Append new layer + newLayers := []string{tarFile.Name()} + img, err := crane.Append(empty.Image, newLayers...) + if err != nil { + return nil, fmt.Errorf("appending %v: %v", newLayers, err) + } + + if err := remote.Write(ref, img, options...); err != nil { + return nil, fmt.Errorf("pushing image %s: %v", ref, err) + } + + // Determine the digest of the image that was pushed + imageDigestHash, err := img.Digest() + if err != nil { + return nil, errors.E(op, fmt.Errorf("error calculating image digest: %w", err)) + } + imageDigest := ref.Context().Digest("sha256:" + imageDigestHash.Hex) + + // Return the image with digest when successful, needed for upstreamLock + return imageDigest, nil +} diff --git a/internal/util/remote/origin.go b/internal/util/remote/origin.go new file mode 100644 index 0000000000..9578fdaeec --- /dev/null +++ b/internal/util/remote/origin.go @@ -0,0 +1,63 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "context" + "fmt" + + "github.com/GoogleContainerTools/kpt/internal/errors" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" +) + +type Origin interface { + fmt.Stringer + LockedString() string + + Validate() error + + Build(digest string) *kptfilev1.Origin + + Fetch(ctx context.Context, dest string) (absPath string, digest string, err error) + Push(ctx context.Context, dest string, kptfile *kptfilev1.KptFile) (digest string, err error) + + Ref() (string, error) + SetRef(ref string) error +} + +func NewOrigin(kf *kptfilev1.KptFile) (Origin, error) { + const op errors.Op = "remote.NewOrigin" + if kf != nil && kf.Origin != nil { + switch kf.Origin.Type { + case kptfilev1.GitOrigin: + if kf.Upstream.Git == nil { + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile origin must have git information")) + } + u := &gitOrigin{ + git: kf.Origin.Git, + } + return u, nil + case kptfilev1.OciOrigin: + if kf.Origin.Oci == nil { + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile origin must have oci information")) + } + u := &ociOrigin{ + oci: kf.Origin.Oci, + } + return u, nil + } + } + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile origin type must be one of: %s,%s", kptfilev1.GitOrigin, kptfilev1.OciOrigin)) +} diff --git a/internal/util/remote/upstream.go b/internal/util/remote/upstream.go new file mode 100644 index 0000000000..e73a1d9a38 --- /dev/null +++ b/internal/util/remote/upstream.go @@ -0,0 +1,79 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "context" + "fmt" + + "github.com/GoogleContainerTools/kpt/internal/errors" + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" +) + +type Upstream interface { + fmt.Stringer + LockedString() string + + Validate() error + + BuildUpstream() *kptfilev1.Upstream + BuildUpstreamLock(digest string) *kptfilev1.UpstreamLock + + FetchUpstream(ctx context.Context, dest string) (absPath string, digest string, err error) + FetchUpstreamLock(ctx context.Context, dest string) (absPath string, err error) + + CloneUpstream(ctx context.Context, dest string) error + + Ref() (string, error) + SetRef(ref string) error + ShouldUpdateSubPkgRef(rootUpstream Upstream, originalRootRef string) bool +} + +func NewUpstream(kf *kptfilev1.KptFile) (Upstream, error) { + const op errors.Op = "remote.NewUpstream" + if kf != nil && kf.Upstream != nil { + switch kf.Upstream.Type { + case kptfilev1.GitOrigin: + if kf.Upstream.Git == nil { + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream must have git information")) + } + u := &gitUpstream{ + git: kf.Upstream.Git, + gitLock: &kptfilev1.GitLock{}, + } + if kf.UpstreamLock != nil && kf.UpstreamLock.Git != nil { + u.gitLock = kf.UpstreamLock.Git + } + return u, nil + case kptfilev1.OciOrigin: + if kf.Upstream.Oci == nil { + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream must have oci information")) + } + u := &ociUpstream{ + oci: kf.Upstream.Oci, + ociLock: &kptfilev1.OciLock{}, + } + if kf.UpstreamLock != nil && kf.UpstreamLock.Oci != nil { + u.ociLock = kf.UpstreamLock.Oci + } + return u, nil + } + } + return nil, errors.E(op, errors.MissingParam, fmt.Errorf("kptfile upstream type must be one of: %s,%s", kptfilev1.GitOrigin, kptfilev1.OciOrigin)) +} + +func ShouldUpdateSubPkgRef(subUpstream Upstream, rootUpstream Upstream, originalRootRef string) bool { + return subUpstream.ShouldUpdateSubPkgRef(rootUpstream, originalRootRef) +} diff --git a/internal/util/remote/upstream_test.go b/internal/util/remote/upstream_test.go new file mode 100644 index 0000000000..8a7e8183e3 --- /dev/null +++ b/internal/util/remote/upstream_test.go @@ -0,0 +1,134 @@ +// Copyright 2021 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package remote + +import ( + "reflect" + "testing" + + kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" +) + +//nolint:scopelint +func TestNewUpstream(t *testing.T) { + type args struct { + kf *kptfilev1.KptFile + } + tests := []struct { + name string + args args + want Upstream + wantErr bool + }{ + { + name: "returns git upstream", + args: args{ + kf: &kptfilev1.KptFile{ + Upstream: &kptfilev1.Upstream{ + Type: kptfilev1.GitOrigin, + Git: &kptfilev1.Git{ + Repo: "repo-name", + Directory: "dir-name", + Ref: "ref-name", + }, + }, + }, + }, + want: &gitUpstream{ + git: &kptfilev1.Git{ + Repo: "repo-name", + Directory: "dir-name", + Ref: "ref-name", + }, + gitLock: &kptfilev1.GitLock{}, + }, + wantErr: false, + }, + { + name: "returns oci upstream", + args: args{ + kf: &kptfilev1.KptFile{ + Upstream: &kptfilev1.Upstream{ + Type: kptfilev1.OciOrigin, + Oci: &kptfilev1.Oci{ + Image: "image-name", + }, + }, + }, + }, + want: &ociUpstream{ + oci: &kptfilev1.Oci{ + Image: "image-name", + }, + ociLock: &kptfilev1.OciLock{}, + }, + wantErr: false, + }, + { + name: "empty type fails", + args: args{ + kf: &kptfilev1.KptFile{ + Upstream: &kptfilev1.Upstream{}, + }, + }, + want: nil, + wantErr: true, + }, + { + name: "nil upstream fails", + args: args{ + kf: &kptfilev1.KptFile{}, + }, + want: nil, + wantErr: true, + }, + { + name: "nil git fails", + args: args{ + kf: &kptfilev1.KptFile{ + Upstream: &kptfilev1.Upstream{ + Type: kptfilev1.GitOrigin, + }, + }, + }, + want: nil, + wantErr: true, + }, + { + name: "nil oci fails", + args: args{ + kf: &kptfilev1.KptFile{ + Upstream: &kptfilev1.Upstream{ + Type: kptfilev1.OciOrigin, + }, + }, + }, + want: nil, + wantErr: true, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := NewUpstream(tt.args.kf) + if (err != nil) != tt.wantErr { + t.Errorf("NewUpstream() error = %v, wantErr %v", err, tt.wantErr) + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Errorf("NewUpstream() = %v, want %v", got, tt.want) + } + }) + } +} diff --git a/internal/util/update/common.go b/internal/util/update/common.go index 51308708a3..f62d4885f1 100644 --- a/internal/util/update/common.go +++ b/internal/util/update/common.go @@ -39,7 +39,7 @@ func PkgHasUpdatedUpstream(local, origin string) (bool, error) { // If the upstream information in local has changed from origin, it // means the user had updated the package independently and we don't // want to override it. - if !reflect.DeepEqual(localKf.Upstream.Git, originKf.Upstream.Git) { + if !reflect.DeepEqual(localKf.Upstream, originKf.Upstream) { return true, nil } return false, nil diff --git a/internal/util/update/update.go b/internal/util/update/update.go index d6e54b236e..74b417caa3 100644 --- a/internal/util/update/update.go +++ b/internal/util/update/update.go @@ -18,20 +18,16 @@ package update import ( "context" "fmt" - "io/ioutil" "os" - "path" "path/filepath" - "strings" "github.com/GoogleContainerTools/kpt/internal/errors" "github.com/GoogleContainerTools/kpt/internal/pkg" "github.com/GoogleContainerTools/kpt/internal/printer" "github.com/GoogleContainerTools/kpt/internal/types" "github.com/GoogleContainerTools/kpt/internal/util/addmergecomment" - "github.com/GoogleContainerTools/kpt/internal/util/fetch" - "github.com/GoogleContainerTools/kpt/internal/util/git" "github.com/GoogleContainerTools/kpt/internal/util/pkgutil" + "github.com/GoogleContainerTools/kpt/internal/util/remote" "github.com/GoogleContainerTools/kpt/internal/util/stack" kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" "github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil" @@ -117,13 +113,19 @@ func (u Command) Run(ctx context.Context) error { return errors.E(op, u.Pkg.UniquePath, err) } - if rootKf.Upstream == nil || rootKf.Upstream.Git == nil { - return errors.E(op, u.Pkg.UniquePath, - fmt.Errorf("package must have an upstream reference")) + rootUps, err := remote.NewUpstream(rootKf) + if err != nil { + return errors.E(op, u.Pkg.UniquePath, fmt.Errorf("package must have an upstream reference: %v", err)) + } + + originalRootKfRef, err := rootUps.Ref() + if err != nil { + return errors.E(op, u.Pkg.UniquePath, err) } - originalRootKfRef := rootKf.Upstream.Git.Ref if u.Ref != "" { - rootKf.Upstream.Git.Ref = u.Ref + if err := rootUps.SetRef(u.Ref); err != nil { + return errors.E(op, u.Pkg.UniquePath, err) + } } if u.Strategy != "" { rootKf.Upstream.UpdateStrategy = u.Strategy @@ -158,13 +160,14 @@ func (u Command) Run(ctx context.Context) error { return errors.E(op, p.UniquePath, err) } - if subKf.Upstream != nil && subKf.Upstream.Git != nil { + if subUps, err := remote.NewUpstream(subKf); err == nil { // update subpackage kf ref/strategy if current pkg is a subpkg of root pkg or is root pkg // and if original root pkg ref matches the subpkg ref - if shouldUpdateSubPkgRef(subKf, rootKf, originalRootKfRef) { - updateSubKf(subKf, u.Ref, u.Strategy) - err = kptfileutil.WriteFile(subPkg.UniquePath.String(), subKf) - if err != nil { + if remote.ShouldUpdateSubPkgRef(subUps, rootUps, originalRootKfRef) { + if err := updateSubKf(subKf, subUps, u.Ref, u.Strategy); err != nil { + return errors.E(op, subPkg.UniquePath, err) + } + if err = kptfileutil.WriteFile(subPkg.UniquePath.String(), subKf); err != nil { return errors.E(op, subPkg.UniquePath, err) } } @@ -182,14 +185,17 @@ func (u Command) Run(ctx context.Context) error { } // updateSubKf updates subpackage with given ref and update strategy -func updateSubKf(subKf *kptfilev1.KptFile, ref string, strategy kptfilev1.UpdateStrategyType) { +func updateSubKf(subKf *kptfilev1.KptFile, subUps remote.Upstream, ref string, strategy kptfilev1.UpdateStrategyType) error { // check if explicit ref provided if ref != "" { - subKf.Upstream.Git.Ref = ref + if err := subUps.SetRef(ref); err != nil { + return err + } } if strategy != "" { subKf.Upstream.UpdateStrategy = strategy } +<<<<<<< HEAD } // shouldUpdateSubPkgRef checks if subpkg ref should be updated. @@ -231,6 +237,9 @@ type nilRepoClone struct { // the nilRepoClone, this will always be an empty directory. func (nrc *nilRepoClone) AbsPath() string { return nrc.dir +======= + return nil +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) } // updateRootPackage updates a local package. It will use the information @@ -246,30 +255,36 @@ func (u Command) updateRootPackage(ctx context.Context, p *pkg.Pkg) error { pr := printer.FromContextOrDie(ctx) pr.PrintPackage(p, !(p == u.Pkg)) - g := kf.Upstream.Git - updated := &git.RepoSpec{OrgRepo: g.Repo, Path: g.Directory, Ref: g.Ref} - pr.Printf("Fetching upstream from %s@%s\n", kf.Upstream.Git.Repo, kf.Upstream.Git.Ref) - if err := fetch.ClonerUsingGitExec(ctx, updated); err != nil { + upstream, err := remote.NewUpstream(kf) + if err != nil { + return errors.E(op, p.UniquePath, err) + } + + updatedTmpPath, err := os.MkdirTemp("", "kpt-updated-") + if err != nil { return errors.E(op, p.UniquePath, err) } - defer os.RemoveAll(updated.AbsPath()) + defer os.RemoveAll(updatedTmpPath) + + pr.Printf("Fetching upstream from %s\n", upstream.String()) + updatedAbsPath, commit, err := upstream.FetchUpstream(ctx, updatedTmpPath) + if err != nil { + return errors.E(op, p.UniquePath, err) + } + + originTmpPath, err := os.MkdirTemp("", "kpt-origin-") + if err != nil { + return errors.E(op, p.UniquePath, err) + } + defer os.RemoveAll(originTmpPath) + originAbsPath := originTmpPath - var origin repoClone if kf.UpstreamLock != nil { - gLock := kf.UpstreamLock.Git - originRepoSpec := &git.RepoSpec{OrgRepo: gLock.Repo, Path: gLock.Directory, Ref: gLock.Commit} - pr.Printf("Fetching origin from %s@%s\n", kf.Upstream.Git.Repo, kf.Upstream.Git.Ref) - if err := fetch.ClonerUsingGitExec(ctx, originRepoSpec); err != nil { - return errors.E(op, p.UniquePath, err) - } - origin = originRepoSpec - } else { - origin, err = newNilRepoClone() - if err != nil { + pr.Printf("Fetching origin from %s\n", upstream.LockedString()) + if originAbsPath, err = upstream.FetchUpstreamLock(ctx, originTmpPath); err != nil { return errors.E(op, p.UniquePath, err) } } - defer os.RemoveAll(origin.AbsPath()) s := stack.New() s.Push(".") @@ -277,8 +292,8 @@ func (u Command) updateRootPackage(ctx context.Context, p *pkg.Pkg) error { for s.Len() > 0 { relPath := s.Pop() localPath := filepath.Join(p.UniquePath.String(), relPath) - updatedPath := filepath.Join(updated.AbsPath(), relPath) - originPath := filepath.Join(origin.AbsPath(), relPath) + updatedPath := filepath.Join(updatedAbsPath, relPath) + originPath := filepath.Join(originAbsPath, relPath) isRoot := false if relPath == "." { @@ -299,7 +314,7 @@ func (u Command) updateRootPackage(ctx context.Context, p *pkg.Pkg) error { } } - if err := kptfileutil.UpdateUpstreamLockFromGit(p.UniquePath.String(), updated); err != nil { + if err := kptfileutil.UpdateUpstreamLock(p.UniquePath.String(), upstream.BuildUpstreamLock(commit)); err != nil { return errors.E(op, p.UniquePath, err) } return nil diff --git a/pkg/api/kptfile/v1/types.go b/pkg/api/kptfile/v1/types.go index 9d483bc305..6fee06dbf4 100644 --- a/pkg/api/kptfile/v1/types.go +++ b/pkg/api/kptfile/v1/types.go @@ -49,6 +49,9 @@ type KptFile struct { // UpstreamLock is a resolved locator for the last fetch of the package. UpstreamLock *UpstreamLock `yaml:"upstreamLock,omitempty" json:"upstreamLock,omitempty"` + // Origin is a resolved locator for the last pull or push of the package. + Origin *Origin `yaml:"origin,omitempty" json:"origin,omitempty"` + // Info contains metadata such as license, documentation, etc. Info *PackageInfo `yaml:"info,omitempty" json:"info,omitempty"` @@ -65,6 +68,9 @@ type OriginType string const ( // GitOrigin specifies a package as having been cloned from a git repository. GitOrigin OriginType = "git" + + // OciOrigin specifies a package as having been pulled from an OCI image repository. + OciOrigin OriginType = "oci" ) // UpdateStrategyType defines the strategy for updating a package from upstream. @@ -121,6 +127,9 @@ type Upstream struct { // Git is the locator for a package stored on Git. Git *Git `yaml:"git,omitempty" json:"git,omitempty"` + // Oci is the locator for a package stored in an OCI image registry. + Oci *Oci `yaml:"oci,omitempty" json:"oci,omitempty"` + // UpdateStrategy declares how a package will be updated from upstream. UpdateStrategy UpdateStrategyType `yaml:"updateStrategy,omitempty" json:"updateStrategy,omitempty"` } @@ -139,6 +148,16 @@ type Git struct { Ref string `yaml:"ref,omitempty" json:"ref,omitempty"` } +// Oci is the user-specified locator for a package in an OCI image registry. +type Oci struct { + // Image is the OCI image repository for the package. + // e.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend:latest' + Image string `yaml:"image,omitempty" json:"image,omitempty"` + + // Directory is the sub-package of the image. + Directory string `yaml:"path,omitempty" json:"path,omitempty"` +} + // UpstreamLock is a resolved locator for the last fetch of the package. type UpstreamLock struct { // Type is the type of origin. @@ -146,6 +165,21 @@ type UpstreamLock struct { // Git is the resolved locator for a package on Git. Git *GitLock `yaml:"git,omitempty" json:"git,omitempty"` + + // Oci is the resolved locator for a package in an OCI image registry. + Oci *OciLock `yaml:"oci,omitempty" json:"oci,omitempty"` +} + +// Origin is a resolved locator when the package was last pulled or pushed. +type Origin struct { + // Type is the type of origin. + Type OriginType `yaml:"type,omitempty" json:"type,omitempty"` + + // Git is the resolved locator for a package on Git. + Git *GitLock `yaml:"git,omitempty" json:"git,omitempty"` + + // Oci is the resolved locator for a package in an OCI image registry. + Oci *OciLock `yaml:"oci,omitempty" json:"oci,omitempty"` } // GitLock is the resolved locator for a package on Git. @@ -167,6 +201,19 @@ type GitLock struct { Commit string `yaml:"commit,omitempty" json:"commit,omitempty"` } +// OciLock is the resolved locator for a package in an OCI image registry. +type OciLock struct { + // Image is the OCI image repository for the package. + // e.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend@sha256:b0c94f11d856e59673daca566857a7ead126ef8e2b6915ed662804f858d7eaea' + Image string `yaml:"image,omitempty" json:"image,omitempty"` + + // Directory is the sub-package of the image. + Directory string `yaml:"path,omitempty" json:"path,omitempty"` + + // Digest is the unique sha of the image when it was last pulled. + Digest string `yaml:"digest,omitempty" json:"digest,omitempty"` +} + // PackageInfo contains optional information about the package such as license, documentation, etc. // These fields are not consumed by any functionality in kpt and are simply passed through. // Note that like any other KRM resource, humans and automation can also use `metadata.labels` and diff --git a/pkg/kptfile/kptfileutil/util.go b/pkg/kptfile/kptfileutil/util.go index 51521b68e7..7bb22f6e97 100644 --- a/pkg/kptfile/kptfileutil/util.go +++ b/pkg/kptfile/kptfileutil/util.go @@ -18,6 +18,7 @@ import ( "bytes" goerrors "errors" "fmt" + "io" "io/ioutil" "os" "path/filepath" @@ -51,6 +52,20 @@ func WriteFile(dir string, k *kptfilev1.KptFile) error { return nil } +func Write(dst io.Writer, k *kptfilev1.KptFile) error { + const op errors.Op = "kptfileutil.WriteFile" + b, err := yaml.MarshalWithOptions(k, &yaml.EncoderOptions{SeqIndent: yaml.WideSequenceStyle}) + if err != nil { + return err + } + + if _, err := dst.Write(b); err != nil { + return errors.E(op, errors.IO, err) + } + + return nil +} + // ValidateInventory returns true and a nil error if the passed inventory // is valid; otherwiste, false and the reason the inventory is not valid // is returned. A valid inventory must have a non-empty namespace, name, @@ -195,12 +210,8 @@ func UpdateKptfile(localPath, updatedPath, originPath string, updateUpstream boo return nil } -// UpdateUpstreamLockFromGit updates the upstreamLock of the package specified -// by path by using the values from spec. It will also populate the commit -// field in upstreamLock using the latest commit of the git repo given -// by spec. -func UpdateUpstreamLockFromGit(path string, spec *git.RepoSpec) error { - const op errors.Op = "kptfileutil.UpdateUpstreamLockFromGit" +func UpdateUpstreamLock(path string, upstreamLock *kptfilev1.UpstreamLock) error { + const op errors.Op = "kptfileutil.UpdateUpstreamLock" // read KptFile cloned with the package if it exists kpgfile, err := pkg.ReadKptfile(path) if err != nil { @@ -208,7 +219,21 @@ func UpdateUpstreamLockFromGit(path string, spec *git.RepoSpec) error { } // populate the cloneFrom values so we know where the package came from - kpgfile.UpstreamLock = &kptfilev1.UpstreamLock{ + kpgfile.UpstreamLock = upstreamLock + + err = WriteFile(path, kpgfile) + if err != nil { + return errors.E(op, types.UniquePath(path), err) + } + return nil +} + +// UpdateUpstreamLockFromGit updates the upstreamLock of the package specified +// by path by using the values from spec. It will also populate the commit +// field in upstreamLock using the latest commit of the git repo given +// by spec. +func UpdateUpstreamLockFromGit(path string, spec *git.RepoSpec) error { + return UpdateUpstreamLock(path, &kptfilev1.UpstreamLock{ Type: kptfilev1.GitOrigin, Git: &kptfilev1.GitLock{ Repo: spec.OrgRepo, @@ -216,12 +241,7 @@ func UpdateUpstreamLockFromGit(path string, spec *git.RepoSpec) error { Ref: spec.Ref, Commit: spec.Commit, }, - } - err = WriteFile(path, kpgfile) - if err != nil { - return errors.E(op, types.UniquePath(path), err) - } - return nil + }) } // merge merges the Kptfiles from various sources and updates localKf with output diff --git a/pkg/test/live/runner.go b/pkg/test/live/runner.go index 304644ab01..2c246a61df 100644 --- a/pkg/test/live/runner.go +++ b/pkg/test/live/runner.go @@ -15,7 +15,6 @@ package live import ( - "bufio" "bytes" "errors" "os" @@ -98,6 +97,7 @@ func (r *Runner) VerifyExitCode(t *testing.T, err error) { } func (r *Runner) VerifyStdout(t *testing.T, stdout string) { +<<<<<<< HEAD assert.Equal(t, strings.TrimSpace(r.Config.StdOut), prepOutput(t, stdout)) } @@ -107,6 +107,13 @@ func (r *Runner) VerifyStderr(t *testing.T, stderr string) { func prepOutput(t *testing.T, s string) string { return strings.TrimSpace(substituteTimestamps(removeStatusEvents(t, s))) +======= + assert.Equal(t, strings.TrimSpace(r.Config.StdOut), strings.TrimSpace(substituteTimestamps(stdout))) +} + +func (r *Runner) VerifyStderr(t *testing.T, stderr string) { + assert.Equal(t, strings.TrimSpace(r.Config.StdErr), strings.TrimSpace(substituteTimestamps(stderr))) +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) } func (r *Runner) VerifyInventory(t *testing.T, name, namespace string) { diff --git a/site/installation/README.md b/site/installation/README.md index ed410bf943..e8fb5ff3b3 100644 --- a/site/installation/README.md +++ b/site/installation/README.md @@ -124,8 +124,14 @@ $ kpt version https://console.cloud.google.com/gcr/images/kpt-dev/GLOBAL/kpt-gcloud?gcrImageListsize=30 [cloud-sdk]: https://github.com/GoogleCloudPlatform/cloud-sdk-docker [linux]: +<<<<<<< HEAD https://github.com/GoogleContainerTools/kpt/releases/download/v1.0.0-beta.12/kpt_linux_amd64 [darwin]: https://github.com/GoogleContainerTools/kpt/releases/download/v1.0.0-beta.12/kpt_darwin_amd64 +======= + https://github.com/GoogleContainerTools/kpt/releases/download/v1.0.0-beta.9/kpt_linux_amd64 +[darwin]: + https://github.com/GoogleContainerTools/kpt/releases/download/v1.0.0-beta.9/kpt_darwin_amd64 +>>>>>>> 45fb5ee9 (Oci support rebased (#2621)) [migration guide]: /installation/migration [bash-completion]: https://github.com/scop/bash-completion#installation diff --git a/site/reference/cli/pkg/get/README.md b/site/reference/cli/pkg/get/README.md index 6288bae159..2fba974125 100644 --- a/site/reference/cli/pkg/get/README.md +++ b/site/reference/cli/pkg/get/README.md @@ -18,7 +18,7 @@ local directory. ``` -kpt pkg get REPO_URI[.git]/PKG_PATH[@VERSION] [LOCAL_DEST_DIRECTORY] [flags] +kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] ``` #### Args @@ -42,6 +42,13 @@ VERSION: A git tag, branch, ref or commit for the remote version of the package to fetch. Defaults to the default branch of the repository. +IMAGE: + Reference to an OCI image containing a package in the root directory. + +TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + LOCAL_DEST_DIRECTORY: The local directory to write the package to. Defaults to a subdirectory of the current working directory named after the upstream package. diff --git a/site/reference/cli/pkg/pull/README.md b/site/reference/cli/pkg/pull/README.md new file mode 100644 index 0000000000..c8b23f92f2 --- /dev/null +++ b/site/reference/cli/pkg/pull/README.md @@ -0,0 +1,126 @@ +--- +title: "`pull`" +linkTitle: "pull" +type: docs +description: > + TODO(oci-support) rewrite: Fetch a package from a git repo. +--- + + + +TODO(oci-support) rewrite: `get` fetches a remote package from a git subdirectory and writes it to a new +local directory. + +### Synopsis + + + +``` +kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] +``` + +#### Args + +``` +REPO_URI: + URI of a git repository containing 1 or more packages as subdirectories. + In most cases the .git suffix should be specified to delimit the REPO_URI + from the PKG_PATH, but this is not required for widely recognized repo + prefixes. If get cannot parse the repo for the directory and version, + then it will print an error asking for '.git' to be specified as part of + the argument. + +PKG_PATH: + Path to remote subdirectory containing Kubernetes resource configuration + files or directories. Defaults to the root directory. + Uses '/' as the path separator (regardless of OS). + e.g. staging/cockroachdb + +VERSION: + A git tag, branch, ref or commit for the remote version of the package + to fetch. Defaults to the default branch of the repository. + +IMAGE: + Reference to an OCI image containing a package in the root directory. + +TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + +LOCAL_DEST_DIRECTORY: + The local directory to write the package to. Defaults to a subdirectory of the + current working directory named after the upstream package. +``` + +#### Flags + +``` +--strategy: + Defines which strategy should be used to update the package. It defaults to + 'resource-merge'. + + * resource-merge: Perform a structural comparison of the original / + updated resources, and merge the changes into the local package. + * fast-forward: Fail without updating if the local package was modified + since it was fetched. + * force-delete-replace: Wipe all the local changes to the package and replace + it with the remote version. +``` + +#### Env Vars + +``` +KPT_CACHE_DIR: + Controls where to cache remote packages when fetching them. + Defaults to /.kpt/repos/ + On macOS and Linux is determined by the $HOME env variable, while on + Windows it is given by the %USERPROFILE% env variable. +``` + + + +### Examples + +{{% hide %}} + + + +``` +# Set up workspace for the test. +TEST_HOME=$(mktemp -d) +cd $TEST_HOME +``` + +{{% /hide %}} + + + + + +```shell +# Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb +# This creates a new subdirectory 'cockroachdb' for the downloaded package. +$ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master +``` + + + +```shell +# Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb +# This will create a new directory 'my-package' for the downloaded package if it +# doesn't already exist. +$ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master ./my-package/ +``` + + + +```shell +# Fetch package examples from github.com/kubernetes/examples at the specified +# git hash. +# This will create a new directory 'examples' for the package. +$ kpt pkg get https://github.com/kubernetes/examples.git/@6fe2792 +``` + + diff --git a/site/reference/cli/pkg/push/README.md b/site/reference/cli/pkg/push/README.md new file mode 100644 index 0000000000..af8489639d --- /dev/null +++ b/site/reference/cli/pkg/push/README.md @@ -0,0 +1,126 @@ +--- +title: "`push`" +linkTitle: "push" +type: docs +description: > + TODO(oci-support) rewrite: Fetch a package from a git repo. +--- + + + +TODO(oci-support) rewrite: `get` fetches a remote package from a git subdirectory and writes it to a new +local directory. + +### Synopsis + + +TODO(oci-support) rewrite: +``` +kpt pkg get {REPO_URI[.git]/PKG_PATH[@VERSION]|IMAGE:TAG} [LOCAL_DEST_DIRECTORY] [flags] +``` + +#### Args + +``` +REPO_URI: + URI of a git repository containing 1 or more packages as subdirectories. + In most cases the .git suffix should be specified to delimit the REPO_URI + from the PKG_PATH, but this is not required for widely recognized repo + prefixes. If get cannot parse the repo for the directory and version, + then it will print an error asking for '.git' to be specified as part of + the argument. + +PKG_PATH: + Path to remote subdirectory containing Kubernetes resource configuration + files or directories. Defaults to the root directory. + Uses '/' as the path separator (regardless of OS). + e.g. staging/cockroachdb + +VERSION: + A git tag, branch, ref or commit for the remote version of the package + to fetch. Defaults to the default branch of the repository. + +IMAGE: + Reference to an OCI image containing a package in the root directory. + +TAG: + An image tag or @sha256 digest for the remote version of the image + to fetch. Defaults to the 'latest' tag on the image. + +LOCAL_DEST_DIRECTORY: + The local directory to write the package to. Defaults to a subdirectory of the + current working directory named after the upstream package. +``` + +#### Flags + +``` +--strategy: + Defines which strategy should be used to update the package. It defaults to + 'resource-merge'. + + * resource-merge: Perform a structural comparison of the original / + updated resources, and merge the changes into the local package. + * fast-forward: Fail without updating if the local package was modified + since it was fetched. + * force-delete-replace: Wipe all the local changes to the package and replace + it with the remote version. +``` + +#### Env Vars + +``` +KPT_CACHE_DIR: + Controls where to cache remote packages when fetching them. + Defaults to /.kpt/repos/ + On macOS and Linux is determined by the $HOME env variable, while on + Windows it is given by the %USERPROFILE% env variable. +``` + + + +### Examples + +{{% hide %}} + + + +``` +# Set up workspace for the test. +TEST_HOME=$(mktemp -d) +cd $TEST_HOME +``` + +{{% /hide %}} + + + + + +```shell +# Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb +# This creates a new subdirectory 'cockroachdb' for the downloaded package. +$ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master +``` + + + +```shell +# Fetch package cockroachdb from github.com/kubernetes/examples/staging/cockroachdb +# This will create a new directory 'my-package' for the downloaded package if it +# doesn't already exist. +$ kpt pkg get https://github.com/kubernetes/examples.git/staging/cockroachdb@master ./my-package/ +``` + + + +```shell +# Fetch package examples from github.com/kubernetes/examples at the specified +# git hash. +# This will create a new directory 'examples' for the package. +$ kpt pkg get https://github.com/kubernetes/examples.git/@6fe2792 +``` + + diff --git a/site/reference/schema/kptfile/kptfile.json b/site/reference/schema/kptfile/kptfile.json index 1eea9f4613..4afdab4a56 100644 --- a/site/reference/schema/kptfile/kptfile.json +++ b/site/reference/schema/kptfile/kptfile.json @@ -180,6 +180,61 @@ }, "x-go-package": "sigs.k8s.io/kustomize/kyaml/yaml" }, + "Oci": { + "type": "object", + "title": "Oci is the user-specified locator for a package in an OCI image registry.", + "properties": { + "image": { + "description": "Image is the OCI image repository for the package.\ne.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend:latest'", + "type": "string", + "x-go-name": "Image" + }, + "path": { + "description": "Directory is the sub-package of the image.", + "type": "string", + "x-go-name": "Directory" + } + }, + "x-go-package": "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + }, + "OciLock": { + "type": "object", + "title": "OciLock is the resolved locator for a package in an OCI image registry.", + "properties": { + "digest": { + "description": "Digest is the unique sha of the image when it was last pulled.", + "type": "string", + "x-go-name": "Digest" + }, + "image": { + "description": "Image is the OCI image repository for the package.\ne.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend@sha256:b0c94f11d856e59673daca566857a7ead126ef8e2b6915ed662804f858d7eaea'", + "type": "string", + "x-go-name": "Image" + }, + "path": { + "description": "Directory is the sub-package of the image.", + "type": "string", + "x-go-name": "Directory" + } + }, + "x-go-package": "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + }, + "Origin": { + "type": "object", + "title": "Origin is a resolved locator when the package was last pulled or pushed.", + "properties": { + "git": { + "$ref": "#/definitions/GitLock" + }, + "oci": { + "$ref": "#/definitions/OciLock" + }, + "type": { + "$ref": "#/definitions/OriginType" + } + }, + "x-go-package": "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1" + }, "OriginType": { "type": "string", "title": "OriginType defines the type of origin for a package.", @@ -356,6 +411,9 @@ "git": { "$ref": "#/definitions/Git" }, + "oci": { + "$ref": "#/definitions/Oci" + }, "type": { "$ref": "#/definitions/OriginType" }, @@ -372,6 +430,9 @@ "git": { "$ref": "#/definitions/GitLock" }, + "oci": { + "$ref": "#/definitions/OciLock" + }, "type": { "$ref": "#/definitions/OriginType" } @@ -424,6 +485,9 @@ "type": "string", "x-go-name": "Namespace" }, + "origin": { + "$ref": "#/definitions/Origin" + }, "pipeline": { "$ref": "#/definitions/Pipeline" }, diff --git a/site/reference/schema/kptfile/kptfile.yaml b/site/reference/schema/kptfile/kptfile.yaml index 3c1dd48b8a..9d18993367 100644 --- a/site/reference/schema/kptfile/kptfile.yaml +++ b/site/reference/schema/kptfile/kptfile.yaml @@ -159,6 +159,51 @@ definitions: x-go-name: Namespace type: object x-go-package: sigs.k8s.io/kustomize/kyaml/yaml + Oci: + properties: + image: + description: |- + Image is the OCI image repository for the package. + e.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend:latest' + type: string + x-go-name: Image + path: + description: Directory is the sub-package of the image. + type: string + x-go-name: Directory + title: Oci is the user-specified locator for a package in an OCI image registry. + type: object + x-go-package: github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1 + OciLock: + properties: + digest: + description: Digest is the unique sha of the image when it was last pulled. + type: string + x-go-name: Digest + image: + description: |- + Image is the OCI image repository for the package. + e.g. 'LOCATION-docker.pkg.dev/PROJECT_ID/REPOSITORY_NAME/app-frontend@sha256:b0c94f11d856e59673daca566857a7ead126ef8e2b6915ed662804f858d7eaea' + type: string + x-go-name: Image + path: + description: Directory is the sub-package of the image. + type: string + x-go-name: Directory + title: OciLock is the resolved locator for a package in an OCI image registry. + type: object + x-go-package: github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1 + Origin: + properties: + git: + $ref: '#/definitions/GitLock' + oci: + $ref: '#/definitions/OciLock' + type: + $ref: '#/definitions/OriginType' + title: Origin is a resolved locator when the package was last pulled or pushed. + type: object + x-go-package: github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1 OriginType: title: OriginType defines the type of origin for a package. type: string @@ -303,6 +348,8 @@ definitions: properties: git: $ref: '#/definitions/Git' + oci: + $ref: '#/definitions/Oci' type: $ref: '#/definitions/OriginType' updateStrategy: @@ -314,6 +361,8 @@ definitions: properties: git: $ref: '#/definitions/GitLock' + oci: + $ref: '#/definitions/OciLock' type: $ref: '#/definitions/OriginType' title: UpstreamLock is a resolved locator for the last fetch of the package. @@ -353,6 +402,8 @@ definitions: description: Namespace is the metadata.namespace field of a Resource type: string x-go-name: Namespace + origin: + $ref: '#/definitions/Origin' pipeline: $ref: '#/definitions/Pipeline' upstream: diff --git a/site/sidebar.md b/site/sidebar.md index 6aa992824e..a04f960599 100644 --- a/site/sidebar.md +++ b/site/sidebar.md @@ -17,6 +17,7 @@ - [3.6 Creating a Package](book/03-packages/06-creating-a-package.md) - [3.7 Composing a Package](book/03-packages/07-composing-a-package.md) - [3.8 Publishing a Package](book/03-packages/08-publishing-a-package.md) + - TODO(oci-support) Add to existing sections. Introduce new section specifically for Artifact Registry. - [4 Using Functions](book/04-using-functions/) - [4.1 Declarative Function Execution](book/04-using-functions/01-declarative-function-execution.md) - [4.2 Imperative Function Execution](book/04-using-functions/02-imperative-function-execution.md) @@ -38,6 +39,8 @@ - [init](reference/cli/pkg/init/) - [tree](reference/cli/pkg/tree/) - [update](reference/cli/pkg/update/) + - [pull](reference/cli/pkg/pull/) + - [push](reference/cli/pkg/push/) - [fn](reference/cli/fn/) - [render](reference/cli/fn/render/) - [eval](reference/cli/fn/eval/)