Bug 2043130: Rebase to v3.1.0 for OCP 4.10

CHANGELOG/CHANGELOG-3.1.md

@@ -0,0 +1,99 @@
+# Release notes for v3.1.0
+
+[Documentation](https://kubernetes-csi.github.io)
+
+# Changelog since v3.0.0
+
+## Changes by Kind
+
+### Feature
+ - Introduces `HonorPVReclaimPolicy` feature gate that ensures PV honors the Reclaim policy irrespective of deletion order. ([#679](https://github.com/kubernetes-csi/external-provisioner/pull/679), [@deepakkinni](https://github.com/deepakkinni))
+ - This release update brings the csi-translation-lib migration support for  in-tree plugin Ceph RBD ( `kubernetes.io/rbd`) to `rbd.csi.ceph.com` CSI driver and Portworx (`kubernetes.io/portworx-volume`)  in-tree plugin to `pxd.portworx.com` CSI driver. ([#689](https://github.com/kubernetes-csi/external-provisioner/pull/689), [@humblec](https://github.com/humblec))
+
+### Bug or Regression
+ - Add getCapacity request timeout to avoid hang forever ([#688](https://github.com/kubernetes-csi/external-provisioner/pull/688), [@bai3shuo4](https://github.com/bai3shuo4))
+ - Fixed retry of CreateVolume calls when the storage backend runs out of space. Such calls won't be repeated when user deletes corresponding PVC. ([#675](https://github.com/kubernetes-csi/external-provisioner/pull/675), [@jsafrane](https://github.com/jsafrane))
+
+## Dependencies
+
+### Added
+- github.com/antlr/antlr4/runtime/Go/antlr: [b48c857](https://github.com/antlr/antlr4/runtime/Go/antlr/tree/b48c857)
+- github.com/cncf/xds/go: [fbca930](https://github.com/cncf/xds/go/tree/fbca930)
+- github.com/getkin/kin-openapi: [v0.76.0](https://github.com/getkin/kin-openapi/tree/v0.76.0)
+- github.com/google/cel-go: [v0.9.0](https://github.com/google/cel-go/tree/v0.9.0)
+- github.com/google/cel-spec: [v0.6.0](https://github.com/google/cel-spec/tree/v0.6.0)
+- github.com/gorilla/mux: [v1.8.0](https://github.com/gorilla/mux/tree/v1.8.0)
+- github.com/kr/fs: [v0.1.0](https://github.com/kr/fs/tree/v0.1.0)
+- github.com/pkg/sftp: [v1.10.1](https://github.com/pkg/sftp/tree/v1.10.1)
+- sigs.k8s.io/json: 9f7c6b3
+- sigs.k8s.io/sig-storage-lib-external-provisioner/v8: v8.0.0
+
+### Changed
+- cloud.google.com/go: v0.65.0 → v0.81.0
+- github.com/benbjohnson/clock: [v1.0.3 → v1.1.0](https://github.com/benbjohnson/clock/compare/v1.0.3...v1.1.0)
+- github.com/bketelsen/crypt: [5cbc8cc → v0.0.4](https://github.com/bketelsen/crypt/compare/5cbc8cc...v0.0.4)
+- github.com/envoyproxy/go-control-plane: [668b12f → 63b5d3c](https://github.com/envoyproxy/go-control-plane/compare/668b12f...63b5d3c)
+- github.com/evanphx/json-patch: [v4.11.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v4.11.0...v4.12.0)
+- github.com/go-logr/logr: [v0.4.0 → v1.2.1](https://github.com/go-logr/logr/compare/v0.4.0...v1.2.1)
+- github.com/go-logr/zapr: [v0.2.0 → v1.2.0](https://github.com/go-logr/zapr/compare/v0.2.0...v1.2.0)
+- github.com/golang/glog: [23def4e → v1.0.0](https://github.com/golang/glog/compare/23def4e...v1.0.0)
+- github.com/golang/mock: [v1.4.4 → v1.5.0](https://github.com/golang/mock/compare/v1.4.4...v1.5.0)
+- github.com/google/go-cmp: [v0.5.5 → v0.5.6](https://github.com/google/go-cmp/compare/v0.5.5...v0.5.6)
+- github.com/google/martian/v3: [v3.0.0 → v3.1.0](https://github.com/google/martian/v3/compare/v3.0.0...v3.1.0)
+- github.com/google/pprof: [1a94d86 → cbba55b](https://github.com/google/pprof/compare/1a94d86...cbba55b)
+- github.com/ianlancetaylor/demangle: [5e5cf60 → 28f6c0f](https://github.com/ianlancetaylor/demangle/compare/5e5cf60...28f6c0f)
+- github.com/json-iterator/go: [v1.1.11 → v1.1.12](https://github.com/json-iterator/go/compare/v1.1.11...v1.1.12)
+- github.com/kr/pty: [v1.1.5 → v1.1.1](https://github.com/kr/pty/compare/v1.1.5...v1.1.1)
+- github.com/magiconair/properties: [v1.8.1 → v1.8.5](https://github.com/magiconair/properties/compare/v1.8.1...v1.8.5)
+- github.com/mattn/go-isatty: [v0.0.4 → v0.0.3](https://github.com/mattn/go-isatty/compare/v0.0.4...v0.0.3)
+- github.com/mitchellh/mapstructure: [v1.1.2 → v1.4.1](https://github.com/mitchellh/mapstructure/compare/v1.1.2...v1.4.1)
+- github.com/modern-go/reflect2: [v1.0.1 → v1.0.2](https://github.com/modern-go/reflect2/compare/v1.0.1...v1.0.2)
+- github.com/pelletier/go-toml: [v1.2.0 → v1.9.3](https://github.com/pelletier/go-toml/compare/v1.2.0...v1.9.3)
+- github.com/prometheus/common: [v0.26.0 → v0.28.0](https://github.com/prometheus/common/compare/v0.26.0...v0.28.0)
+- github.com/spf13/afero: [v1.2.2 → v1.6.0](https://github.com/spf13/afero/compare/v1.2.2...v1.6.0)
+- github.com/spf13/cast: [v1.3.0 → v1.3.1](https://github.com/spf13/cast/compare/v1.3.0...v1.3.1)
+- github.com/spf13/cobra: [v1.1.3 → v1.2.1](https://github.com/spf13/cobra/compare/v1.1.3...v1.2.1)
+- github.com/spf13/jwalterweatherman: [v1.0.0 → v1.1.0](https://github.com/spf13/jwalterweatherman/compare/v1.0.0...v1.1.0)
+- github.com/spf13/viper: [v1.7.0 → v1.8.1](https://github.com/spf13/viper/compare/v1.7.0...v1.8.1)
+- github.com/stretchr/objx: [v0.2.0 → v0.1.1](https://github.com/stretchr/objx/compare/v0.2.0...v0.1.1)
+- github.com/yuin/goldmark: [v1.3.5 → v1.4.0](https://github.com/yuin/goldmark/compare/v1.3.5...v1.4.0)
+- go.opencensus.io: v0.22.4 → v0.23.0
+- go.uber.org/zap: v1.17.0 → v1.19.0
+- golang.org/x/crypto: 513c2a4 → 32db794
+- golang.org/x/net: 37e1c6a → 491a49a
+- golang.org/x/oauth2: cd4f82c → 2bc19b1
+- golang.org/x/sys: 59db8d7 → f4d4317
+- golang.org/x/term: de623e6 → 6886f2d
+- golang.org/x/text: v0.3.6 → v0.3.7
+- golang.org/x/tools: v0.1.2 → d4cc65f
+- google.golang.org/api: v0.30.0 → v0.44.0
+- google.golang.org/genproto: f16073e → fe13028
+- google.golang.org/grpc: v1.38.0 → v1.40.0
+- google.golang.org/protobuf: v1.26.0 → v1.27.1
+- gopkg.in/ini.v1: v1.51.0 → v1.62.0
+- k8s.io/api: v0.22.0 → v0.23.0
+- k8s.io/apiextensions-apiserver: v0.20.1 → v0.23.0
+- k8s.io/apimachinery: v0.22.0 → v0.23.0
+- k8s.io/apiserver: v0.22.0 → v0.23.0
+- k8s.io/client-go: v0.22.0 → v0.23.0
+- k8s.io/code-generator: v0.20.1 → v0.23.0
+- k8s.io/component-base: v0.22.0 → v0.23.0
+- k8s.io/component-helpers: v0.22.0 → v0.23.0
+- k8s.io/csi-translation-lib: v0.22.0 → 135d37b
+- k8s.io/gengo: 83324d8 → 485abfe
+- k8s.io/klog/v2: v2.9.0 → v2.30.0
+- k8s.io/kube-openapi: 9528897 → e816edb
+- k8s.io/utils: 4b05e18 → 7d6a63d
+- sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.22 → v0.0.25
+- sigs.k8s.io/structured-merge-diff/v4: v4.1.2 → v4.2.0
+
+### Removed
+- github.com/docker/spdystream: [449fdfc](https://github.com/docker/spdystream/tree/449fdfc)
+- github.com/go-openapi/spec: [v0.19.3](https://github.com/go-openapi/spec/tree/v0.19.3)
+- github.com/mattn/go-runewidth: [v0.0.2](https://github.com/mattn/go-runewidth/tree/v0.0.2)
+- github.com/olekukonko/tablewriter: [a0225b3](https://github.com/olekukonko/tablewriter/tree/a0225b3)
+- github.com/urfave/cli: [v1.20.0](https://github.com/urfave/cli/tree/v1.20.0)
+- go.etcd.io/etcd: dd1b699
+- gopkg.in/cheggaaa/pb.v1: v1.0.25
+- gotest.tools: v2.2.0+incompatible
+- sigs.k8s.io/sig-storage-lib-external-provisioner/v7: v7.0.1

README.md

@@ -25,6 +25,7 @@ Following table reflects the head of this branch.
 | CSIMigration   | Beta    | On      | [Migrating in-tree volume plugins to CSI](https://kubernetes.io/docs/concepts/storage/volumes/#csi-migration). | No |
 | CSIStorageCapacity | Beta  | On  | Publish [capacity information](https://kubernetes.io/docs/concepts/storage/volumes/#storage-capacity) for the Kubernetes scheduler. | No |
 | ReadWriteOncePod   | Alpha | Off | [Single pod access mode for PersistentVolumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes). | No |
+| HonorPVReclaimPolicy| Alpha |Off | [Honor the PV reclaim policy](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/2644-honor-pv-reclaim-policy) | No
 
 All other external-provisioner features and the external-provisioner itself is considered GA and fully supported.
 
@@ -126,7 +127,7 @@ See the [storage capacity section](#capacity-support) below for details.
 External-provisioner interacts with Kubernetes by watching PVCs and
 PVs and implementing the [external provisioner
 protocol](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#provisioning-and-deleting).
-The [design document](./docs/design.md) explains this in more detail.
+The [design document](./doc/design.md) explains this in more detail.
 
 ### Topology support
 When `Topology` feature is enabled and the driver specifies `VOLUME_ACCESSIBILITY_CONSTRAINTS` in its plugin capabilities, external-provisioner prepares `CreateVolumeRequest.AccessibilityRequirements` while calling `Controller.CreateVolume`. The driver has to consider these topology constraints while creating the volume. Below table shows how these `AccessibilityRequirements` are prepared:
@@ -275,7 +276,7 @@ with some minor change:
   variables to make the DaemonSet that contains the external-provisioner
   the owner of CSIStorageCapacity objects for the node.
 
-Deployments of external-provisioner outside of the Kubernetes cluster
+Deployments of external-provisioner outside the Kubernetes cluster
 are also possible, albeit only without an owner for the objects.
 `NAMESPACE` still needs to be set to some existing namespace also
 in this case.

cmd/csi-provisioner/csi-provisioner.go

@@ -20,6 +20,7 @@ import (
 	"context"
 	goflag "flag"
 	"fmt"
+	"github.com/kubernetes-csi/external-provisioner/pkg/features"
 	"math/rand"
 	"net/http"
 	"os"
@@ -50,7 +51,7 @@ import (
 	_ "k8s.io/component-base/metrics/prometheus/workqueue"               // register work queues in the default legacy registry
 	csitrans "k8s.io/csi-translation-lib"
 	"k8s.io/klog/v2"
-	"sigs.k8s.io/sig-storage-lib-external-provisioner/v7/controller"
+	"sigs.k8s.io/sig-storage-lib-external-provisioner/v8/controller"
 
 	"github.com/kubernetes-csi/csi-lib-utils/leaderelection"
 	"github.com/kubernetes-csi/csi-lib-utils/metrics"
@@ -73,7 +74,7 @@ var (
 	workerThreads        = flag.Uint("worker-threads", 100, "Number of provisioner worker threads, in other words nr. of simultaneous CSI calls.")
 	finalizerThreads     = flag.Uint("cloning-protection-threads", 1, "Number of simultaneously running threads, handling cloning finalizer removal")
 	capacityThreads      = flag.Uint("capacity-threads", 1, "Number of simultaneously running threads, handling CSIStorageCapacity objects")
-	operationTimeout     = flag.Duration("timeout", 10*time.Second, "Timeout for waiting for creation or deletion of a volume")
+	operationTimeout     = flag.Duration("timeout", 10*time.Second, "Timeout for waiting for volume operation (creation, deletion, capacity queries)")
 
 	enableLeaderElection = flag.Bool("leader-election", false, "Enables leader election. If leader election is enabled, additional RBAC rules are required. Please refer to the Kubernetes CSI documentation for instructions on setting up these RBAC rules.")
 
@@ -359,6 +360,10 @@ func main() {
 		controller.NodesLister(nodeLister),
 	}
 
+	if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) {
+		provisionerOptions = append(provisionerOptions, controller.AddFinalizer(true))
+	}
+
 	if supportsMigrationFromInTreePluginName != "" {
 		provisionerOptions = append(provisionerOptions, controller.AdditionalProvisionerNames([]string{supportsMigrationFromInTreePluginName}))
 	}
@@ -470,6 +475,7 @@ func main() {
 			factoryForNamespace.Storage().V1beta1().CSIStorageCapacities(),
 			*capacityPollInterval,
 			*capacityImmediateBinding,
+			*operationTimeout,
 		)
 		legacyregistry.CustomMustRegister(capacityController)
 

go.mod

@@ -1,10 +1,12 @@
 module github.com/kubernetes-csi/external-provisioner
 
-go 1.16
+go 1.17
 
 require (
 	github.com/container-storage-interface/spec v1.5.0
-	github.com/golang/mock v1.4.4
+	github.com/go-logr/logr v1.2.1 // indirect
+	github.com/golang/mock v1.5.0
+	github.com/google/go-cmp v0.5.6 // indirect
 	github.com/google/gofuzz v1.2.0 // indirect
 	github.com/google/uuid v1.2.0 // indirect
 	github.com/imdario/mergo v0.3.12 // indirect
@@ -15,20 +17,111 @@ require (
 	github.com/prometheus/client_golang v1.11.0
 	github.com/spf13/pflag v1.0.5
 	github.com/stretchr/testify v1.7.0
-	golang.org/x/crypto v0.0.0-20210317152858-513c2a44f670 // indirect
-	golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84 // indirect
-	golang.org/x/term v0.0.0-20210317153231-de623e64d2a6 // indirect
-	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/grpc v1.38.0
-	google.golang.org/protobuf v1.26.0
-	k8s.io/api v0.22.0
-	k8s.io/apimachinery v0.22.0
-	k8s.io/apiserver v0.22.0
-	k8s.io/client-go v0.22.0
-	k8s.io/component-base v0.22.0
-	k8s.io/component-helpers v0.22.0
-	k8s.io/csi-translation-lib v0.22.0
-	k8s.io/klog/v2 v2.9.0
+	golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect
+	google.golang.org/grpc v1.40.0
+	google.golang.org/protobuf v1.27.1
+	k8s.io/api v0.23.0
+	k8s.io/apimachinery v0.23.0
+	k8s.io/apiserver v0.23.0
+	k8s.io/client-go v0.23.0
+	k8s.io/component-base v0.23.0
+	k8s.io/component-helpers v0.23.0
+	k8s.io/csi-translation-lib v0.0.0-20211208082742-135d37b906fc
+	k8s.io/klog/v2 v2.30.0
+	k8s.io/utils v0.0.0-20211208161948-7d6a63dca704 // indirect
 	sigs.k8s.io/controller-runtime v0.8.3
-	sigs.k8s.io/sig-storage-lib-external-provisioner/v7 v7.0.1
+	sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect
+	sigs.k8s.io/sig-storage-lib-external-provisioner/v8 v8.0.0
+	sigs.k8s.io/structured-merge-diff/v4 v4.2.0 // indirect
+)
+
+require (
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/blang/semver v3.5.1+incompatible // indirect
+	github.com/cespare/xxhash/v2 v2.1.1 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/evanphx/json-patch v4.12.0+incompatible // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/googleapis/gnostic v0.5.5 // indirect
+	github.com/inconshreveable/mousetrap v1.0.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/prometheus/client_model v0.2.0 // indirect
+	github.com/prometheus/common v0.28.0 // indirect
+	github.com/prometheus/procfs v0.6.0 // indirect
+	github.com/spf13/cobra v1.2.1 // indirect
+	golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 // indirect
+	golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect
+	golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e // indirect
+	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect
+	golang.org/x/text v0.3.7 // indirect
+	golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
+	k8s.io/klog v1.0.0 // indirect
+	k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect
+	sigs.k8s.io/yaml v1.2.0 // indirect
 )
+
+replace k8s.io/api => k8s.io/api v0.23.0
+
+replace k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.23.0
+
+replace k8s.io/apimachinery => k8s.io/apimachinery v0.23.0
+
+replace k8s.io/apiserver => k8s.io/apiserver v0.23.0
+
+replace k8s.io/cli-runtime => k8s.io/cli-runtime v0.23.0
+
+replace k8s.io/client-go => k8s.io/client-go v0.23.0
+
+replace k8s.io/cloud-provider => k8s.io/cloud-provider v0.23.0
+
+replace k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.23.0
+
+replace k8s.io/code-generator => k8s.io/code-generator v0.23.0
+
+replace k8s.io/component-base => k8s.io/component-base v0.23.0
+
+replace k8s.io/component-helpers => k8s.io/component-helpers v0.23.0
+
+replace k8s.io/controller-manager => k8s.io/controller-manager v0.23.0
+
+replace k8s.io/cri-api => k8s.io/cri-api v0.23.0
+
+replace k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.0.0-20211208082742-135d37b906fc
+
+replace k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.23.0
+
+replace k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.23.0
+
+replace k8s.io/kube-proxy => k8s.io/kube-proxy v0.23.0
+
+replace k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.23.0
+
+replace k8s.io/kubectl => k8s.io/kubectl v0.23.0
+
+replace k8s.io/kubelet => k8s.io/kubelet v0.23.0
+
+replace k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.23.0
+
+replace k8s.io/metrics => k8s.io/metrics v0.23.0
+
+replace k8s.io/mount-utils => k8s.io/mount-utils v0.23.0
+
+replace k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.23.0
+
+replace k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.23.0
+
+replace k8s.io/sample-cli-plugin => k8s.io/sample-cli-plugin v0.23.0
+
+replace k8s.io/sample-controller => k8s.io/sample-controller v0.23.0