Bug 1993934: Merge upstream v4.2.0 #56

bertinatto · 2021-08-17T13:22:56Z

kubernetes-csi/external-snapshotter@v4.2.0...bertinatto:merge-v4.2.0

CC @openshift/storage

…ate-master

master: update release-tools

…esnapshots-request Update volumesnapshots request to list across all namespaces

…ate-master

master: update release-tools + OWNERS

Two new timeout values ( retryIntervalStart & retryIntervalMax ) have been added to set the ratelimiter for volumesnapshotcontent queue. Fix# kubernetes-csi#463 Signed-off-by: Humble Chirammal <hchiramm@redhat.com> ```release-note `retry-interval-start` and `retry-interval-max` arguments are added to csi-snapshotter sidecar which controls retry interval of failed volume snapshot creation and deletion. These values set the ratelimiter for volumesnapshotcontent queue. ``` Signed-off-by: Humble Chirammal <hchiramm@redhat.com>

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>

correct snapshot controller installation doc

Add ability to customize VolumeSnapshotContent workqueue

… retryIntervalMax` This patch adds two new parameters `retryIntervalStart & retryIntervalMax` which can be configured to adjust the ratelimiters of snapshotqueue and contentqueue in the controller. Signed-off-by: Humble Chirammal <hchiramm@redhat.com> ```release-note `retry-interval-start` and `retry-interval-max` arguments are added to common-controller which controls retry interval of failed volume snapshot creation and deletion. These values set the ratelimiter for snapshot and content queues. ``` Signed-off-by: Humble Chirammal <hchiramm@redhat.com>

Allow tuning common-controller Ratelimiter with `retryIntervalStart & retryIntervalMax`

Signed-off-by: Grant Griffiths <ggriffiths@purestorage.com>

Use CSI spec v1.5.0

…ngcreated_returncontent Return VolumeSnapshotContent from various functions instead of nil

…space Add VS namespace to VSC printed columns

…config Add command line arguments to configure leader election options

…v4.1.1

update setup-csi-snapshotter.yaml csi-snapshotter image

…ate-master

openshift-ci · 2021-08-17T17:04:13Z

@bertinatto: An error was encountered querying GitHub for users with public email (wduan@redhat.com) for bug 1993934 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message.


non-200 OK status code: 403 Forbidden body: "{\n  \"documentation_url\": \"https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits\",\n  \"message\": \"You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\"\n}\n"

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bertinatto · 2021-08-17T17:04:44Z

/bugzilla refresh

openshift-ci · 2021-08-17T17:05:49Z

@bertinatto: This pull request references Bugzilla bug 1993934, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.9.0) matches configured target release for branch (4.9.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (wduan@redhat.com), skipping review request.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jsafrane · 2021-08-18T12:39:44Z

/verify-owners

jsafrane · 2021-08-18T12:42:09Z

/test e2e-gcp-csi
/test e2e-aws-csi
I don't like the snapshot failures

jsafrane · 2021-08-18T15:19:14Z

/test e2e-gcp-csi
/test e2e-aws-csi
just to be sure...

bertinatto · 2021-08-18T16:08:46Z

/test all

bertinatto · 2021-08-18T17:54:18Z

/test e2e-gcp-csi

jsafrane · 2021-08-19T07:13:20Z

/test e2e-gcp-csi
/test e2e-aws-csi

jsafrane · 2021-08-19T08:16:44Z

Something is wrong. In run https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_csi-external-snapshotter/56/pull-ci-openshift-csi-external-snapshotter-master-e2e-gcp-csi/1428026187141091328

A test wants to create a snapshot at 16:51:34.497:

Aug 18 16:51:34.497: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-9qr8s to become ready

But snapshot-controller logs its last line at 2021-08-18T16:50:04.903499802Z. What does it do for the rest of the job run?
Periodic 4.9 job doesn't show any recent breakage (yet): https://testgrid.k8s.io/redhat-openshift-ocp-release-4.9-broken#periodic-ci-openshift-release-master-nightly-4.9-e2e-gcp-csi

jsafrane · 2021-08-19T11:18:52Z

Reproduced on a cluster built from this PR. Just run snapshot tests 5x in parallel:

for i in `seq 5`; do ( TEST_CSI_DRIVER_FILES=manifest.yaml ./openshift-tests run openshift/csi  --run=VolumeSnapshotDataSource  |& tee log$i ) & done

Some tests timed out. Looking at the snapshot-controller container, all its snapshotWorker goroutines get stuck at:

goroutine 113 [semacquire, 13 minutes]:
sync.runtime_SemacquireMutex(0xc0004f59ec, 0x0, 0x1)
        /usr/lib/golang/src/runtime/sema.go:71 +0x47
sync.(*Mutex).lockSlow(0xc0004f59e8)
        /usr/lib/golang/src/sync/mutex.go:138 +0x105
sync.(*Mutex).Lock(...)
        /usr/lib/golang/src/sync/mutex.go:81
github.com/kubernetes-csi/external-snapshotter/v4/pkg/metrics.(*operationMetricsManager).OperationStart(0xc0004f59e0, 0x19aba21, 0xe, 0xc000703e60, 0x24, 0xc0004b6050, 0xf, 0x19a600c, 0x7, 0x0, ...)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/metrics/metrics.go:186 +0x217
github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).processSnapshotWithDeletionTimestamp(0xc0005980f0, 0xc000732f00, 0xc000732f00, 0x0)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller.go:252 +0x3cf
github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).syncSnapshot(0xc0005980f0, 0xc000732f00, 0x19723a0, 0xc000732f00)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller.go:205 +0x1175
github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).updateSnapshot(0xc0005980f0, 0xc000732f00, 0x0, 0x0)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller_base.go:374 +0x326
github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).syncSnapshotByKey(0xc0005980f0, 0xc000703f50, 0x24, 0xc00044e000, 0x1)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller_base.go:230 +0xed7
github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).snapshotWorker(0xc0005980f0)
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller_base.go:195 +0xf8
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc0003c8000)
        /go/src/github.com/kubernetes-csi/external-snapshotter/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0003c8000, 0x1b91080, 0xc00024c030, 0x1, 0xc00044e060)
        /go/src/github.com/kubernetes-csi/external-snapshotter/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0x9b
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0003c8000, 0x0, 0x0, 0x1, 0xc00044e060)
        /go/src/github.com/kubernetes-csi/external-snapshotter/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc0003c8000, 0x0, 0xc00044e060)
        /go/src/github.com/kubernetes-csi/external-snapshotter/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x4d
created by github.com/kubernetes-csi/external-snapshotter/v4/pkg/common-controller.(*csiSnapshotCommonController).Run
        /go/src/github.com/kubernetes-csi/external-snapshotter/pkg/common-controller/snapshot_controller_base.go:146 +0x305

jsafrane · 2021-08-19T11:49:25Z

The tests are pretty flaky upstream too: https://prow.k8s.io/job-history/gs/kubernetes-jenkins/pr-logs/directory/pull-kubernetes-csi-external-snapshotter-1-21-on-kubernetes-1-21

jsafrane · 2021-08-19T11:50:44Z

I am testing a fix in #57 - metrics.go tries to acquire a mutex twice in the same goroutine and go mutexes are not recursive.

jsafrane · 2021-08-19T16:21:58Z

Reported upstream in kubernetes-csi#580. We may need to introduce kubernetes-csi#581 here. I hope upstream is going to release a new version soon.

RecordMetrics() grabs a mutex and calls recordCancelMetric(), which wants to grab the same mutex. Go mutexes are not recursive, so recordCancelMetric blocks forever. recordCancelMetric should not grab the mutex, it can be sure that the caller did it already.

openshift-ci · 2021-08-19T17:26:32Z

The following users are mentioned in OWNERS file(s) but are untrusted for the following reasons. One way to make the user trusted is to add them as members of the openshift org. You can then trigger verification by writing /verify-owners in a comment.

kubernetes-csi-reviewers
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.
kubernetes-csi-approvers
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.

bertinatto · 2021-08-19T17:26:58Z

Reported upstream in kubernetes-csi#580. We may need to introduce kubernetes-csi#581 here. I hope upstream is going to release a new version soon.

Done. Let's run the CSI jobs a few times before merging to make sure the flake is gone.

bertinatto · 2021-08-19T17:27:10Z

/test all

openshift-ci · 2021-08-20T18:37:43Z

@bertinatto: This pull request references Bugzilla bug 1993934, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.9.0) matches configured target release for branch (4.9.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (wduan@redhat.com), skipping review request.

In response to this:

Bug 1993934: Merge upstream v4.2.0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

gnufied · 2021-08-20T19:05:54Z

/lgtm

openshift-ci · 2021-08-20T19:12:27Z

@bertinatto: Some pull requests linked via external trackers have merged:

The following pull requests linked via external trackers have not merged:

openshift/csi-external-attacher#34 is open

These pull request must merge or be unlinked from the Bugzilla bug in order for it to move to the next state. Once unlinked, request a bug refresh with /bugzilla refresh.

Bugzilla bug 1993934 has not been moved to the MODIFIED state.

In response to this:

Bug 1993934: Merge upstream v4.2.0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2023-11-25T01:12:45Z

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-csi-snapshot-validation-webhook-container-v4.9.0-202311250023.p0.g52ab893.assembly.stream for distgit csi-snapshot-validation-webhook.
All builds following this will include this PR.

pohly and others added 30 commits May 5, 2021 14:47

Merge commit '915dbb98ef6e5e0e1f03863c48af68f5391f8ec2' into prow-upd…

e8a5aa1

…ate-master

Merge pull request kubernetes-csi#512 from pohly/prow-update-master

f235e24

master: update release-tools

remove volumesnapshots requests

77211d3

Merge pull request kubernetes-csi#515 from mauriciopoppe/remove-volum…

51ad855

…esnapshots-request Update volumesnapshots request to list across all namespaces

Merge commit '3aefc40813efe34f0e8de035925eea7f74733b47' into prow-upd…

ceabf92

…ate-master

OWNERS update with aliases

5a68e98

Merge pull request kubernetes-csi#520 from pohly/prow-update-master

87ef4df

master: update release-tools + OWNERS

Correct CRD installation in the doc

de7589c

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>

Merge pull request kubernetes-csi#529 from humblec/beta

a06aa19

correct snapshot controller installation doc

Merge pull request kubernetes-csi#308 from humblec/ratelimiter

af117aa

Add ability to customize VolumeSnapshotContent workqueue

Merge pull request kubernetes-csi#530 from humblec/rate-limiter

81542bd

Allow tuning common-controller Ratelimiter with `retryIntervalStart & retryIntervalMax`

Return VolumeSnapshotContent from various functions instead of nil

5cf92fc

Signed-off-by: Grant Griffiths <ggriffiths@purestorage.com>

Use CSI spec v1.5.0

b2709ea

Merge pull request kubernetes-csi#532 from chrishenzie/csi-spec-v1.5.0

629e862

Use CSI spec v1.5.0

Merge pull request kubernetes-csi#527 from ggriffiths/removeannvs_bei…

d6d967c

…ngcreated_returncontent Return VolumeSnapshotContent from various functions instead of nil

Add VS namespace to VSC printed columns

0bd24e2

Merge pull request kubernetes-csi#535 from tsmetana/vsc-print-vs-name…

e6f0b49

…space Add VS namespace to VSC printed columns

Add command line arguments to configure leader election options

db972e7

Merge pull request kubernetes-csi#538 from RaunakShah/leaderelection_…

5e46b4b

…config Add command line arguments to configure leader election options

update set-csi-snapshotter.yaml csi-snapshotter image from v4.0.0 to …

c9c8c38

…v4.1.1

update webhook.yaml snapshot-validation-webhook image from v4.0.0 to …

c020e5b

…v4.1.1

Merge branch 'kubernetes-csi:master' into update-image-version

8107014

Merge pull request kubernetes-csi#537 from f10atin9/update-image-version

c1db509

update setup-csi-snapshotter.yaml csi-snapshotter image

Add "SnapshotCreated" and "SnapshotReady" events for snapshot-controller

60824e5

Merge commit 'd7f0e3f0ee40d68c00dffd632675aeafa0539907' into prow-upd…

b88670c

…ate-master

openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Aug 17, 2021

openshift-ci bot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Aug 17, 2021

bertinatto mentioned this pull request Aug 18, 2021

Bug 1992255: Fix leader election defaults in snapshot-controller openshift/cluster-csi-snapshot-controller-operator#99

Merged

openshift-ci bot assigned gnufied Aug 20, 2021

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 20, 2021

gnufied removed the do-not-merge/invalid-owners-file Indicates that a PR should not merge because it has an invalid OWNERS file in it. label Aug 20, 2021

openshift-merge-robot merged commit 52ab893 into openshift:master Aug 20, 2021

openshift-ci bot mentioned this pull request Aug 23, 2021

Bug 1993934: Update CSI sidecars openshift/csi-external-provisioner#44

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1993934: Merge upstream v4.2.0 #56

Bug 1993934: Merge upstream v4.2.0 #56

bertinatto commented Aug 17, 2021 •

edited

Loading

openshift-ci bot commented Aug 17, 2021

bertinatto commented Aug 17, 2021

openshift-ci bot commented Aug 17, 2021

jsafrane commented Aug 18, 2021

jsafrane commented Aug 18, 2021

jsafrane commented Aug 18, 2021

bertinatto commented Aug 18, 2021

bertinatto commented Aug 18, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021 •

edited

Loading

jsafrane commented Aug 19, 2021 •

edited

Loading

openshift-ci bot commented Aug 19, 2021

bertinatto commented Aug 19, 2021

bertinatto commented Aug 19, 2021

openshift-ci bot commented Aug 20, 2021

gnufied commented Aug 20, 2021

openshift-ci bot commented Aug 20, 2021

openshift-bot commented Nov 25, 2023

Bug 1993934: Merge upstream v4.2.0 #56

Bug 1993934: Merge upstream v4.2.0 #56

Conversation

bertinatto commented Aug 17, 2021 • edited Loading

openshift-ci bot commented Aug 17, 2021

bertinatto commented Aug 17, 2021

openshift-ci bot commented Aug 17, 2021

jsafrane commented Aug 18, 2021

jsafrane commented Aug 18, 2021

jsafrane commented Aug 18, 2021

bertinatto commented Aug 18, 2021

bertinatto commented Aug 18, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021

jsafrane commented Aug 19, 2021 • edited Loading

jsafrane commented Aug 19, 2021 • edited Loading

openshift-ci bot commented Aug 19, 2021

bertinatto commented Aug 19, 2021

bertinatto commented Aug 19, 2021

openshift-ci bot commented Aug 20, 2021

gnufied commented Aug 20, 2021

openshift-ci bot commented Aug 20, 2021

openshift-bot commented Nov 25, 2023

bertinatto commented Aug 17, 2021 •

edited

Loading

jsafrane commented Aug 19, 2021 •

edited

Loading

jsafrane commented Aug 19, 2021 •

edited

Loading