You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When I create multiple action sets simultaneously, they are processed with delays on Kanister side.
To Reproduce
Steps to reproduce the behavior:
Add debug messages into onAdd event handler in Kanister's pkg/controller/controller.go file.
Create multiple action sets simultaneously
Observe in Kanister log that onAdd debug messages appear with delays.
Expected behavior
I suppose that processing of multiple action sets should be faster. At the moment this looks like a performance bottleneck for applications using Kanister.
Then in a test application I created 10 action sets at once via golang k8s api, waited for their completion and deleted all of them. Observed the following in logs of a test application:
"Time: 20220922-16:07:35.351Z, Message Created ActionSet pvc-6-ngir57wj" <---- Such messages are logged right after calling ActionSet.Create()
"Time: 20220922-16:07:36.123Z, Message Created ActionSet pvc-1-ngil6thx"
"Time: 20220922-16:07:39.113Z, Message Created ActionSet pvc-9-ngikbwr6"
"Time: 20220922-16:07:39.917Z, Message Created ActionSet pvc-3-ngixrv48"
"Time: 20220922-16:07:40.708Z, Message Created ActionSet pvc-5-ngipkztn"
"Time: 20220922-16:07:42.316Z, Message Created ActionSet pvc-8-ngizglzm"
"Time: 20220922-16:07:42.520Z, Message Created ActionSet pvc-10-ngt67pv"
"Time: 20220922-16:07:43.711Z, Message Created ActionSet pvc-2-ngifjqwn"
"Time: 20220922-16:07:44.509Z, Message Created ActionSet pvc-4-ngix545n"
"Time: 20220922-16:07:44.716Z, Message Created ActionSet pvc-7-ngiz5nrt"
"Time: 20220922-16:07:52.116Z, Message Deleted ActionSet pvc-6-ngir57wj" <---- Such messages are logged right after ActionSet.Delete()
"Time: 20220922-16:07:54.313Z, Message Deleted ActionSet pvc-1-ngil6thx"
"Time: 20220922-16:07:58.332Z, Message Deleted ActionSet pvc-9-ngikbwr6"
"Time: 20220922-16:08:00.142Z, Message Deleted ActionSet pvc-3-ngixrv48"
"Time: 20220922-16:08:10.946Z, Message Deleted ActionSet pvc-5-ngipkztn"
"Time: 20220922-16:08:21.973Z, Message Deleted ActionSet pvc-8-ngizglzm"
"Time: 20220922-16:08:32.215Z, Message Deleted ActionSet pvc-10-ngt67pv"
"Time: 20220922-16:08:52.594Z, Message Deleted ActionSet pvc-2-ngifjqwn"
"Time: 20220922-16:08:53.194Z, Message Deleted ActionSet pvc-4-ngix545n"
"Time: 20220922-16:08:53.782Z, Message Deleted ActionSet pvc-7-ngiz5nrt"
Thanks for opening this issue 👍. The team will review it shortly.
If this is a bug report, make sure to include clear instructions how on to reproduce the problem with minimal reproducible examples, where possible. If this is a security report, please review our security policy as outlined in SECURITY.md.
If you haven't already, please take a moment to review our project's Code of Conduct document.
Describe the bug
When I create multiple action sets simultaneously, they are processed with delays on Kanister side.
To Reproduce
Steps to reproduce the behavior:
onAdd
event handler in Kanister'spkg/controller/controller.go
file.onAdd
debug messages appear with delays.Expected behavior
I suppose that processing of multiple action sets should be faster. At the moment this looks like a performance bottleneck for applications using Kanister.
Environment
Kubernetes Version/Provider: 1.21
Storage Provider: GKE
Cluster Size (#nodes): 3
Additional context
As an experiment I modified
pkg/controller/controller.go
with the following log calls:Then in a test application I created 10 action sets at once via golang k8s api, waited for their completion and deleted all of them. Observed the following in logs of a test application:
and on Kanister side:
Here we can see that there's 44 sec delay between moments of adding pvc-6-ngir57wj and pvc-7-ngiz5nrt.
The text was updated successfully, but these errors were encountered: