Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cache for connections #159

Closed
haorenfsa opened this issue Nov 28, 2022 · 1 comment · Fixed by #179
Closed

Add cache for connections #159

haorenfsa opened this issue Nov 28, 2022 · 1 comment · Fixed by #179
Labels
enhancement New feature or request

Comments

@haorenfsa
Copy link

What problem are you facing?

provider-helm has client-side throttling every time connect to the cluster

k -n crossplane-system logs deploy/provider-helm-3d2f09bcd965 --since 1m -f
I1128 12:43:29.919990       1 request.go:665] Waited for 1.190135507s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/storagetransfer.cnrm.cloud.google.com/v1beta1?timeout=32s
I1128 12:43:40.120392       1 request.go:665] Waited for 11.390239384s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/milvus.io/v1beta1?timeout=32s
I1128 12:43:50.127379       1 request.go:665] Waited for 6.190781086s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/sfn.aws.crossplane.io/v1alpha1?timeout=32s
I1128 12:44:00.336327       1 request.go:665] Waited for 1.189155236s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/milvus.io/v1alpha1?timeout=32s
I1128 12:44:10.336621       1 request.go:665] Waited for 11.188964474s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/cloudidentity.cnrm.cloud.google.com/v1beta1?timeout=32s
I1128 12:44:20.345096       1 request.go:665] Waited for 5.989111872s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/binaryauthorization.cnrm.cloud.google.com/v1beta1?timeout=32s
I1128 12:44:31.086707       1 request.go:665] Waited for 1.174828735s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/servicedirectory.cnrm.cloud.google.com/v1beta1?timeout=32s

How could Crossplane help solve your problem?

Add a connection cache to avoid call discovery api every time
@haorenfsa haorenfsa added the enhancement New feature or request label Nov 28, 2022
@haorenfsa
Copy link
Author

Maybe add cache to the runtime framework is a better way to solve this

ytsarev added a commit to ytsarev/provider-helm that referenced this issue Feb 25, 2023
@ytsarev
Configure higher burst to avoid client side throttling
57aca38 
* Fixes crossplane-contrib#159

* No more client-side throttling timeouts like

```
I1128 12:44:10.336621       1 request.go:665] Waited for 11.188964474s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/cloudidentity.cnrm.cloud.google.com/v1beta1?timeout=32s
I1128 12:44:20.345096       1 request.go:665] Waited for 5.989111872s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/binaryauthorization.cnrm.cloud.google.com/v1beta1?timeout=32s
I1128 12:44:31.086707       1 request.go:665] Waited for 1.174828735s due to client-side throttling, not priority and fairness, request: GET:https://10.255.0.1:443/apis/servicedirectory.cnrm.cloud.google.com/v1beta1?timeout=32s
```

* 300 value is aligned with associated kubectl fix at kubernetes/kubernetes#109141

Signed-off-by: Yury Tsarev <yury@upbound.io>
@ytsarev ytsarev mentioned this issue Feb 25, 2023
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Configure higher burst limit to avoid client side throttling ytsarev/provider-helm
1 participant
@haorenfsa