Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add QPS & burst options & flags for ClusterCacheTracker #10880

Merged
merged 1 commit into from
Jul 17, 2024

Conversation

sbueringer
Copy link
Member

Signed-off-by: Stefan Büringer buringerst@vmware.com

What this PR does / why we need it:
Before this PR we were using client-go defaults of 5 & 10 qps & burst. This PR changes the defaults to use the same as we use for the mgmt cluster client. It also makes these values configurable to allow fine-tuning at scale.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-area PR is missing an area label size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 16, 2024
@sbueringer sbueringer added this to the v1.8 milestone Jul 16, 2024
@sbueringer sbueringer added the area/clustercachetracker Issues or PRs related to the clustercachetracker label Jul 16, 2024
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/needs-area PR is missing an area label label Jul 16, 2024
@sbueringer
Copy link
Member Author

/assign @fabriziopandini @chrischdi

@sbueringer
Copy link
Member Author

/test pull-cluster-api-e2e-main

Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold

bootstrap/kubeadm/main.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 16, 2024
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 967378494ff97ec7a2a0aba9173dc7e9fc641309

Copy link
Member

@chrischdi chrischdi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

ok for me to cleanup flags as follow-up

bootstrap/kubeadm/main.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024
@sbueringer
Copy link
Member Author

Let's keep the hold until after the beta tag is created

Copy link
Member

@neolit123 neolit123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 2349030491e1e1e769055962df11fa59c7cdc953

Copy link
Member

@fabriziopandini fabriziopandini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit (no strong opinion)
Also, should we use new flags to increase QPS/Burst values in testenv?

bootstrap/kubeadm/main.go Outdated Show resolved Hide resolved
Signed-off-by: Stefan Büringer buringerst@vmware.com
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024
@fabriziopandini
Copy link
Member

/lgtm
/hold for the release

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 16, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 9b380c62de0715a90e66ea3482ddff9cd51e4739

@sbueringer
Copy link
Member Author

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 16, 2024
@sbueringer
Copy link
Member Author

(needs approve though @fabriziopandini @chrischdi )

@neolit123
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 16, 2024
@neolit123
Copy link
Member

/meow space

@k8s-ci-robot
Copy link
Contributor

@neolit123: cat image

In response to this:

/meow space

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Comment on lines +156 to +162

if opts.ClientQPS == 0 {
opts.ClientQPS = 20
}
if opts.ClientBurst == 0 {
opts.ClientBurst = 30
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I interpret that this case will be occurred when the someone specifies ClientQPS or ClientBurst to 0. If it fills out with default value arbitrarily, someone may think the 0 is valid value. So I think this case should return error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, this cases get in two cases:

  • when an implementer does not set ClientQPS and/or ClientBurst on the opts ClusterCacheTrackerOptions
  • when an implementer sets them, but the user passes in 0 as value

IMHO its fine as is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @chrischdi

Thank you for teaching me 👍

when an implementer sets them, but the user passes in 0 as value

I just worried whether it's better that we let implementer know about the information when the actual value of ClientQPS and/or ClientBurst is different from args implementer sets. But I think it is fine too. Thanks.

Copy link
Member Author

@sbueringer sbueringer Jul 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem here is that we don't know if someone set the fields explicitly to 0, or they are just 0 because 0 is the "zero value" of these fields.

I don't want to force everyone to set these values, which would happen if we always return an error if the fields are 0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sbueringer
I understood it and I agree with your opinion. Surely, this field is option as it is named and it should not force everyone to set these.
Thanks.

@chrischdi
Copy link
Member

/retest

@k8s-ci-robot k8s-ci-robot merged commit 3e47858 into kubernetes-sigs:main Jul 17, 2024
21 checks passed
@sbueringer sbueringer deleted the pr-cct-qps-burst branch July 17, 2024 08:50
jimmidyson pushed a commit to jimmidyson/cluster-api that referenced this pull request Jul 17, 2024
…urst

✨ Add QPS & burst options & flags for ClusterCacheTracker
@vincepri
Copy link
Member

Can we backport this @sbueringer ?

@sbueringer
Copy link
Member Author

Yup

/cherry-pick release-1.7

@k8s-infra-cherrypick-robot

@sbueringer: #10880 failed to apply on top of branch "release-1.7":

Applying: Add QPS & burst options & flags for ClusterCacheTracker
Using index info to reconstruct a base tree...
M	bootstrap/kubeadm/main.go
M	controllers/remote/cluster_cache_tracker.go
M	controlplane/kubeadm/main.go
A	docs/book/src/developer/providers/migrations/v1.7-to-v1.8.md
M	main.go
M	test/extension/main.go
M	test/infrastructure/docker/main.go
M	test/infrastructure/inmemory/main.go
Falling back to patching base and 3-way merge...
Auto-merging test/infrastructure/inmemory/main.go
Auto-merging test/infrastructure/docker/main.go
Auto-merging test/extension/main.go
Auto-merging main.go
CONFLICT (modify/delete): docs/book/src/developer/providers/migrations/v1.7-to-v1.8.md deleted in HEAD and modified in Add QPS & burst options & flags for ClusterCacheTracker. Version Add QPS & burst options & flags for ClusterCacheTracker of docs/book/src/developer/providers/migrations/v1.7-to-v1.8.md left in tree.
Auto-merging controlplane/kubeadm/main.go
Auto-merging controllers/remote/cluster_cache_tracker.go
Auto-merging bootstrap/kubeadm/main.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Add QPS & burst options & flags for ClusterCacheTracker
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

Yup

/cherry-pick release-1.7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@sbueringer
Copy link
Member Author

If you open a cherry-pick PR 😀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/clustercachetracker Issues or PRs related to the clustercachetracker cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants