-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Github rate limit #8426
Comments
/triage accepted |
@fabriziopandini: GuidelinesPlease ensure that the issue body includes answers to the following questions:
For more details on the requirements of such an issue, please see here and ensure that they are met. If this request no longer meets these requirements, the label can be removed In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Q1: I think lots of stuff in prow uses the ghproxy. But does not look like a real option here. Q2: what about adding the GITHUB_TOKEN, as we do for But cluster-api seems to be very unique in doing this 🤔, there not much jobs mounting a github token in test-infra. |
I didn't realize this existed! Why wouldn't it work here in your opinion? I'm not sure how mounting the token works for that job either - need to do a bit of digging. |
Its heavily used by prow components itself to cache requests and not hit rate limiting :D. I think its currently not an option, because we have no ability to reconfigure the github url?! I also don't know if ghproxy is supposed to get used by prowjobs or only by prow components. |
Based on recent discussions we're trying to set the GitHub token for the jobs right? I would like to avoid introducing a dependency to the ghproxy. It feels like depending on an implementation detail of Prow |
It looks like this is an even more substantial problem on the community eks cluster. The release-1.3 e2e testing jobs are failing ~80 percent of the time due to this error: Ref:
The second link isn't obvious from the triage page, but clicking through it's the same error but hit at an earlier point during |
Yup. Looks to me like now we actually have to work on providing a proper GITHUB_TOKEN |
Thanks for working on a solution! I haven't done a review of those PRs yet - but I think we'll still have the problem with any release we don't backport those to - e.g. clusterctl upgrade tests for 1.0, 1.2, 1.3. This should massively reduce the problem which is great for our test signal, but I think we should still push for getting a github token on the EKS cluster as a more permanent solution here. |
+1 Was thinking exactly the same. It's good going forward but especially because of v1.0 it will take years to not require a GITHUB_TOKEN in CI |
+1 |
Yup let's see. But If possible I would like to avoid new v1.0, v1.2 and v1.3 patch releases |
As of 27 Oct 2023 this issue does not seem to be occurring anymore, including on the release-1.3 tests which are running on the EKS community cluster. Closing this, but we can reopen if this becomes a problem again in future. /close |
@killianmuldoon: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Which jobs are flaking?
CAPI e2e main
Which tests are flaking?
e.g. When testing clusterctl upgrades using ClusterClass (v1.2=>current) [ClusterClass] Should create a management cluster and then upgrade all the providers
Since when has it been flaking?
For some months.
Testgrid link
https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/periodic-cluster-api-e2e-main/1641014387735007232
Reason for failure (if possible)
Multiple jobs in the CAPI e2e test run are flaking due to hitting a rate limit with the Github API.
The error looks like:
Anything else we need to know?
One possible solution (from @sbueringer): vendor the cert-manager yaml in most tests to cut down on this flake. We could consider vendoring it for all tests, but we would lose coverage of the cert-manager download in e2e testing which is a trade-off.
Label(s) to be applied
/kind flake
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.
The text was updated successfully, but these errors were encountered: