Rate limit for web3signer_tests on CI

[michaelsproul]

Description

We're occasionally getting rate-limited calling the Github API to download the web3signer_tests. E.g. https://github.com/sigp/lighthouse/actions/runs/3708544134/jobs/6286220721

The code triggering the rate limit is:

lighthouse/testing/web3signer_tests/build.rs

Lines 35 to 45 in bf533c8

	// Get the latest release of the web3 signer repo.
	let latest_response: Value = client
	.get("https://api.github.com/repos/ConsenSys/web3signer/releases/latest")
	.send()
	.await
	.unwrap()
	.error_for_status()
	.unwrap()
	.json()
	.await
	.unwrap();

Running that API call in a loop in my terminal I can trigger a 403 response by making more than 60 requests/second:

{"message":"API rate limit exceeded for x.x.x.x. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}

Steps to resolve

I think the best option would be to plumb the GITHUB_TOKEN into that build script so that it can use it to bypass the rate limit. The build script could read an optional secret from its env when it runs and add this to the request headers (when running locally the header would not be added).

I think if we do something like echo ${{ secrets.GITHUB_TOKEN }} >> $GITHUB_ENV that might be OK, as long as Github scrubs the token from the logs. Tokens are unique to each CI run and have very limited read permissions on pull requests. See: https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token

[michaelsproul]

This is still an issue despite my fix. More debugging required

Maybe @antondlr could be tempted? 😏

Recent failure here: https://github.com/sigp/lighthouse/actions/runs/4180285702/jobs/7241123847

[michaelsproul]

I recall Anton saying that even authenticated requests are subject to a rate limit

[jimmygchen]

ah ok looks like with authenticated request, you raised the limit from 60 per hour to 1000 per hour.

https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limits-for-requests-from-github-actions

[jimmygchen]

Pushed a fix in #4163 to only download the web3signer binary when need (web3signer-tests), and only download once. This should reduce the API calls.

[jimmygchen]

Resolved via #4163