-
Notifications
You must be signed in to change notification settings - Fork 921
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow use action archive cache to speed up workflow jobs. #2857
Conversation
474d7a3
to
5620568
Compare
@@ -778,11 +780,6 @@ private async Task DownloadRepositoryActionAsync(IExecutionContext executionCont | |||
executionContext.Output($"Download action repository '{downloadInfo.NameWithOwner}@{downloadInfo.Ref}' (SHA:{downloadInfo.ResolvedSha})"); | |||
} | |||
|
|||
await DownloadRepositoryActionAsync(executionContext, downloadInfo, destDirectory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i removed the nested method DownloadRepositoryActionAsync(arg1, arg2, arg3)
out of the DownloadRepositoryActionAsync(arg1, arg2)
5620568
to
95ee6cc
Compare
Trace.Info($"Save archive '{link}' into {archiveFile}."); | ||
try | ||
{ | ||
int retryCount = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i move the entire download logic behind the new DownloadRepositoryArchive()
Should not have impact on functionality.
@@ -796,7 +796,48 @@ private async Task DownloadRepositoryActionAsync(IExecutionContext executionCont | |||
Trace.Info($"Save archive '{link}' into {archiveFile}."); | |||
try | |||
{ | |||
await DownloadRepositoryArchive(executionContext, link, downloadInfo.Authentication?.Token, archiveFile); | |||
var useActionArchiveCache = false; | |||
if (executionContext.Global.Variables.GetBoolean("DistributedTask.UseActionArchiveCache") == true) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FF controlled by the service.
@@ -795,97 +792,50 @@ private async Task DownloadRepositoryActionAsync(IExecutionContext executionCont | |||
string link = downloadInfo?.TarballUrl; | |||
#endif | |||
|
|||
Trace.Info($"Save archive '{link}' into {archiveFile}."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
e870708
to
bdcbefc
Compare
bdcbefc
to
eee25df
Compare
This is exactly what we've been looking for to mitigate some API rate limiting issues. Can anyone provide an estimate for when this would go out in a release? |
@alexklibisz do you have a log shows me where you are getting rate limiting? i want to double check to make sure this feature actually can fix your problem. |
I don't have the exact log on hand, but it's something like |
@TingluoHuang
|
This feature will not help the Rate limit problem happen during |
That's too bad. What if we explicitly use |
@alexklibisz that's not a bad idea, we can make change to let the runner skip the resolving API call if you are referencing actions via |
Thanks! These rate limiting issues have been pretty disruptive, so anything that can reduce API calls when the jobs are starting is helpful. For context, we are using Actions Runner Controller with Horizontal Runner Autoscalers and Runner Deployments on top of AWS spot instances. We really like the scalability and the ephemeral nature of the runners. But as we've onboarded more projects, we're getting disrupted by these API rate limits basically daily. The API rate limits seem reasonable. It seems like there is some room for optimization in the runner initialization. I think these optimizations will be important as Github has officially adopted/recommended Actions Runner Controller and teams are starting to adopt it. |
@TingluoHuang Can you help me understand one thing more precisely:
Does downloading the action count against the API rate limit? In other words, will the change in this PR help at all with rate limit issues? |
@alexklibisz bad news is my change won't help rate limiting at all, there are some action policy we need to check when the runner try to use certain actions, so we have to call the resolving actions endpoint. 😞 If you are hitting rate limiting on your GHES, you can reach out to GitHub support, and they should be able to help you bump the rate limit setting on your GHES instance. |
To avoid network roundtrip for downloading each action's archive and speed up workflow jobs, we are introduce a new ENV to the runner, and the runner will try to find the action tarball/zipball of the resolved version in the action archive cache folder and avoid downloading it from the service.
The cache folder will be point by
ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE
env and have the following structure:For Linux/macOS:
For Windows:
The tarball or zipball will be the content from https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#download-a-repository-archive-tar
https://github.com/github/actions-assignment/issues/17