Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Size of github actions cache continually increasing #465

Closed
ezimanyi opened this issue May 7, 2022 · 1 comment · Fixed by #466
Closed

Size of github actions cache continually increasing #465

ezimanyi opened this issue May 7, 2022 · 1 comment · Fixed by #466
Labels
area: cache enhancement New feature or request

Comments

@ezimanyi
Copy link
Contributor

ezimanyi commented May 7, 2022

I've been using this action on a repository for a while, and the size of the github actions cache has been continuously increasing.

It's now up to ~1.3 GB, and most of the action's time is spent downloading and re-uploading the cached directories. A recent run spent 56s fetching the cache, 18s linting, and 1m27s re-uploading the cache. (This is for a reasonably small repository; the total size of code and all downloaded dependencies is ~20 MB.)

I believe this is primarily because the package cache never prunes old dependencies, so as dependencies are updgraded we end up with all dependency versions ever used by the project in the package cache.

It looks like the github actions cache key is generated as golangci-lint.cache-[sequence num]-[go.mod checksum] where the sequence num rotates every 7 days. When restoring we're passing the following as restore keys:

  • golangci-lint.cache-
  • golangci-lint.cache-[sequence num]-
  • golangci-lint.cache-[sequence num]-[go.mod checksum]

This means that even when the sequence number rotates, we'll still restore the previous cache because of the golangci-lint.cache-, use it for the build, then re-upload using a new sequence number. As a result, all the cached packages are carried forward as we never start with a clean build.

If there were a way to have go prune the package cache to remove unneeded packages that might be a good option to run at the end of the build before uploading the cache, but I don't think such an option exists. (You can only fully clear the package cache and regenerate it, but of course doing this on every build would defeat the purpose of having the cache at all.)

I think the best solution here is probably just to remove golangci-lint.cache- from the restore keys. This will force a clean build every time the sequence number rotates (once every 7 days) so we'll avoid continuously carrying forward all prior dependencies of the project. (Based on the comment in the code I think that may have been the intention initially.)

@SVilgelm
Copy link
Member

@SVilgelm SVilgelm added the enhancement New feature or request label May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: cache enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants