Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate CI jobs, add job caching #357

Merged
merged 1 commit into from
Aug 26, 2020

Conversation

AngelOnFira
Copy link
Contributor

This PR separates CI jobs for better parallelism across runners. There are now 5 separate jobs that run. A build and test for stable and nightly, as well as a "clean" job. Clean includes both cargo fmt and cargo clippy as they are both quciker than normal cargo check or cargo test, and so they will complete around the same time.

As we can see below, there were definitely some issues with adding caching to CI vanilla. There were a few improvements I added:

  • Turn off incremental builds 1, 2. This decreased the size of the cache from about 2GB to 600MB.
  • Added the compiler flag -C debuginfo=0, which decreased the size further. I don't think this will have negative effects with cargo test or cargo check.
  • Only save the target folder. Normally, both the registry and the git folder are saved. These can be redownloaded fast enough, and would just clutter the zip with more small files to open upon downloading the cache.
total time cache size open cache time cargo test time create cache time
stable, no cache 8:20 - - 8:06 -
nightly, no cache 7:50 - - 7:29 -
stable, cache, incremental 9:04 2202 MB 3:34 5:07 ~ 4:00
nightly, cache, incremental 8:56 2186 MB 3:34 4:51 ~ 4:00
stable, cache, non incremental 4:22 586 MB 0:42 3:24 0:23
nightly, cache, non incremental 4:52 600 MB 0:40 3:32 0:22

Likely CI will change further with the addition of more steps, however, this should be a good starting point for separation of jobs. Some unknowns that I will have to examine once this is running on master:

  • How does the cache work once a new nightly is released the day after? The cache won't be busted unless Cargo.lock changes, so another busting method might be needed for nightlies.

@qarmin qarmin mentioned this pull request Aug 26, 2020
8 tasks
@cart
Copy link
Member

cart commented Aug 26, 2020

Brilliant work. Thanks!

@cart cart merged commit 4562a09 into bevyengine:master Aug 26, 2020
@AngelOnFira AngelOnFira deleted the add-separate-ci-jobs branch August 28, 2020 10:36
@CleanCut CleanCut mentioned this pull request Sep 2, 2020
mrk-its pushed a commit to mrk-its/bevy that referenced this pull request Oct 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants