Run more tests concurrently in CI #1514
Conversation
jyemin
force-pushed
the
evergreen-experiment
branch
from
2c6997a
to
de54e61
Compare
| - name: "test-reactive" | ||
| - name: "test-core-and-legacy" | ||
|
|
||
| - matrix_name: "tests-unit" |
There was a problem hiding this comment.
Just run unit tests once on each JDK, instead of with every topology and release of MongoDB.
.evergreen/.evg.yml
Outdated
| - func: "bootstrap mongo-orchestration" | ||
| - func: "run tests" | ||
| vars: | ||
| TESTS: 'driver-core:test driver-legacy:test' |
There was a problem hiding this comment.
Collectively these take around 4 minutes to run on a replica set
There was a problem hiding this comment.
Seems a strange combo - would it make more sense for legacy tests to be in with sync tests?
There was a problem hiding this comment.
Logically, yes. But mostly this is about balancing the test run times, and the driver-sync tests are already the slowest at around 12 minutes, whereas driver-core and driver-legacy together are less than 5 minutes.
There was a problem hiding this comment.
I decided to split them out to separate tasks
* Run sync tests together * Run reactive tests together * Run core and legacy together * Run bson and crypt tests together JAVA-5628
jyemin
force-pushed
the
evergreen-experiment
branch
from
de54e61
to
dfc553e
Compare
|
I like it. One comment about the legacy tests but it looks good 👍 |
|
So kotlin tests are duplicated here but the Scala driver tests aren't. As we already limit the versions that we run kotlin / scala on and Scala tests multiple scala runtimes. I'd recommend removing them from the bson and reactive tests. |
* Separate legacy tests
There was a problem hiding this comment.
LGTM
I have an optional suggestion - put the "test" prefix after.
eg:
- "test-sync" -> "sync-driver-tests"
- "test-reactive" -> "reactive-streams-driver-tests"
- "test-core" -> "core-driver-tests"
- "test-legacy" -> "legacy-driver-tests"
Ideally I'd want to use a display_name and whitespace in the evergreen UI but doesn't look like its supported 🤷
It reads better with |
|
Added @katcharov to get one more set of eyes on this. |
Make these suites run concurrently on the main matrices
Additionally, run unit tests once per JDK rather than on every MongoDB deployment variant
JAVA-5628
Patch builds show that the slowest task drops from ~45 minutes to ~20 minutes