Fork all sbt tasks #1000
Merged
Fork all sbt tasks #1000
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
One caveat with this is that this seems to serialize the tests in the forked JVM... That isn't the behavior I intended here. Looking into this. |
seldridge
force-pushed
the
fork
branch
2 times, most recently
from
b087030 to
5ff347f
Compare
|
Sbt needs an additional option to re-enable parallel execution in the forked JVM. That option took too long to find. This should provide the same performance as before (documented above) without the metaspace issues that crop up in testing. |
seldridge
force-pushed
the
fork
branch
3 times, most recently
from
12805f7 to
98107b9
Compare
This causes sbt tasks (run, test, etc.) to fork to a separate JVM to avoid running out of metaspace. This issue crops up for developers or users repeatedly running sbt tasks in the same sbt session. Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
ucbjrl
approved these changes
Apr 16, 2019
ucbjrl
pushed a commit
that referenced
this pull request
Apr 24, 2019
This causes sbt tasks (run, test, etc.) to fork to a separate JVM to avoid running out of metaspace. This issue crops up for developers or users repeatedly running sbt tasks in the same sbt session. Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
jackkoenig
pushed a commit
that referenced
this pull request
Feb 28, 2023
…pbundles (#1000) * Attempt to deal with timing vagaries in UniquifySpec.quicklyrenamedeepbundles Switching to Scala 2.12.8 cause this test to start failing on OSX. Try earlier scheme to compare shallow vs deep to reduce brittleness. * Address review concerns; update comment.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This causes all
sbttest/run tasks (test,testOnly, andtestQuick,run,runMain,Test / run, andTest / runMain) to fork to a separate JVM. This avoids a situation where repeatedly running tasks exhausts the JVM's metaspace. This doesn't necessarily appear to be our problem (I think it's in sbt/scalaTest), but this buys us some time to figure it out.This happens for developers repeatedly running tasks (e.g.,
testorrun) in the same sbt session. Tasks that are run in a-DminimalResourcesenvironment (e.g., CircleCI) will not fork and will continue to run serially.More information:
The performance impact of this should only be the fork overhead. I spot checked this on a 6-core machine:
Baseline
Test / fork := trueTest / fork := true; Test / testForkedParallel := trueRelated issue: chipsalliance/firrtl#978
Type of change: bug report
Impact: no functional change
Development Phase: implementation
Release Notes
(No release notes as this is a testing-only change and we don't publish tests.)