dev: replace stress with --runs_per_test
#104228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
|
This change is |
rickystewart
changed the title
stress with --runs_per_test
rickystewart
force-pushed
the
replace-stress-in-dev
branch
2 times, most recently
from
196234d
to
b87822d
Compare
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
b87822d
to
80f8e4e
Compare
|
No rush to merge this PR by the way, we can take our time to make sure this is working correctly. |
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
80f8e4e
to
6637424
Compare
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
6637424
to
5723df9
Compare
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
5723df9
to
48c804a
Compare
|
This is ready to review. |
healthy-pod
approved these changes
Jul 25, 2023
rail
approved these changes
Jul 26, 2023
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
48c804a
to
ef0fe46
Compare
`--runs_per_test` does the same thing that `stress` does (runs a test many times to weed out flakes), but better, in that: 1. Better Bazel support -- we had to hack Bazel support into `stress` to keep it working 2. Bazel gives us statistics and control over how the tests are scheduled in a way that is standardized as opposed to the ad-hoc arguments one can pass to `stress` 3. Bazel can collect logs and artifacts for different test runs and expose them to the user in a way that `stress` cannot 4. Drop-in support for remote execution when we have access to it, obviating the need for `roachprod-stress` For now, the default implementation of `dev test --stress` is to run the test 1,000 times, stopping the build if any test fails. This is meant to replicate how people use `stress` (i.e., just start it running and stop if there are any failures). The 1,000 figure is arbitrary and meant to just be a "very high number" of times to run unit tests. The default figure of 1,000 can be adjusted with `--count`. Also update documentation with some new recipes and add extra logging to warn about the change in behavior. Closes cockroachdb#102879 Epic: none Release note: None
rickystewart
force-pushed
the
replace-stress-in-dev
branch
from
ef0fe46
to
c232fa8
Compare
|
bors r=rail |
|
Build succeeded: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
--runs_per_testdoes the same thing thatstressdoes (runs a testmany times to weed out flakes), but better, in that:
stressto keep it working
scheduled in a way that is standardized as opposed to the ad-hoc
arguments one can pass to
stressexpose them to the user in a way that
stresscannotobviating the need for
roachprod-stressFor now, the default implementation of
dev test --stressis torun the test 1,000 times, stopping the build if any test fails.
This is meant to replicate how people use
stress(i.e., just start itrunning and stop if there are any failures). The 1,000 figure is
arbitrary and meant to just be a "very high number" of times to run unit
tests. The default figure of 1,000 can be adjusted with
--count. Alsoupdate documentation with some new recipes and add extra logging to
warn about the change in behavior.
Closes #102879
Epic: none
Release note: None