[WIP] Run test suite with executionOrder="random" #3450
Conversation
Majkl578
commented
Feb 6, 2019
Majkl578
commented
| Q | A |
|---|---|
| Type | improvement |
| BC Break | no |
| Fixed issues | #3449 |
Majkl578
force-pushed
the
randomized-test-suite
branch
from
2caded5
to
18ec6b9
Compare
|
@Ocramius Yes, see sebastianbergmann/phpunit#3515 + sebastianbergmann/phpunit#3524 - the seed will be displayed by default in the next PHPUnit release. 👍 |
Majkl578
force-pushed
the
randomized-test-suite
branch
from
18ec6b9
to
f958626
Compare
Majkl578
force-pushed
the
randomized-test-suite
branch
from
f958626
to
e2a25ae
Compare
| @@ -650,7 +652,7 @@ | |||
| "homepage": "https://nette.org/contributors" | |||
| } | |||
| ], | |||
| "description": "💎 Nette Dependency Injection Container: Flexible, compiled and full-featured DIC with perfectly usable autowiring and support for all new PHP 7.1 features.", | |||
| "description": "? Nette Dependency Injection Container: Flexible, compiled and full-featured DIC with perfectly usable autowiring and support for all new PHP 7.1 features.", | |||
There was a problem hiding this comment.
This looks like a composer bug on Windows.
There was a problem hiding this comment.
Did you run the update on Windows? 😱
There was a problem hiding this comment.
Yes. 😹 Composer should be portable, right? 😅
There was a problem hiding this comment.
Apparently, it's not only a Windows issue. I'm getting the same consistently on one of my two Linux machines (see #3604). Did you have a chance to look deeper or maybe report an issue? I haven't found anything related.
|
While we are fixing the failures, let's try to document what the end result should like from the CI perspective. IIRC, we considered running PR tests with a fixed seed in order to avoid irrelevant failures. The question is, what if a PR introduces an implicit dependency? Should cron-scheduled builds and the push ones have the same order settings? Or we just go wild and enable the random order everywhere? |
|
Closing in favor of #4731. There are no more known failures in failures in random mode but there doesn't seem to be a way to allow build failures on GitHub Actions for a certain configuration. |
