[SUREFIRE-2041] Ordering test classes and methods according to -Dtest property #560
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is a new implementation for PR#348 and PR#495.
The changes in this pull request enable Surefire to run test classes and test methods in any order specified by a newly added <runOrder> (testorder).
PR#348 attempted to implement this feature but @Tibor17 suggested that such a feature should be implemented differently. PR#495 supports only test class ordering, while this PR supports test class ordering and test method ordering within each test class.
Overall, the changes include
The newly added testorder <runOrder> is supported for JUnit 3 and JUnit 4 tests. We have also implemented support for JUnit 5, but we are unsure on how we should get and sort all the test methods before sorting them for JUnit 5. Unlike JUnit 4, JUnit5 does not provide a interface like
org.junit.runner.Request#sortWith()
in JUnit4 to help us sort the methods. To help with this issue, we current rely on Java reflection to get the methods and then sort them. For this reason, we created another PR to show that change, but we left the changes out of this PR. Please let us know if you have any suggestions for supporting JUnit 5.
Simple (no regex) example using Http-Request:
Output from Maven should say that the test class
HttpRequestTest
ran beforeEncodeTest
The file
HttpRequestTest.xml
(http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml
) should say
Regex example using Http-Request:
Output from Maven should say
The file
HttpRequestTest.xml
(http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml
) should say
Include/Exclude example using Http-Request:
Output from Maven should say
The file
HttpRequestTest.xml
(http-request/lib/target/surefire-reports/TEST-com.github.kevinsawicki.http.HttpRequestTest.xml
) should say