Surefire Report order results

[nhojpatrick]

Have rebased PR #208 from master.

Will raise Jira Ticket, rename commits, confirm license and icla, if this stands a chance of getting merged.

Following this checklist to help us incorporate your
contribution quickly and easily:

• Make sure there is a JIRA issue filed
  for the change (usually before you start working on it). Trivial changes like typos do not
  require a JIRA issue. Your pull request should address just this issue, without
  pulling in other changes.
• Each commit in the pull request should have a meaningful subject line and body.
• Format the pull request title like [SUREFIRE-XXX] - Fixes bug in ApproximateQuantiles,
  where you replace SUREFIRE-XXX with the appropriate JIRA issue. Best practice
  is to use the JIRA issue title in the pull request title and in the first line of the
  commit message.
• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• Run mvn clean install to make sure basic checks pass. A more thorough check will
  be performed on your pull request automatically.
• You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an
Individual Contributor License Agreement if you are unsure
please ask on the developers list.

To make clear that you license your contribution under
the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• In any other case, please file an Apache Individual Contributor License Agreement.

[slawekjaranowski]

Please follow [SUREFIRE-XXX] in commit subject and PR title.

[olamy]

CheckTestNgVersionsIT is failing.
I'm not sure if the test is really relevant now with this change.

[nhojpatrick]

Please follow [SUREFIRE-XXX] in commit subject and PR title.

@slawekjaranowski please read 2nd line of PR description.

[olamy]

Please follow [SUREFIRE-XXX] in commit subject and PR title.

@slawekjaranowski please read 2nd line of PR description.

@nhojpatrick we do have sone automation tooling which parse subject etc.. to automatically update jira issues and generate release notes.
So as it we are sure to link a PR or any commit to jira issue.
if we integrate your change as is we do not have any way to know (when looking at history) to what this commit is fixing etc...

[slawekjaranowski]

In #208 we have linked issue https://issues.apache.org/jira/browse/SUREFIRE-1615 which has topic:
surefire.includesProperty is ignored
it is something different what you do in this PR.
So please create new issue with description what problem / feature you want to resolve.

[nhojpatrick]

In #208 we have linked issue https://issues.apache.org/jira/browse/SUREFIRE-1615 which has topic: surefire.includesProperty is ignored it is something different what you do in this PR. So please create new issue with description what problem / feature you want to resolve.

The original PR #208 appeared to be stable, so I wanted to help out.

I tried to rebase the original PR branch, but that was a lot of conflicts, so then tried cherry-pick, then just tried to implement it again.

So looking at https://issues.apache.org/jira/browse/SUREFIRE-1615 it talks about the includeProperties, but reading the authors description his requirement or usage was to create reproducible output order based upon the test and not the order of execution.

[Tibor17]

@nhojpatrick
The entries cannot be reordered as you like.
First the order is defined by the configuration and the XML reporter respects the execution order by runOrder. All the inclusions and exclusions config params have nothing to do with the test order.
Second, it does not make sense to do anything about TestSetRunListener and StatelessXmlReporter like this because the purpose according to the roadmap is make a refactoring exactly of these entries. That's the reason why we have a new property testRunId in ReportEntry because it is the prereq of the refactoring. If we apply this, we would again spend the time to analyse what custom requirements have been done, evaluate what they broke, etc. etc. etc.
Regarding the run order, there is #348 we discussed yesterday. There are people from the Illinios University like Winglam and Professor Jonathan Bell who has helped us with several other features.
If you still think that the order of tests in the XML should be ordered other way than runOrder, you should follow StatelessReportMojoConfiguration, but still the PRs like this would complicate refactoring and it is much safer to make the refactoring first, then use builder for the StatelessXmlReporter constructor, and then apply the customer features.