Post-test crashes - Incorrect assumptions in Blame logic

[vikukush]

In the BlameCollector.cs there is this comment:

            // If the last test crashes, it will not invoke a test case end and therefore
            // In case of crash testStartCount will be greater than testEndCount and we need to write the sequence
            // And send the attachment. This won't indicate failure if there are 0 tests in the assembly, or when it fails in setup

And then the logic compares number of tests ran vs number of tests total and concludes that if they're equal, then there was no crash.

In case testhost.exe crashes during wind down after executing the tests, in [AssemblyCleanup] or elsewhere, this logic will assume that there was no crash since all test have been executed, even though procdump will create a dump.

Blame should (also?) examine testhost.exe exit code to see if it was successful.

[nohwnd]

Yes you are correct, but the process is not owned by datacollector, so we need to have another message coming from vstest.console to datacollector that sends the exit code.

And even in that case, we don't know if the crash happned or not, because user can exit with 0 and we still can consider that a crash if it happened too early, but that is not happening that often.

This is a new feature and we are not adding new features at the moment to vstest, so this won't be implemented.