Adding documetation for VSTest Integration #402
Conversation
Codecov Report
@@ Coverage Diff @@
## master #402 +/- ##
=======================================
Coverage 86.96% 86.96%
=======================================
Files 16 16
Lines 2178 2178
=======================================
Hits 1894 1894
Misses 284 284 |
documentation/VSTestIntegration.md
Outdated
| #### Default | ||
| | Option | Summary | | ||
| |-------------|------------------------------------| | ||
| |Format | Results format in which coverage output is generated. Default format is cobertura.| |
There was a problem hiding this comment.
Should we provide default CoverletOutput?
With msbuild we are using MSBuildProjectDirectory https://github.com/tonerdo/coverlet/blob/0956d1e8da6b7265b0b5e4f93af3499c4be99cc8/src/coverlet.msbuild.tasks/coverlet.msbuild.props#L14 but there is some discussion in place #152
I'm in agreement with @eerhardt comment I prefer put it in output directory.
There was a problem hiding this comment.
We will only be getting the coverage output from coverlet.core APIs and the responsibility of sending the file will be with the datacollector.
By default, currently the files are put into the TestResults directory, Where this TestResults directory will be created is configurable via runsettings.
documentation/VSTestIntegration.md
Outdated
|
|
||
| | Entry point | How will code coverage be enabled? | Syntax | | ||
| |-------------|------------------------------------|----------------------------------------------------------------------| | ||
| |dotnet test CLI | Through a switch to condition data collection | `dotnet test --collect:"XPlat Code Coverage"` | |
There was a problem hiding this comment.
Is this real final name or only a placeholder?
There was a problem hiding this comment.
XPlat Code Coverage is the final name at present.
documentation/VSTestIntegration.md
Outdated
| <RunSettings> | ||
| <DataCollectionRunSettings> | ||
| <DataCollectors> | ||
| <DataCollector friendlyName="XPlat code coverage"> |
There was a problem hiding this comment.
Tech question, test runner search for a class with DataCollectorFriendlyName attribute named XPlat code coverage from output build folder, right?
There was a problem hiding this comment.
It searches for the class in various folders including the output build folder and the nuget folders.
Basically we'll call a target before VSTest that would add the nuget path of the datacollectors to the search directories(it will be picked up directly from the nuget folder)
documentation/VSTestIntegration.md
Outdated
|
|
||
|
|
||
| #### Scope of Enhancement | ||
| Currently, advanced options are supported via runsettings. Providing support through additional command line arguments in vstest can be taken up separately. |
There was a problem hiding this comment.
Providing support through additional command line arguments in vstest can be taken up separately.
Will be better user experience...but I think that it's require non trivial internal changes.
There was a problem hiding this comment.
Yes. This will include some internal vstest changes which we can pick up as an enhancement.
documentation/VSTestIntegration.md
Outdated
| ## Implementation Details | ||
| The proposed solution is implemented with the help of [datacollectors](https://github.com/Microsoft/vstest-docs/blob/master/docs/extensions/datacollector.md). | ||
| 1. Outproc Datacollector : The outproc collector would always run in a separate process(datacollector.exe/datacollector.dll) than the process in which tests are being executed(testhost*.exe/testhost.dll). This datacollector would be responsible for calling into coverlet APIs for instrumenting dlls, collecting coverage results and sending the coverage output file back to test platform. | ||
| 2. Inproc Datacollector : The inproc collector in the testhost process executing the tests. This collector will be needed to remove the dependency on the exit handler to flush the hit files. |
There was a problem hiding this comment.
I don't see config to plug inproc datacollector...am I missing something?
There was a problem hiding this comment.
The inproc datacollector will be automatically plugged in with --collect:"XPlat Code Coverage" . The inproc datacollector will lie in the same nuget and get picked from there.
There was a problem hiding this comment.
ah ok thank's now it's more clear in/out behaviours of my past tests.
There was a problem hiding this comment.
Workflow is enough clear to me at this point.
Wait for @tonerdo thoughts/question.
I could try to implement collectors with MS guys supervision if we agree on it.
@MarcoRossignoli Sure. |
No problem to me! |
|
@tonerdo before go on with this and let @vagisha-nidhi start to work on collector we should try to merge #393 so we'll use new contracts. |
|
@vagisha-nidhi can you capitalize folder name |
|
@MarcoRossignoli I have addressed the comments here. |
|
@MarcoRossignoli @tonerdo It'll be really convenient for me if you could provide an ETA on #393? |
|
Got some review from Toni #393 (comment), I'll do my best to replace PR with simpler new one in a pair of days...I won't be available between Thursday-Sunday for a fast holiday. |
|
@vagisha-nidhi #409 has been merged in |
|
@tonerdo we can merge this...if during developments something change we'll update |
|
@tonerdo This can be merged, right? |
Adding documetation for VSTest Integration. Laying out the details for the user experience and implementation.
contributes to #395