[Question] how Distinguish Describe from Context block in NunitXML #1284
Comments
|
Well, as you can see from the XML structure, the Context shows up as a nested, child
So basically, the outer-most HTH. |
|
I think you missed the point @fourpastmidnight I cannot trust your saing of
because how the pester DSL is currently built, I could have hundereds of describes and contextes one in each other. See this example Describe "My DEscribe block"{
Describe "My DEscribe block 2"{
Context "Context block 2" {
Context "Context block 3" {
}
Context "Context block 4" {
Describe "Describe block 3"{
it "My It block 1"{
$false | should be $false
}
it "My It block 2"{
$true | should be $false
}
}
}
}
}
}Which creates the following NunitXML <?xml version="1.0" encoding="utf-8" standalone="no"?>
<test-results xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="nunit_schema_2.5.xsd" name="Pester" total="2" errors="0" failures="1" not-run="0" inconclusive="0" ignored="0" skipped="0" invalid="0" date="2019-04-03" time="07:15:32">
<environment user="TAAVAST3" machine-name="U279610" cwd="C:\Users\taavast3\OneDrive\Repo\Projects\OpenSource\PsNunit" user-domain="CORPROOT" platform="Microsoft Windows 10 Enterprise|C:\Windows|\Device\Harddisk0\Partition4" nunit-version="2.5.8.0" os-version="10.0.16299" clr-version="4.0.30319.42000" />
<culture-info current-culture="en-US" current-uiculture="en-US" />
<test-suite type="TestFixture" name="Pester" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="Pester">
<results>
<test-suite type="TestFixture" name="C:\Users\taavast3\OneDrive\Repo\Projects\OpenSource\PsNunit\_TestMaterial\Woop.Tests.ps1" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="C:\Users\taavast3\OneDrive\Repo\Projects\OpenSource\PsNunit\_TestMaterial\Woop.Tests.ps1">
<results>
<test-suite type="TestFixture" name="My DEscribe block" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="My DEscribe block">
<results>
<test-suite type="TestFixture" name="My DEscribe block 2" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="My DEscribe block 2">
<results>
<test-suite type="TestFixture" name="Context block 2" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="Context block 2">
<results>
<test-suite type="TestFixture" name="Context block 3" executed="True" result="Success" success="True" time="0" asserts="0" description="Context block 3">
<results />
</test-suite>
<test-suite type="TestFixture" name="Context block 4" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="Context block 4">
<results>
<test-suite type="TestFixture" name="Describe block 3" executed="True" result="Failure" success="False" time="2.7653" asserts="0" description="Describe block 3">
<results>
<test-case description="My It block 1" name="My DEscribe block.My DEscribe block 2.Context block 2.Context block 4.Describe block 3.My It block 1" time="2.5251" asserts="0" success="True" result="Success" executed="True" />
<test-case description="My It block 2" name="My DEscribe block.My DEscribe block 2.Context block 2.Context block 4.Describe block 3.My It block 2" time="0.2402" asserts="0" success="False" result="Failure" executed="True">
<failure>
<message>Expected $false, but got $true.</message>
<stack-trace>at <ScriptBlock>, C:\Users\taavast3\OneDrive\Repo\Projects\OpenSource\PsNunit\_TestMaterial\Woop.Tests.ps1: line 14
14: $true | should be $false</stack-trace>
</failure>
</test-case>
</results>
</test-suite>
</results>
</test-suite>
</results>
</test-suite>
</results>
</test-suite>
</results>
</test-suite>
</results>
</test-suite>
</results>
</test-suite>
</test-results>Imagine my blocks were not so obviously named, and try to get only the context blocks using xpath? I would actually expect a 'property' somehwere in the 'test-suite' node that would make this differentiation. Since 'type' is already used, something like 'pesterblocktype' or 'pestertype' would make sense to me. Is that something that already exists ? Would it make sense to think of having something like that? |
|
@Stephanevg The XML format we use is NUnit (2.5). That format has no idea about contexts or describes. For Pester those blocks are also virtually the same, the only place where they are distinguished is That said it could be added into the metadata, there was work going on to move to NUnit 3.0 which is much better documented than 2.5 and so adding this metadata might become part of that. Why are you parsing the XML output, is the information you get from |
|
@nohwnd The original need I have is that I wanted to persist the results from a Pester run to have a historical view of the runs (at what date did test X stopped beeing successfull for example). I just used the NunitXML format as it seemed to me that that the official output was for persistance scenarios. I also had the impression I could always trust that format, as it is a defined standard, and not 'only' for Pester. That beeing said, I am not closed to change my methodology. I could convert the ouptut to JSON, if that would be the recommended approach. Also, it might be interesting to add a value |
|
There are several tools online, which I already mentioned, which parse this format. Such as NunitHtmlReportGenerator, and ReportUnit. They are both on Github and perhaps you could use those as the basis for what you’re trying to do.
I think a Json output would be a nice addition, too, it’s a bit easier to work with, especially for manipulating with JS to create nice HTML-based reports. I’m sure Rspec probably has a Json output format, Cucumber most certainly does.
…Sent from my Windows 10 phone
From: Van Gulick
Sent: Wednesday, April 3, 2019 5:20
To: pester/Pester
Cc: Craig E. Shea; Mention
Subject: Re: [pester/Pester] [Question] how Distinguish Describe from Contextblock in NunitXML (#1284)
@nohwnd The original need I have is that I wanted to persist the results from a Pester run to have a historical view of the runs (at what date did test X stopped beeing successfull for example).
The data I want to parse could be old output, which would be persisted in a DB.
I just used the NunitXML format as it seemed to me that that the official output was for persistance scenarios. I also had the impression I could always trust that format, as it is a defined standard, and not 'only' for Pester.
That beeing said, I am not closed to change my methodology. I could convert the ouptut to JSON, if that would be the recommended approach.
Also, it might be interesting to add a value JSON to the parameter set from '-OutputFormat' switch from 'Invoke-Pester' ? Is that planned / or already present in v5?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Yeah, ReportUnit 1.5 has a bug, and it is pretty difficult to find the .exe of 1.3. Also, the tool only works if you have an internet connection, and it works only Windows. For these reasons I was looking into this. |
|
@Stephanevg I managed to get both versions of ReportUnit by downlading the NuGet package and extracting it from there. Simply change the downloaded file name from I have been able to run both ReportUnit 1.2.1 and 1.5.0-beta1 without issues. Although, the output format (for Gherkin, anyway) is not all that friendly. I found NunitHtmlReportGenerator to be "better" in that you can actually get understandable output from it--though it's presentation is definitely not as flashy looking as ReportUnit's output. If only I could have my cake and eat it, too--I suppose if I really wanted to, I could combine the way NUnitHtmlReportGenerator parses the NUnitXml with the way ReportUnit presents that information. But alas, I didn't have time to do that, and so as a matter of choosing "the lesser of two evils", for now I went with NUnitHtmlReportGenerator myself. |
|
Ultimately, what I would really love to do (but would need buy-in from @nohwnd because we've discussed not implementing further output formats at one point) is (again, for Gherkin anyway) provide an option to output Cucumber Json format (again, for Gherkin--not sure if RSpec has an equivalent Json format) so that I could use a tool such as DonutReports (GitHub, Sample), which proivdes a really flashy looking report! |
The question is in the title actually.
This is my script
Which generates the following XML
How can I distinguish the context from the describe test-suite? Both of them are 'Test-Suites' and are both of type 'TestFixture'.
Parsing this XML strucutre is pretty difficult, because I cannot find a trustfull way to identify what I am dealing with (Or perhaps I am missing something?)
The text was updated successfully, but these errors were encountered: