Gitlab Attachments with .NET - XML missing attachment markup

[jdlegan]

I originally posted this on another issue and I will delete that comment shortly. The issue I commented on is: #40

We are a .NET shop and utilize NUnit and are a Gitlab customer. We use JUnitXml.Testlogger 3.0.134 configured per your recommendation and Gitlabs own docs and it works great... until we tried to add attachments. No matter what I seem to do, I cannot get the necessary markup into the test results xml output. I thought it may be a matter of me simply using and extensibility point to insert the markup myself using a globally accessible object or method for the JUnitXml.Testlogger lib but I quickly reviewed the source of it and the core logger nothing jumped out at me.

What I have done per other issues in this project and in the NUnit Testlogger which is how I started pulling on this thread where support for attachments was discussed and I traced support back to the core logger for is:

• Make sure the attachments are available from the root of the CI Root in the pipeline (had originally be using temp files but moved them under the same directory as the test results xml themselves.
• We are using the following command to run the tests: dotnet test ./v${VERSION}.csproj --test-adapter-path:. --logger:"junit;LogFilePath=..\artifacts\sdktest\{assembly}-${VERSION}-test-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose" --configuration Release
• We are adding attachments using TestContext.AddTestAttachment(filename, description) where filename is accessible (bullet 1).
• Using NUnit 4.0.1 (.NET 8 but same behavior was observed in .NET 7).
• Attachments work as expected in VS using MSTest (attachments included in test results, clickable, view contents).
• Artifacts are confirmed as stored in the stage, along with the test results XML.

I would think the easiest solution would be to figure out how to get the attachment detail into the resultant XML before its written, either because it is handled by the JUnitXML or core logger OR because it was manually inserted but would love to avoid doing it as a post-processing of the files after they are written but before the stage completes and Gitlab parses the results. The format that Gitlab uses for XML attachments is: https://github.com/testmoapp/junitxml?tab=readme-ov-file#attachments-in-test-output

Does anyone have any pointers on how to achieve this?

[codito]

@jdlegan the core testlogger library supports attachments but this logger doesn't (yet). I think we need to update https://github.com/spekt/junit.testlogger/blob/master/src/JUnit.Xml.TestLogger/JunitXmlSerializer.cs to add logic similar to the following. Xml elements should be updated to match Junit's attachment format.

            // Add attachments if available
            if (result.Attachments.Any())
            {
                // See spec here: https://docs.nunit.org/articles/nunit/technical-notes/usage/Test-Result-XML-Format.html#attachments
                var attachmentElement = new XElement("attachments");
                foreach (var attachment in result.Attachments)
                {
                    attachmentElement.Add(new XElement(
                                "attachment",
                                new XElement("filePath", attachment.FilePath),
                                new XElement("description", new XCData(attachment.Description))));
                }

                element.Add(attachmentElement);
            }

Reference commit in similar change we added for Nunit logger.

I don't have a gitlab CI setup to validate these changes. If you'd like to raise a PR, I am happy to review and we can cut a release with this support. /cc:@Siphonophora

[SavchenkoAlexander]

Hi, @codito. I was trying to implement adding attachements to the result xml report using attachments in test output approach, but it's structure becomes incompatible with https://github.com/spekt/junit.testlogger/blob/master/test/assets/JUnit.xsd. Also I've found the alternate xsd scheme https://github.com/junit-team/junit5/blob/main/platform-tests/src/test/resources/jenkins-junit.xsd that looks more suitable for test reports utilized by CI/CD systems. The main incompatability of this scheme with current is that the 'properties' section either should contain some child elements either be dropped. So my question is: may I replace the current xsd scheme with the scheme from JUnit repository?

[codito]

Hello @SavchenkoAlexander, thank you looking into this issue. I believe you meant the below gitlab format to output attachments:

<testcase time="1.00" name="Test">
  <system-out>[[ATTACHMENT|/path/to/some/file]]</system-out>
</testcase>

Is it possible to update the constraints in current junit.xsd that are blocking above attachment block? Also, would it be possible to add a flag to disable attachments in case the relaxed XSD will break any CI system (we could do it via an env variable, or a key similar to this.

jenkins-junit.xsd also appears less strict for several attributes on TestSuite etc.