Unexpected warning in Test Results related to "Unknown XML tag: TestCase"

[0xg0nz0]

Describe the bug

When running a test -- and it appears to be the first test only -- there is an unexpected warning in the test window:

Unknown XML tag: TestCase with {"name":"ping Iggy server","tags":"[E2E Tests]","filename":"/workspaces/iggy-cpp-client/tests/ping_cmd_test.cc","line":"5"}

To Reproduce

Sequence is very simple: I am just running two tests, and see the warning shown in screenshot during first test run. The test succeeds. In case it helps, the test is trivial:

#include <catch.hpp>
#include "e2e_testutils.h"
#include "../sdk/client.h"

TEST_CASE("ping Iggy server", "[E2E Tests]") {
    // Start the Docker container; shuts down when this object goes out of scope
    IggyRunner runner;

    // Create a client object with all defaults
    iggy::client::Options options;
    iggy::client::Client client(options);

    // Perform the ping operation
    client.ping();
}

and this is the only TestMate configuration in settings.json:

    "testMate.cpp.test.advancedExecutables": [
        {
            "pattern": "${workspaceFolder}/build/tests/**/*_test",
            "catch2": {
                "testGrouping": {
                    "groupByTags": {
                        "tags": []
                    }
                }
            }
        }
    ],

Note even without that config, the warning still appears.

Screenshots

Details

For additional detail, below is the output of running test command with "-r XML" to show output format.

<?xml version="1.0" encoding="UTF-8"?>
<Catch2TestRun name="iggy_e2e_test" rng-seed="1916246528" xml-format-version="3" catch2-version="3.5.2" filters="[.],&quot;*&quot;">
  <TestCase name="ping Iggy server" tags="[E2E Tests]" filename="/workspaces/iggy-cpp-client/tests/ping_cmd_test.cc" line="5">
83b7e379aa733f8cb7b600b632b7cc8d430a1b8d972d78af99432fa66eef31fe
Executing ping command
Ping sequence id: 1 time: 0.16 ms

Ping statistics for 1 ping commands
min/avg/max/mdev = 0.160/0.160/0.160/0.000 ms
    <OverallResult success="true" skips="0"/>
  </TestCase>
  <OverallResults successes="0" failures="0" expectedFailures="0" skips="0"/>
  <OverallResultsCases successes="1" failures="0" expectedFailures="0" skips="0"/>
</Catch2TestRun>

Desktop

• Extension Version: 4.9.0
• VS Code Version: 1.87.0
• Catch2 / Google Test / DOCTest Version: Catch2 3.5.2
• OS Type and Version: Ubuntu 22.4
• Using remote-ssh/docker/wsl?: Docker dev container

[0xg0nz0]

In case it helps to compare, the second test group which produces no warning is:

TEST_CASE("simple test for model objects", "[Unit Tests]") {
    iggy::model::system::Stats stats;
    REQUIRE(&stats != nullptr);
}

and if I modify the pattern to not run the E2E test shown in the bug report, there's no warning. So it appears to be specific to that particular test case; the XML output may help, but it looks very similar:

<?xml version="1.0" encoding="UTF-8"?>
<Catch2TestRun name="iggy_cpp_test" rng-seed="784361046" xml-format-version="3" catch2-version="3.5.2" filters="[.],&quot;*&quot;">
  <TestCase name="simple test for model objects" tags="[Unit Tests]" filename="/workspaces/iggy-cpp-client/tests/model_test.cc" line="5">
    <OverallResult success="true" skips="0"/>
  </TestCase>
  <OverallResults successes="1" failures="0" expectedFailures="0" skips="0"/>
  <OverallResultsCases successes="1" failures="0" expectedFailures="0" skips="0"/>
</Catch2TestRun>

[matepek]

Can you show me your test window?

[matepek]

Fixed in v4.11.0.

This issue was mentioned in CHANGELOG.md under a released entry so it is assumed to be fixed.
User verifications are always welcome.

[thendrickson]

@matepek I am seeing this issue in v4.12.0 when my test prints to stdout.