Changing to tasks.json v2.0.0 breaks Pester problem matching

[rkeithhill]

System Details

• Operating system name and version:
• VS Code version:
• PowerShell extension version:
• Output from $PSVersionTable:

Windows 10 CU
1.12.2
1.1.0
5.1.15063.296

Issue Description

When the version of tasks.json is set to 2.0.0 and there is a single Pester error, the error shows up multiple times - on the order of 6 to 12 times for the same error. Switch the version back to 0.1.0 and the problem matcher (correctly) finds just one error.

[pcgeek86]

Related to #802

[daviwil]

Maybe related, but I think this is more of an issue with the new terminal-based task runner not reading the terminal output correctly. In theory it shouldn't require any changes on our side.

[mattmcnabb]

chiming in because my experience is a bit different. Using task version 2.0.0, I'm actually seeing tests invoked multiple times. After that, even with a failing test, there is nothing in the Problems console.

Switched to task version 0.1.0 and the tests are executed once and I do see matched problems.

[daviwil]

@mattmcnabb weird, can you show us your tasks.json file?

[mattmcnabb]

The Task

{
    "version": "2.0.0",
    "isShellCommand": true,
    "command": "${env.windir}\\system32\\windowspowershell\\v1.0\\PowerShell.exe",
    "args": [
        "-NoProfile",
        "-ExecutionPolicy",
        "Bypass"
    ],
    "suppressTaskName": true,
    "tasks": [
        {
            "taskName": "Test",
            "args": [
                "-Command",
                "Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true}"
            ],
            "isTestCommand": true,
            "problemMatcher": "$pester"
        }
    ]
}

Task Version 2.0.0

Notice that the test loop several times and the end result is 71 passing and 8 failing. This is a template project with only 12 tests.

Task Version 0.1.0

Here you can see in the output pane that the tests only execute once and the end result is 10 passing and 2 failing as expected, and the failed tests are reflected in the problem pane.

Originally I thought my task was just broken, so I trimmed a bunch of it out to get down to this, and it still doesn't work. I also tried flipping from isTestCommand to isBuildCommand but got the same result.

Notes

PSES Version: 1.2.0.0
Code Version: 1.12.2
PSVersion: 5.1.15063.296

[rkeithhill]

@mattmcnabb remove the line "isShellCommand": true and try again.

[mattmcnabb]

@rkeithhill That seems to have fixed it for the v2.0.0 task. I no longer see the tests looping, and the problems now show up in the problem pane. I do see the duplicate problem entries as described in the issue.

Thanks!

[rkeithhill]

This problem has been fixed with PR #941. You can implement the fix without waiting on a new drop as the fix applies to the tasks.json file. Simply change this line:

"Write-Host 'Invoking Pester...'; Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true};",

to

"Write-Host 'Invoking Pester...'; $ProgressPreference = 'SilentlyContinue'; Invoke-Pester -PesterOption @{IncludeVSCodeMarker=$true};",

The bug has to do with the PowerShell progress overlay causing the terminal to see a single failure multiple times. The above "fix" is really more of a "workaround".

There is another bug lurking here with the 2.0.0 tasks version. If the terminal window is not wide enough to display the entire line that contains the file path & line number, the problem matcher will not find the failed test.

[fourpastmidnight]

Can this be closed? For one thing, I know that Pester has disabled progress reporting during the test run since it broke running tests on Linux/macOS due to PSv6 not supporting it very well. I also know that detecting problems using a problem matcher with Pester does work, as I've been using Pester for some time now with VS Code. I am, however, not using the $pester problem matcher that comes with this extension, but rather I am using an explicit problem matcher since I have been contributing to the Pester project itself. So, I'm just wondering if this issue has been fixed and can be closed, or whether the $pester problem matcher still needs to be updated?