[BUG]: AzureCLI@2 failOnStandardError does not fail with stderr on Windows

[WangWeiLin-MV]

New issue checklist

• I searched for existing GitHub issues
• I read pipeline troubleshooting guide
• I checked how to collect logs

Task name

AzureCLI

Task version

2

Issue Description

On Windows, it seems that task AzureCLI@2 does not fail with ::WriteFile stderr output, while PowerShell@2 fails as expected. I tried std::cerr before but did not reproduce.

Repro result see https://dev.azure.com/vcpkg/public/_build/results?buildId=106047, which all four tasks run the same program, two of them redirecting stderr to null for comparison.

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

windows-latest

Relevant log output

AzureCLI@2:

"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\********.ps1'"
stdout
stderr
C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account clear"
Finishing: AzureCLI

---

PowerShell@2:

"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\********.ps1'"
stdout
##[error]stderr
Finishing: PowerShell

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION] 

Repro steps

pool:
  vmImage: "windows-latest"
jobs:
  - job: buildTest
    displayName: "Build test.exe"
    steps:
      - checkout: none
      - task: PowerShell@2
        displayName: "build"
        inputs:
          pwsh: true
          targetType: inline
          script: |
            @'
            #include <windows.h>
            #include <iostream>
            int main() {
                std::cout << "stdout" << std::endl;
                HANDLE hStderr = ::GetStdHandle(STD_ERROR_HANDLE);
                if (hStderr == INVALID_HANDLE_VALUE) {
                  return 1;
                }
                DWORD written = 0;
                const char* msg = "stderr\n";
                if (!::WriteFile(hStderr, msg, strlen(msg), &written, nullptr)) {
                    return 1;
                }
                return 0;
            }
            '@ > test.cxx
            $dir = $PWD
            & "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\Tools\Launch-VsDevShell.ps1"
            cd $dir
            cl /nologo /std:c++20 /EHsc /Fe:test.exe test.cxx
          failOnStderr: true
      - publish: "test.exe"
        artifact: test_bin
  - job: test1
    dependsOn: buildTest
    displayName: Run stderr PowerShell@2
    steps:
      - checkout: none
      - download: current
        artifact: test_bin
      - task: PowerShell@2
        inputs:
          pwsh: true
          targetType: inline
          script: |
            & $(Pipeline.Workspace)/test_bin/test.exe
          failOnStderr: true
  - job: test2
    dependsOn: buildTest
    displayName: Run stderr PowerShell@2 redirect stderr to null
    steps:
      - checkout: none
      - download: current
        artifact: test_bin
      - task: PowerShell@2
        inputs:
          pwsh: true
          targetType: inline
          script: |
            & $(Pipeline.Workspace)/test_bin/test.exe 2>$null
          failOnStderr: true
  - job: test3
    dependsOn: buildTest
    displayName: Run stderr AzureCLI@2
    steps:
      - checkout: none
      - download: current
        artifact: test_bin
      - task: AzureCLI@2
        inputs:
          azureSubscription: "********"
          scriptType: "pscore"
          scriptLocation: "inlineScript"
          inlineScript: |
            & $(Pipeline.Workspace)/test_bin/test.exe
          failOnStandardError: true
  - job: test4
    dependsOn: buildTest
    displayName: Run stderr AzureCLI@2 redirect stderr to null
    steps:
      - checkout: none
      - download: current
        artifact: test_bin
      - task: AzureCLI@2
        inputs:
          azureSubscription: "********"
          scriptType: "pscore"
          scriptLocation: "inlineScript"
          inlineScript: |
            & $(Pipeline.Workspace)/test_bin/test.exe 2>$null
          failOnStandardError: true