warnaserror not failing the build with exit 1 if only errors present were warnings

[tf-jorge]

When there are no errors in a build, but only warnings that have been promoted to error with WarnAsError then the build will not fail with an exit of 1.

$ dotnet build --no-incremental /WarnAsError

...

Build FAILED.

....cs(28,1): error SA1507: Code must not contain multiple blank lines in a row [....csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:01.55
Exit code =>  0

Given that there are errors in the build, would expect an exit code of 1

Note that if there are genuine errors in the build, the build will fail correctly with the correct exit code

OS: Linux Ubuntu 18.04
dotnet version: 2.2.3-preview

[rainersigwald]

This appears to work in the trivial case:

<Project>
 <Target Name="Build">
  <Warning Text="F" />
 </Target>
</Project>

$ dotnet build /warnaserror     
Microsoft (R) Build Engine version 16.1.54-preview+gd004974104 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

S:\repro\dotnet\core-sdk\issues\1708\test.proj(3,3): error : F

Build FAILED.

S:\repro\dotnet\core-sdk\issues\1708\test.proj(3,3): error : F
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.35
$ $LASTEXITCODE
1

Can you tell us more about your repro? What kinds of projects do you have, and how many are in the solution? What target is generating the error?

[juddgaddie]

Thanks for your response. I will test your trivial case and look at the target and see if we experience the same problem. One obvious difference is we are running dotnet on Linux.

[juddgaddie]

4 projects in the solution

I just discovered that if I run the exact same build without making any other changes I get an intermittnet 1 exit code, here is my exit codes for running the same command and make no other changes. The output is always displayed correctly.
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
0
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
1
$ /dotnet build --no-incremental /WarnAsError
0

$ /dotnet build --no-incremental /WarnAsError

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
        <CodeAnalysisRuleSet>Myproject.ruleset</CodeAnalysisRuleSet>
        <AssemblyName>My.Project</AssemblyName>
        <RootNamespace>My.Project</RootNamespace>
    </PropertyGroup>

    <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
      <DocumentationFile>docgen\docfile.xml</DocumentationFile>
      <TreatWarningsAsErrors>false</TreatWarningsAsErrors>
      <NoWarn>1701;1702;1591</NoWarn>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.CodeAnalysis.FxCopAnalyzers" Version="2.9.2">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <PackageReference Include="StyleCop.Analyzers" Version="1.0.2">
          <PrivateAssets>all</PrivateAssets>
          <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
        <AdditionalFiles Include="stylecop.json" />
    </ItemGroup>

    <ItemGroup>
      <ProjectReference Include="..\My.Project\My.Project.csproj" />
    </ItemGroup>

</Project>

[BTSmolders]

@juddgaddie @rainersigwald any news on this issue?
I'm experiencing the same issues (since last month) where the exitcode is not consistent when using dotnet build /WarnAsError and alternates between 0 and 1 on different runs without code changes.

This used to work in the past in a consistent way. Is there a workaround maybe?

[BTSmolders]

I researched various dotnet sdks en think the issue was introduced in the 2.2.2* series/ since msbuild 16.0.450+ga8dc7f1d34:
2.2.101 works; msbuild: 15.9.20+g88f5fadfbe
2.2.104 works; msbuild: 15.9.20+g88f5fadfbe
2.2.202 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.203 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.300 flaky; msbuild: 16.1.76+g14b0a930a7
2.2.301 flaky; msbuild: 16.1.76+g14b0a930a7
3.0-preview untested

Hopes this helps in tracking down the issue.

[juddgaddie]

Good to hear its been reproduced independently. I unfortunately haven't made any progress from above.

[BTSmolders]

In the meantime I tested 3.0-preview7 and it has the same flaky behaviour as previous versions.

@richlander I read your blog posts on 3.0-preview and was wondering if you (or dotnet/msbuild team) know about this issue, since I cannot find it on any backlog/roadmap.
Maybe you can mention someone that can look into this issue?

[TysonMN]

I too have a solution with only warnings. I am building with -warnAsError and using .NET Core SDK 3.0.100 on Azure DevOps.

• If I build with "Enable system diagnostics" checked, then the build fails, the exit code is 1, and the pipeline step fails.
• If I build without "Enable system diagnostics" checked, then the build fails, the exit code is not given in the logs, and the pipeline step passes.

I expected the pipeline step to fail with or without "Enable system diagnostics" checked.

Is this essentially the same problem that this issue is about or should I create a new issue?

[TysonMN]

Oh, maybe this is just the flaky behavior. I will retest later.

[TysonMN]

Indeed. Retesting has confirmed that I was just experiencing the previously described flaky behavior.

[TysonMN]

I researched various dotnet sdks en think the issue was introduced in the 2.2.2* series/ since msbuild 16.0.450+ga8dc7f1d34:
2.2.101 works; msbuild: 15.9.20+g88f5fadfbe
2.2.104 works; msbuild: 15.9.20+g88f5fadfbe
2.2.202 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.203 flaky; msbuild: 16.0.450+ga8dc7f1d34
2.2.300 flaky; msbuild: 16.1.76+g14b0a930a7
2.2.301 flaky; msbuild: 16.1.76+g14b0a930a7
3.0-preview untested

Hopes this helps in tracking down the issue.

@BTSmolders identified 2.2.202 as the earliest SDK he tested that exhibited the bug, but there were more SDKs that could have been tested. I think I have tested all the relevant ones, and I would like to share what I learned.

• 2.1.602 exhibited the bug for me
• 2.1.506 worked for me
• 2.1.600-preview-009426 worked for me

The git history of tag 2.1.602 has no parallel branches at tag v2.1.500. The released SDKs between those two tags form two parallel branches: the "2.1.600-preview-009426 branch" and the "2.1.50X branch". The last 2.1.50X release before the release of 2.1.602 was 2.1.506.

In summary, I think the bug was introduced somewhere before 2.1.602 and after 2.1.600-preview-009426 and 2.1.506.

I tested all these SDKs on Azure DevOps using the Use .NET Core task.

[pleonex]

We have this issue all the time with .NET Core 3.1.200 building in Azure DevOps, which makes harder to review PR (you can't rely in the build result but actually go and check the build task). Should this issue be moved to the dotnet/sdk (before dotnet/cli) repository? Maybe there it can be tracked better and it doesn't look a problem of installation.

[rainersigwald]

@pleonex Can you share a repro of the problem?

Some of the behaviors described sound like #3046, but I'm not sure that describes everything.

[pleonex]

I was able to reproduce it here:

• Build pipeline
• Code

I am not sure if it happened by accident, but this repo started to show the behavior when the only warning started with CSC. In my company projects it's reproduced also when there are warnings without the CSC start (but it's in a on-premise installation of Azure DevOps).

[adhoc-am]

Hello,
I've the same issue with Azure DevOps and task dotnet build with -warnaserror arguments.
First build exit with code 1 and dotnet build task considered as failed, but if further re-run build without changes, inside task logs I see Build Failed, but task considered as success.
In my case with Azure DevOps seems work below solution:

• I separated dotnet restore and dotnet build tasks.

• Run dotnet build task with -warnaserror --no-restore arguments

after that dotnet build task considered as failed for first build attempt and all subsequent attempts.
But need more investigation and also not clear why it works like this.

also be sure that don't used sinarqube prepare task before build
https://docs.sonarqube.org/latest/analysis/scan/sonarscanner-for-msbuild/

The "begin" step will modify your build like this:

the active CodeAnalysisRuleSet will be updated to match the SonarQube quality profile
WarningsAsErrors will be turned off
If your build process cannot tolerate these changes we recommend creating a second build job for SonarQube analysis.

[jpactor]

We're facing this issue exactly as described. When all of our errors are derived from warnings, msbuild only intermittently exits with an error (exit code 1). So sometimes the build task passes, and sometimes it doesn't.

The desired behavior is that we fail the task when "warnings as errors" is enabled, every time, consistently. We use this for enhanced code analysis when we have pull requests targeting master. We have a significant amount of code churn and we need an automated solution. Intermittent behavior obviously makes this difficult.

This is our current pipeline YAML:
`trigger:
batch: true

branches:
include:
- master
exclude:
- release

pr:
autoCancel: true
branches:
include:
- master
exclude:
- project/*

pool:
vmImage: "windows-latest"

steps:

• task: NuGetCommand@2
  displayName: Restore NuGet Packages
  inputs:
  command: 'restore'
  restoreSolution: 'Alf.sln'
  feedsToUse: 'config'
  nugetConfigPath: 'NuGet.config'

• task: MSBuild@1
  continueOnError: false
  displayName: Build the ALF Solution
  inputs:
  solution: 'Alf.sln'
  msbuildVersion: '15.0'
  msbuildArguments: '-p:TreatWarningsAsErrors=true -warnaserror -p:LangVersion=7.3'
  clean: true
  maximumCpuCount: true
  `

We've tried setting "-p:TreatWarningsAsErrors=true" alone, "-warnaserror" alone, and a combination of the two. We never get consistent results.

Please advise.

[TysonMN]

I thought of a good workaround that I am now using in my application at work.

The behavior I want is to allow warnings on developer machines but treat warnings as errors on our build server. The configuration on a developer machine is Debug and the configuration on the build server is Release. Therefore, doing this in each project files achieves the behavior that I want.

<PropertyGroup Condition="'$(Configuration)' != 'Debug'">
  <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>

I haven't noticed this approach exhibiting the bug described in this issue.

[jpactor]

The behavior I want is to allow warnings on developer machines but treat warnings as errors on our build server.

This is exactly the behavior I want as well. It's just really unfortunate we have to use a workaround, instead of just having the feature working as expected.

Thanks for pointing out the workaround, though! 👍

[PaulVrugt]

So, is there any progress on this?

[BrWelsh]

dotnet core build does not fail azure devops build when -WarnAsError

having this issue now too. build errors created with -WarnAsError and/or TreatWarningsAsErrors=true

adding TreatWarningsAsErrors to project files makes no difference in behavior.

[BrWelsh]

Update: I have since found if building using msbuild instead of dotnet, TreatWarningsAsErrors does work as expected.

[default-kramer]

I encountered this issue also. Here is the workaround I used:

dotnet build --no-incremental -warnAsError | tee ./buildlog.txt
grep '0 Error(s)' ./buildlog.txt

The tee command duplicates the output to the buildlog file. Then grep searches for "0 Error(s)" in the buildlog, and if it is not found grep will return non-zero, failing the build.

[atniomn]

#5689

I filed a related issue against msbuild, a few months ago.

I use /graph:true as a workaround. It reliably reproduces the error.

[pchilds]

I thought of a good workaround that I am now using in my application at work.

The behavior I want is to allow warnings on developer machines but treat warnings as errors on our build server. The configuration on a developer machine is Debug and the configuration on the build server is Release. Therefore, doing this in each project files achieves the behavior that I want.

<PropertyGroup Condition="'$(Configuration)' != 'Debug'">
  <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
</PropertyGroup>

I haven't noticed this approach exhibiting the bug described in this issue.

How is this actually a workaround? Applying it only to your build server should mean that only your build server will get this behaviour; the one on which intermittent behaviour is crucial.
As to the other above workarounds, they are all platform dependent. It would be good to get some traction on this issue as it is soon to have a birthday (though if due to 2.1.602 it is actually hitting 24 months) and a build toolchain with intermittent problems is a MAJOR issue. @rainersigwald maybe you can pull in a colleague @microsoft and keep enjoying leave.

[TysonMN]

How is this actually a workaround? Applying it only to your build server should mean that only your build server will get this behaviour; the one on which intermittent behaviour is crucial.

I have been using this solution for six months now, and it is continuing to work perfectly.

I think we have a misunderstanding. You seem to not understand my workaround for my use case, and I don't know what it is about my workaround that you don't understand.

Can you elaborate about confusion? I will do my best to clarify my original comment that you quoted.

[benvillalobos]

This was likely resolved by #6174, is anyone still seeing this?

Edit: It should be available as early as VS16.10 preview 1

[mungojam]

I just got it today with .net 6 :(

[ghost]

Happening still.
I'm writing a SourceGenerator and I want to make sure it at least runs on our test project and fail the build if it fails. I added WarningsAsError for CS8785. It's now an error, but I was kind of expecting this would fail the build....and now Im here.

[simonvane]

Still happening in 6.0.401.

[marcpopMSFT]

Old issue triage: Ben thought he fixed it in msbuild but there were later report so moving over for analysis.