WebScenarioTests smoke tests fail on mono with unexpected exit code.

[tmds]

From the log:

[xUnit.net 00:00:55.38]         Executing: kill -s TERM 108304
    Failed Microsoft.DotNet.SourceBuild.SmokeTests.WebScenarioTests.VerifyScenario(scenario: TestScenario { Commands = Build | Run | Publish, Language = CSharp, NoHttps = False, ScenarioName = "WebScenarioTests", Template = BlazorWasm }) [54 s]
EXEC : error Message:  [/home/tester/dotnet/build.proj]
     System.InvalidOperationException : Failed to execute /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/.dotnet/dotnet run /bl:/home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/logs/WebScenarioTests_BlazorWasm_CSharp-run.binlog
  Exit code: 143
  Building...
  info: Microsoft.Hosting.Lifetime[14]
        Now listening on: http://localhost:5199/
  info: Microsoft.Hosting.Lifetime[0]
        Application started. Press Ctrl+C to shut down.
  info: Microsoft.Hosting.Lifetime[0]
        Hosting environment: Development
  info: Microsoft.Hosting.Lifetime[0]
        Content root path: /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/projects-202212131302478513/WebScenarioTests_BlazorWasm_CSharp
  
    Stack Trace:
       at Microsoft.DotNet.SourceBuild.SmokeTests.ExecuteHelper.ValidateExitCode(ValueTuple`3 result, Int32 expectedExitCode) in /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/ExecuteHelper.cs:line 124
     at Microsoft.DotNet.SourceBuild.SmokeTests.DotNetHelper.ExecuteCmd(String args, String workingDirectory, Action`1 additionalProcessConfigCallback, Nullable`1 expectedExitCode, Int32 millisecondTimeout) in /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/DotNetHelper.cs:line 109
     at Microsoft.DotNet.SourceBuild.SmokeTests.DotNetHelper.ExecuteRunWeb(String projectName) in /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/DotNetHelper.cs:line 201
     at Microsoft.DotNet.SourceBuild.SmokeTests.TestScenario.Execute(DotNetHelper dotNetHelper) in /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/TestScenario.cs:line 38
     at Microsoft.DotNet.SourceBuild.SmokeTests.WebScenarioTests.VerifyScenario(TestScenario scenario) in /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/WebScenarioTests.cs:line 23
     at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
     at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
    Standard Output Messages:
   Executing: /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/.dotnet/dotnet new blazorwasm --name WebScenarioTests_BlazorWasm_CSharp --output /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/projects-202212131302478513/WebScenarioTests_BlazorWasm_CSharp --language "C#" 
   Executing: /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/.dotnet/dotnet build /bl:/home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/logs/WebScenarioTests_BlazorWasm_CSharp-build.binlog
   Executing: /home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/.dotnet/dotnet run /bl:/home/tester/dotnet/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net7.0/logs/WebScenarioTests_BlazorWasm_CSharp-run.binlog
   Executing: kill -s TERM 108304

This test expects 0 as an exit value of dotnet run after sending SIGTERM to it.
However the exit code is 143, which is the expected exit code for a process that gets killed with SIGTERM but doesn't handle it.

It can be reproduced easily with a mono sdk by sending SIGTERM to a dotnet run process that is running the web template.

The exit code for this process is 143 with a mono runtime, and 0 with a coreclr runtime.

Has someone run this test on mono before? Did it ever pass?

cc @omajid @crummel @MichaelSimons

[MichaelSimons]

Has someone run this test on mono before? Did it ever pass?

I would guess not since we don't have mono CI.

[tmds]

@omajid @uweigand I assume you haven't ran the smoke tests with mono based builds either?

I'll report an issue for it in dotnet/runtime.
Until the difference gets fixed, I'd like to update the test so it also passes with mono.

[uweigand]

@omajid @uweigand I assume you haven't ran the smoke tests with mono based builds either?

I ran them on .NET6, but not yet on .NET7. I haven't seen this issue back then.

[MichaelSimons]

[Triage] #2596 tracks adding a mono configuration to CI.

[MichaelSimons]

@tmds - Is this issue still applicable? You enabled CI on Mono with #2596 but I still see dotnet/runtime#81093 is still active. What am I missing?

[tmds]

We've updated the test suite to account for the Mono behavior.

When the fix is merged in the runtime repo, the suite will fail again, and we have to undo those changes.