Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 #40727

daxian-dbw · 2020-08-12T17:30:23Z

Description

This was reported in PowerShell repo (PowerShell/PowerShell#13407), and it turns out to be a .NET 5 issue on Ubuntu 18.04 with WSL1.

Repro

On Ubuntu 18.04 with WSL1, run the following in PowerShell 7.1.0-preview.5 (using .NET 5 preview.6)

PS:29> date > blah
ResourceUnavailable: Program 'date' failed to run: The operation is not allowed on non-connected sockets.At line:1 char:1
+ date > blah
+ ~~~~~~~~~~~.

The exception is:

PS:33> $e.Exception.InnerException                                                                                                                                                                                                                                                                          Type           : System.IO.IOException
TargetSite     : Void .ctor(System.Net.Sockets.Socket, System.IO.FileAccess, Boolean)
StackTrace     :    at System.Net.Sockets.NetworkStream..ctor(Socket socket, FileAccess access, Boolean ownsSocket)
                    at System.Diagnostics.Process.OpenStream(Int32 fd, FileAccess access)
                    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
                    at System.Diagnostics.Process.Start()
                    at System.Management.Automation.NativeCommandProcessor.InitNativeProcess() in /PowerShell/src/System.Management.Automation/engine
                 /NativeCommandProcessor.cs:line 466
Message        : The operation is not allowed on non-connected sockets.
Data           : {}
InnerException :
HelpLink       :
Source         : System.Net.Sockets
HResult        : -2146232800

Configuration

Which version of .NET is the code running on:
- .NET 5 preview.6
What OS and version, and what distro if applicable:
- Ubuntu 18.04.4 LTS, WSL1
What is the architecture (x64, x86, ARM, ARM64):
- x64
Do you know whether it is specific to that configuration:
- not sure, it works fine on my standalone Ubuntu 16.04 machine, but didn't try with other configurations.

Regression?

Yes, it doesn't repro with PowerShell 7.1.0-preview.3, which is on top of .NET 5 preview.4.
That means the issue was a regression introduced after .NET 5 preview.4.

The text was updated successfully, but these errors were encountered:

ghost · 2020-08-12T17:30:27Z

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

jkotas · 2020-08-12T18:27:21Z

Related #34861 / #34158

cc @stephentoub @tmds

stephentoub · 2020-08-12T18:35:07Z

Presumably this has to do with the code in Socket's SafeSocketHandle ctor that tries to determine whether the supplied socket is connected or not. I'm out right now and not in a position to debug what's going on. Maybe @wfurt can. If not, we can revert #34861, which uses that new ctor.

stephentoub · 2020-08-12T18:38:26Z

(It would also be good to understand what's different about the repro vs the Process tests we have, and augment the process tests accordingly.)

daxian-dbw · 2020-08-12T19:18:07Z

The code in PowerShell is the normal operations to start a process, you can find the code here.

I think the WSL1 environment is the key factor in play for this issue. I also ran into troubles building PowerShell with .NET 5 preview.6 SDK in WSL1, and it was reporting the same exception:

/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error MSB3883: Unexpected exception:  [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : System.IO.IOException: The operation is not allowed on non-connected sockets. [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Net.Sockets.NetworkStream..ctor(Socket socket, FileAccess access, Boolean ownsSocket) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.OpenStream(Int32 fd, FileAccess access) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.Start() [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.CodeAnalysis.BuildTasks.ManagedCompiler.HandleResponse(BuildResponse response, String pathToTool, String responseFileCommands, String commandLineCommands) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]
/usr/share/dotnet/sdk/5.0.100-preview.6.20318.15/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.CodeAnalysis.BuildTasks.ManagedCompiler.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/daxian/repos/Modules/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters.csproj]

I reported this in the '.NET First Party Outreach' channel. I was told it worked in WSL2 and a workaround was to use -p:UseCompilerServer=false in WSL1. The workaround worked in a simple project but not when building PowerShell. I gave up trying at the time and haven't tried the preview.7 SDK since then.

Speaking of WSL1 and WSL2, are .NET 5 (including SDK) and near-future .NET versions going to continue supporting WSL1?

jeffhandley · 2020-08-13T00:57:29Z

We'll take a look at this before the end of the week.

eiriktsarpalis · 2020-08-13T12:11:23Z

The same issue seems to impact dotnet sdk too:

$ dotnet --version
5.0.100-preview.8.20362.3
$ dotnet new console && dotnet build
Microsoft (R) Build Engine version 16.7.0-preview-20360-03+188921e2f for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error MSB3883: Unexpected exception:  [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : System.IO.IOException: The operation is not allowed on non-connected sockets. [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Net.Sockets.NetworkStream..ctor(Socket socket, FileAccess access, Boolean ownsSocket) [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.OpenStream(Int32 fd, FileAccess access) [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at System.Diagnostics.Process.Start() [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.CodeAnalysis.BuildTasks.ManagedCompiler.HandleResponse(BuildResponse response, String pathToTool, String responseFileCommands, String commandLineCommands) [/home/eirik/devel/private/test/test.csproj]
/home/eirik/devel/dotnet/runtime/.dotnet/sdk/5.0.100-preview.8.20362.3/Roslyn/Microsoft.CSharp.Core.targets(59,5): error : at Microsoft.CodeAnalysis.BuildTasks.ManagedCompiler.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/home/eirik/devel/private/test/test.csproj]

Build FAILED.

eiriktsarpalis · 2020-08-13T12:58:03Z

This reproduces the error:

using System.Diagnostics;

namespace consoleapp
{
    class Program
    {
        static void Main()
        {
            var psi = new ProcessStartInfo("ls");
            psi.RedirectStandardOutput = true;
            Process.Start(psi);
        }
    }
}

eiriktsarpalis · 2020-08-13T17:50:56Z

So, after running this on the debugger, it looks like the root cause is in the SystemNative_GetSocketType function which does not set an address family in this case. Ultimately that results in this check not being satisfied, so none of the _isConnected detection logic kicks in. I modified the code so that _isConnected is forced to true, and sure enough the problem went away (and System.Diagnotics.Process tests went back to green).

I'm not sure what the best approach might be to fix the problem. Perhaps @dotnet/ncl could weigh in on this.

Note that dotnet/runtime on WSL1 is mostly broken because of this bug. If trying to debug, you might want to check out the original commit c44dc40 that introduced the regression, since it relies on a bootstrapping sdk that doesn't suffer from the bug.

wfurt · 2020-08-13T18:01:14Z

Does it fail with specific error or does it just stay unknown? I'm wondering if we can somehow detect this specific condition and do some damage control.

eiriktsarpalis · 2020-08-13T18:07:35Z

Does it fail with specific error or does it just stay unknown?

It succeeds without populating the address family. Same for protocol type.

eiriktsarpalis · 2020-08-16T14:58:36Z

Addressed by #40851

Dotnet-GitSync-Bot added area-System.Diagnostics.Process untriaged New issue has not been triaged by the area owner labels Aug 12, 2020

daxian-dbw mentioned this issue Aug 12, 2020

Regression: native app doesn't run in parens on Linux PowerShell/PowerShell#13407

Closed

daxian-dbw changed the title ~~Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6~~ Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 Aug 12, 2020

jkotas added the bug label Aug 12, 2020

jkotas added this to the 5.0.0 milestone Aug 12, 2020

am11 mentioned this issue Aug 12, 2020

Running dotnet in Linux chroot with FreeBSD host fails #33051

Closed

eiriktsarpalis removed the untriaged New issue has not been triaged by the area owner label Aug 14, 2020

eiriktsarpalis self-assigned this Aug 14, 2020

This was referenced Aug 14, 2020

Revert "Use socketpair to implement Process.Start redirection (#34861)" #40832

Closed

Workaround for WSL1 process IO redirection bug #40851

Merged

eiriktsarpalis closed this as completed Aug 16, 2020

eiriktsarpalis mentioned this issue Aug 16, 2020

[WSL1] Socket created on top of socketpair fd reported as not connected #40901

Closed

ericstj mentioned this issue Sep 8, 2020

how to build release version of coreclr for android? #33237

Open

adamsitnik mentioned this issue Sep 9, 2020

WSL and WSL2 test runs #42011

Open

ghost locked as resolved and limited conversation to collaborators Dec 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 #40727

Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 #40727

daxian-dbw commented Aug 12, 2020 •

edited

Loading

ghost commented Aug 12, 2020

jkotas commented Aug 12, 2020

stephentoub commented Aug 12, 2020 •

edited

Loading

stephentoub commented Aug 12, 2020

daxian-dbw commented Aug 12, 2020 •

edited

Loading

jeffhandley commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020 •

edited

Loading

eiriktsarpalis commented Aug 13, 2020

wfurt commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020

eiriktsarpalis commented Aug 16, 2020

Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 #40727

Redirecting stdout/stderr of a process throws exception with .NET 5 preview.6 on WSL1 #40727

Comments

daxian-dbw commented Aug 12, 2020 • edited Loading

Description

Repro

Configuration

Regression?

ghost commented Aug 12, 2020

jkotas commented Aug 12, 2020

stephentoub commented Aug 12, 2020 • edited Loading

stephentoub commented Aug 12, 2020

daxian-dbw commented Aug 12, 2020 • edited Loading

jeffhandley commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020 • edited Loading

eiriktsarpalis commented Aug 13, 2020

wfurt commented Aug 13, 2020

eiriktsarpalis commented Aug 13, 2020

eiriktsarpalis commented Aug 16, 2020

daxian-dbw commented Aug 12, 2020 •

edited

Loading

stephentoub commented Aug 12, 2020 •

edited

Loading

daxian-dbw commented Aug 12, 2020 •

edited

Loading

eiriktsarpalis commented Aug 13, 2020 •

edited

Loading