Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using pipes other than stdin, stdout, stderr for streaming data into a child process, ENOTCONN error is thrown #13542

Open
ferencbeutel4711 opened this issue Jun 8, 2017 · 6 comments
Labels
child_process Issues and PRs related to the child_process subsystem. process Issues and PRs related to the process subsystem. stdio Issues and PRs related to stdio.

Comments

@ferencbeutel4711
Copy link

Version:

tested with 7.5 and 8.0

Platform:

OSX Sierra; Darwin Kernel Version 16.3.0 root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64

Example code:

brew install imagemagick

const request = require('request');
const spawn = require('child_process').spawn;

// const subProcess = spawn('convert', ['fd:0', '-size', '100x100', '-'], {stdio: [null, null, null, 'pipe']});
// request.get('https://tinyurl.com/zyksbz7').pipe(subProcess.stdio[0]); // this works

const subProcess = spawn('convert', ['fd:3', '-size', '100x100', '-'], {stdio: [null, null, null, 'pipe']});
request.get('https://tinyurl.com/zyksbz7').pipe(subProcess.stdio[3]); // this does not

subProcess.stdout.on('data', (data) => {
  console.log('new data: ', data);
});

Details:

I am trying to pipe a stream, which i got from request, to the command line in order to manipulate the received data. When i am sending the data to stdin of the child process, everything works fine. In my example, you can see that result by running the commented code rather than the uncommented.
If I pipe the stream to a pipe other than stdin, stdout or stderr, which should be possible according to the docs I get the following exception:

events.js:182
      throw er; // Unhandled 'error' event
      ^

Error: shutdown ENOTCONN
    at exports._errnoException (util.js:1026:11)
    at Socket.onSocketFinish (net.js:298:25)
    at emitNone (events.js:105:13)
    at Socket.emit (events.js:207:7)
    at finishMaybe (_stream_writable.js:579:14)
    at endWritable (_stream_writable.js:587:3)
    at Socket.Writable.end (_stream_writable.js:538:5)
    at Socket.end (net.js:490:31)
    at Request.onend (internal/streams/legacy.js:44:10)
    at emitOne (events.js:120:20)
    at Request.emit (events.js:210:7)
    at IncomingMessage.<anonymous> (/Users/ferencbeutel/IdeaProjects/imageprocessing/node_modules/request/request.js:1091:12)
    at Object.onceWrapper (events.js:312:19)
    at emitNone (events.js:110:20)
    at IncomingMessage.emit (events.js:207:7)
    at endReadableNT (_stream_readable.js:1045:12)
    at /Users/ferencbeutel/IdeaProjects/imageprocessing/node_modules/async-listener/glue.js:188:31
    at _combinedTickCallback (internal/process/next_tick.js:102:11)
    at process._tickCallback (internal/process/next_tick.js:161:9)

I tried multiple things, for example passing in the stream object directly when specifying the stdio object (which should be possible too according to the linked doc entry) but then I get a type error, even though I am clearly passing in a stream. I cant see the difference between the two examples I provided and am rather puzzled by the different result.

Thanks for looking into this!

@Fishrock123 Fishrock123 added child_process Issues and PRs related to the child_process subsystem. process Issues and PRs related to the process subsystem. labels Jun 8, 2017
@wsxiaoys
Copy link

wsxiaoys commented Jun 9, 2017

I met the same issue, after a bit investigation it seems a legacy child process bug on handling any FD >= 3:
nodejs/node-v0.x-archive#9419

@noahnu
Copy link

noahnu commented Jul 1, 2017

Haven't tested with imagemagick, however the following works on CentOS 7:

fd-writer.c

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[])
{
  if (argc == 1) {
    printf("Usage: %s fd1 [fd2] ...\n", argv[0]);
    return 1;
  }

  char* text = "Hello\n";

  int fd = 1, i = 1;
  while (i < argc && (fd = atoi(argv[i++]))) {
    if (!fd) continue;
    write(fd, text, strlen(text) + 1);
    close(fd);
  }

  return 0;
}
$ gcc ./fd-writer.c -o ./fd-writer

fd-writer will write "Hello" to each file descriptor passed in.

test.js

const { spawn } = require('child_process');

const child = spawn('./fd-writer', [3], {
  stdio: [null, null, null, 'pipe']
});

child.stdio[3].on('data', chunk => {
  process.stdout.write(chunk);
});
$ node test.js

So definitely not an issue with fd >= 3. Have you tried writing directly to the stream? e.g.

subProcess.stdio[3].write(data);

@ferencbeutel4711
Copy link
Author

@noahnu Hi, yes, I've tried that. Writing the data seems fine, however, on closing the input stream (which tells the spawned process that no more data is coming in) I get a similar error. The subprocess I called in this example is imagemagick which supports inputs with fds >= 3, I've tested that aswell.

@ahmadsholehin
Copy link

I'm experiencing this as well, under OS X.

@matianfu
Copy link

the same error here, node version 8.7, linux

@phal0r
Copy link

phal0r commented Jun 14, 2018

We can confirm this error under Linux (Debian):

  • the code runs reliably in windows
  • it throws the aforementioned error in linux
  • listening to the error event and ignoring the error leads to successful communication between child and master process
  • using stdin instead of stdio[3] does not throw the error (the rest of the code is the same)

We used fork to spawn a node js child with ipc as the last parameter

@jasnell jasnell added the stdio Issues and PRs related to stdio. label Jun 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
child_process Issues and PRs related to the child_process subsystem. process Issues and PRs related to the process subsystem. stdio Issues and PRs related to stdio.
Projects
None yet
Development

No branches or pull requests

8 participants