TCP memory leak when aborting the initiating request when using a pipe

[scic]

• Version: v8.9.4
• Platform: Linux hostname 4.13.0-36-generic Working with nvm, nvmw and similar installation managers #40~16.04.1-Ubuntu SMP Fri Feb 16 23:25:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: http

I have a server that processes a GET request from the browser by issuing a GET request to another server. I use a pipe to forward the response.

http.createServer(function (req, res) {
  var url = 'http://jsonplaceholder.typicode.com/photos';
  var connector = http.get(url, received => received.pipe(res));
  req.pipe(connector);
}).listen(9001);

The problem is if the browser aborts the request then tcp memory gets leaked. The mem field of the following output keeps getting bigger until the server is restarted or the limit in /proc/sys/net/ipv4/tcp_mem is reached.

cat /proc/net/sockstat 
sockets: used 9243
TCP: inuse 7591 orphan 0 tw 7659 alloc 7671 mem 504111

After a while this might lead to an TCP: out of memory exception in the syslog (Happend on a CentOS but not on a Ubuntu):
kernel: TCP: out of memory -- consider tuning tcp_mem

If the server is stopped then the memory is released without stopping the memory is never recovered.

I tried to provide a simple example in https://gist.github.com/scic/466db4687f62fa23c0bc059c31ae179a.

I expect abort events to be automatically propagated to the piped stream or at least that the piped stream is properly destroyed.
Or must this be manually cleaned up and how?

[addaleax]

Just so I’m not misunderstanding, you’re basically describing the problem as in https://www.npmjs.com/package/pump#what-problem-does-it-solve, right?

I think in that case either using pump or waiting for #13506 (to get a built-in pump()) would be the way to go … my understanding is that we can’t change the behavior in pipe() for backwards compatibility reasons?

/cc @nodejs/streams

[mcollina]

Yes.

[scic]

Yes it seems to be the problem pump solves. Although I could not get the two pump streams connected with pump only thus i propagated the close event manually like this:

http.createServer(function (req, res) {
  var connector = http.get('http://jsonplaceholder.typicode.com/photos', received => pump(received, res));
  req.on('close', () => connector.destroy());
  return pump(req, connector);
}).listen(9001);

Now the tcp memory stays low.

I would appreciate an integrated pump functionality in Node.js like #13506 provides. I think its a very common use case.