http2: fix condition where data is lost #18895

mcollina · 2018-02-21T09:34:21Z

Fix a condition that would make a Http2Stream lose some data if pipe is used with a very short stream.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

http2

mscdex · 2018-02-21T13:32:00Z

lib/internal/http2/core.js

-  stream.push(null);
-  stream[kMaybeDestroy](null, code);
+
+  // defer this until we actuallhy emit end


mscdex · 2018-02-21T13:32:29Z

lib/internal/http2/core.js

-  stream.push(null);
-  stream[kMaybeDestroy]();
+
+  // defer this until we actuallhy emit end


mcollina · 2018-02-21T15:53:57Z

@mscdex done.

CI: https://ci.nodejs.org/job/node-test-pull-request/13304/

addaleax · 2018-02-21T15:58:11Z

lib/internal/http2/core.js

+    stream.on('end', stream[kMaybeDestroy]);
+    stream.push(null);
+
+    // TODO mcollina: are we sure this is correct?


The comment style we generally use for these is // TODO(mcollina): …

addaleax · 2018-02-21T16:01:52Z

lib/internal/http2/core.js

+    // TODO mcollina: are we sure this is correct?
+    if (stream.readableLength === 0) {
+      // autoresume, the stream needs to finish
+      stream.resume()


What net.Sockets do is calling .read(0) unconditionally after .push(null). Could you try to see if that makes sense? I would really like to merge these bits of code and it would be great if we could avoid these discrepancies…

BridgeAR

I think it would be nice to combine the two tests as they are so similar, most parts could be reused.

BridgeAR · 2018-02-21T16:06:38Z

lib/internal/http2/core.js

+        this.closed) {
+      this.destroy();
+      // this should return, but eslint complains
+      // return


The last return is just considered obsolete. I am not sure if the comment really adds a benefit.

the comment states that, if any more code is added in there, a return should be included.
This makes it explicit. Can you recommend some alternative wording that are ok for you?

(I'm generally 👎 on that eslint rule)

BridgeAR · 2018-02-21T16:07:10Z

lib/internal/http2/core.js

-  stream.push(null);
-  stream[kMaybeDestroy](null, code);
+
+  // defer this until we actually emit end


Would you be so kind and upper case the first character of each comment and punctuate the comments? :-)

BridgeAR · 2018-02-21T16:09:05Z

lib/internal/http2/core.js

+    stream[kMaybeDestroy]();
+  } else {
+    stream.on('end', stream[kMaybeDestroy]);
+    stream.push(null);


Is the stream.push necessary here?

yes it is. It's needed to close the stream.

BridgeAR · 2018-02-21T16:12:11Z

lib/internal/http2/core.js

-          !this.closed)) {
-        return;
-      }
+    if (error || code !== NGHTTP2_NO_ERROR) {


I would move the code !== NGHTTP2... check to the next if. That way it is clearer that destroy is not called with error in that case. Or do I miss something here?

The code is correct as it is. There can be an error (from net or tls) or the stream can return an error code - but that's not mapped to a JS ERROR yet.

What I meant is if code !== NGHTTP2_NO_ERROR, error will be falsy. And in that case error does not have to be passed to this.destroy. This is the case in the next if statement and therefore I would move the code !== NGHTTP2... case down.

But it is not important.

BridgeAR · 2018-02-21T16:17:26Z

Could this maybe also have some influence on some of our test failures?

mcollina · 2018-02-21T16:27:43Z

@BridgeAR yes, this could be the cause of some of our spurious failures.

mcollina · 2018-02-21T16:41:50Z

@addaleax PTAL

Can you also check if that TODO that I left is correct? I'm starting to think it is.

addaleax · 2018-02-21T16:44:11Z

lib/internal/http2/core.js

+      return;
+    }
+
+    // TODO mcollina: remove usage of _*State properties


This TODO is still in the previous style ;)

mcollina · 2018-02-26T09:10:44Z

I think it would be nice to combine the two tests as they are so similar, most parts could be reused.

The two tests checks two different things, one for the compat API and one for the stream. I would keep them separate as it might be more maintainable in the long run. We have a really poor isolation logic between various sub-tests.

mcollina · 2018-02-26T09:19:20Z

CI: https://ci.nodejs.org/job/node-test-pull-request/13370/

@BridgeAR @mscdex PTAL.

mcollina · 2018-02-26T17:37:37Z

Landed in dbe645f

PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

PR-URL: nodejs#18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

PR-URL: nodejs#18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

mcollina requested review from apapirovski, sebdeckers, jasnell and addaleax February 21, 2018 09:34

nodejs-github-bot added dont-land-on-v4.x http2 Issues or PRs related to the http2 subsystem. labels Feb 21, 2018

mscdex reviewed Feb 21, 2018

View reviewed changes

lib/internal/http2/core.js Outdated

stream.push(null);

stream[kMaybeDestroy]();

// defer this until we actuallhy emit end

Copy link

Contributor

mscdex Feb 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

mcollina force-pushed the fix-http2-destroy-flow branch from 661f3ff to cd0f03c Compare February 21, 2018 15:51

addaleax reviewed Feb 21, 2018

View reviewed changes

BridgeAR reviewed Feb 21, 2018

View reviewed changes

mcollina force-pushed the fix-http2-destroy-flow branch from cd0f03c to b23cd58 Compare February 21, 2018 16:38

addaleax approved these changes Feb 21, 2018

View reviewed changes

mcollina force-pushed the fix-http2-destroy-flow branch from b23cd58 to eb76f83 Compare February 21, 2018 16:49

http2: fix condition where data is lost

c3dcc3c

mcollina force-pushed the fix-http2-destroy-flow branch from eb76f83 to c3dcc3c Compare February 26, 2018 09:18

BridgeAR approved these changes Feb 26, 2018

View reviewed changes

mcollina closed this Feb 26, 2018

mcollina deleted the fix-http2-destroy-flow branch February 26, 2018 17:37

mcollina added a commit that referenced this pull request Feb 26, 2018

http2: fix condition where data is lost

dbe645f

PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

addaleax pushed a commit to addaleax/node that referenced this pull request Mar 5, 2018

http2: fix condition where data is lost

ef8f90f

PR-URL: nodejs#18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

MylesBorins mentioned this pull request Mar 6, 2018

v9.8.0 proposal #19181

Merged

kjin pushed a commit to kjin/node that referenced this pull request May 1, 2018

http2: fix condition where data is lost

0cdcd58

PR-URL: nodejs#18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

MylesBorins pushed a commit that referenced this pull request May 2, 2018

http2: fix condition where data is lost

b1d95ec

Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

MylesBorins mentioned this pull request May 2, 2018

v8.11.2 proposal #20478

Merged

MayaLekova pushed a commit to MayaLekova/node that referenced this pull request May 8, 2018

http2: fix condition where data is lost

b2ee10c

PR-URL: nodejs#18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

MylesBorins pushed a commit that referenced this pull request May 15, 2018

http2: fix condition where data is lost

cd914ca

Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

MylesBorins pushed a commit that referenced this pull request May 15, 2018

http2: fix condition where data is lost

f366373

Backport-PR-URL: #20456 PR-URL: #18895 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>

kanongil mentioned this pull request Sep 26, 2020

Http2Stream emit connection related 'error' after receiving all data of stream. #29929

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

http2: fix condition where data is lost #18895

http2: fix condition where data is lost #18895

mcollina commented Feb 21, 2018

mscdex Feb 21, 2018

mscdex Feb 21, 2018

mcollina commented Feb 21, 2018

addaleax Feb 21, 2018

addaleax Feb 21, 2018

BridgeAR left a comment

BridgeAR Feb 21, 2018

mcollina Feb 26, 2018

BridgeAR Feb 21, 2018

BridgeAR Feb 21, 2018

mcollina Feb 26, 2018

BridgeAR Feb 21, 2018

mcollina Feb 26, 2018

BridgeAR Feb 26, 2018

BridgeAR commented Feb 21, 2018

mcollina commented Feb 21, 2018

mcollina commented Feb 21, 2018

addaleax Feb 21, 2018

mcollina Feb 21, 2018

mcollina commented Feb 26, 2018

mcollina commented Feb 26, 2018

mcollina commented Feb 26, 2018

http2: fix condition where data is lost #18895

http2: fix condition where data is lost #18895

Conversation

mcollina commented Feb 21, 2018

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcollina commented Feb 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BridgeAR commented Feb 21, 2018

mcollina commented Feb 21, 2018

mcollina commented Feb 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcollina commented Feb 26, 2018

mcollina commented Feb 26, 2018

mcollina commented Feb 26, 2018