test: fix sequential test-net-connect-local-error #13064

sebastianplesciuc · 2017-05-16T20:41:25Z

Fixed sequential test-net-connect-local-error by swapping port and localPort in net.connect options.

Fixes: #13055

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

test

refack · 2017-05-16T22:20:48Z

test/sequential/test-net-connect-local-error.js

@@ -14,7 +14,7 @@ client.on('error', common.mustCall(function onError(err) {
  assert.strictEqual(err.code, 'ECONNREFUSED');
  assert.strictEqual(
    err.localPort,
-    common.PORT,
+    common.PORT + 1,
    `${err.localPort} !== ${common.PORT} in ${err}`


Nit: fix assertion fail message

refack · 2017-05-16T22:25:21Z

CI: https://ci.nodejs.org/job/node-test-commit/9932/
freeBSD stress: https://ci.nodejs.org/job/node-stress-single-test/1224/nodes=freebsd10-64/

Trott

LGTM with nit (about the assert message--either fix it or simply remove it) addressed and green CI.

sebastianplesciuc · 2017-05-17T04:43:45Z

@Trott @refack Fixed. I removed the message as Trott suggested. I think it looks neater this way. If it fails, the assertion is simple enough to read it even without the message.

refack · 2017-05-17T12:22:08Z

Stress whole /sequential/ on freeBSD: https://ci.nodejs.org/job/node-stress-single-test/nodes=freebsd10-64/1225/
PR CI: https://ci.nodejs.org/job/node-test-commit/9939/

refack · 2017-05-17T15:37:28Z

Still flaky (1 out of 100 runs of the whole sequential suite):

not ok 25 sequential/test-net-connect-local-error
  ---
  duration_ms: 0.364
  severity: fail
  stack: |-
    assert.js:92
      throw new AssertionError({
      ^
    
    AssertionError [ERR_ASSERTION]: 'EADDRINUSE' === 'ECONNREFUSED'
        at Socket.onError (/usr/home/iojs/build/workspace/node-stress-single-test/nodes/freebsd10-64/test/sequential/test-net-connect-local-error.js:14:10)
        at Socket.<anonymous> (/usr/home/iojs/build/workspace/node-stress-single-test/nodes/freebsd10-64/test/common/index.js:504:15)
        at emitOne (events.js:115:13)
        at Socket.emit (events.js:210:7)
        at emitErrorNT (net.js:1337:8)
        at _combinedTickCallback (internal/process/next_tick.js:102:11)
        at process._tickCallback (internal/process/next_tick.js:161:9)

sebastianplesciuc · 2017-05-17T16:26:08Z

Could it be because of this? test-dgram-bind-shared-ports runs before this one, right?

refack · 2017-05-17T16:42:24Z

Could it be because of this? test-dgram-bind-shared-ports runs before this one, right?

Good catch, that's probably it...

As I see it we have several options:

Keep this PR as is (1% chance of a falke out)
Rename the test so it runs before test-dgram-bind-shared-ports (Hacky but simple)
Use common.PORT + 2 and common.PORT + 3 with a comment about clashing with test-dgram-bind-shared-ports (not my favorite)
I'll finish my PR that salts common.PORT even in sequential mode... (will take time, and might get blocked)
Add a timeout around the test code, with comment why (also not my favorite)

sebastianplesciuc · 2017-05-17T16:46:00Z

About no. 2, wouldn't test-dgram-bind-shared-ports start to flake if we choose to do this?

refack · 2017-05-17T16:47:13Z

About no. 2, wouldn't test-dgram-bind-shared-ports start to flake if we choose to do this?

I don't think so because here we don't actually bind the port.

sebastianplesciuc · 2017-05-17T19:29:54Z

What if we use some IANA reserved ports? How much of a guarantee could we have that those ports won't be used by anything else? Could we define some port constants that indicate "hey, we should never directly bind to this port in any test ever"?

refack · 2017-05-17T19:55:35Z

What if we use some IANA reserved ports? How much of a guarantee could we have that those ports won't be used by anything else? Could we define some port constants that indicate "hey, we should never directly bind to this port in any test ever"?

That was my thought when we started with #12964, also a valid solution.
/cc @nodejs/build can you guarantee that there is at least one port (e.g. 26) that is never bound on the all CI machines?

refack · 2017-05-17T20:01:03Z

Flying a test balloon:
freeBSD strees: https://ci.nodejs.org/job/node-stress-single-test/1228/nodes=freebsd10-64/
all platforms: https://ci.nodejs.org/job/node-test-commit/9959/

refack · 2017-05-18T03:15:20Z

So stress test on const NEVER_PORT = 26; passes 150 iterations.
@Trott PTAL

/cc @nodejs/testing
The idea is to add const NEVER_PORT = 26; to test/common as a port that is valid but should never be bound. Any feedback?

santigimeno · 2017-05-18T09:40:11Z

I'm confused. If the unexpected error we're receiving is EADDRINUSE, it looks like the problem is with localPort and not port right?

refack · 2017-05-18T10:49:05Z

I'm confused. If the unexpected error we're receiving is EADDRINUSE, it looks like the problem is with localPort and not port right?

The expected error is ECONNREFUSED.
If some other test binds to common.PORT + 1 (or common.PORT when we switch ports) we get EADDRINUSE because of TIME_WAIT on the socket, and that's not what we were looking for.

That is why we need a destination port that will never be bound and will always cause ECONNREFUSED.
I hope that makes the problem more clear.

Trott · 2017-05-18T15:04:56Z

Is there any chance at all that the solution is to call socket1.close() and socket2.close() in test-dgram-bind-shared-ports?

refack · 2017-05-18T15:09:40Z

Is there any chance at all that the solution is to call socket1.close() and socket2.close() in test-dgram-bind-shared-ports?

I'll try

refack · 2017-05-19T16:39:38Z

@Trott @cjihrig PTAL at current solution (IMHO this change needs reapproval)

refack

~~technical blocking until others reapprove~~ code smell removed

Dismissing my review

refack · 2017-05-19T21:58:40Z

Cross-ref: #13033

significant change from original approved approach, not sure how I feel about it yet...more in a subsequent comment

Trott · 2017-05-20T04:57:33Z

Not sure how I feel about this. I suspect NEVER_PORT needs either a different name or else a comment explaining what it's about. Is the idea that it should be a port we can reasonably rely on never listening? Seems like a code smell a bit, but not sure I have a better solution offhand.

/cc @nodejs/testing for other opinions...

santigimeno · 2017-05-20T09:30:05Z

Maybe I'm missing something but, considering that the purpose of this test is checking that the properties of the error object are properly set, why not consider EADDRINUSE a legit error and move on?

For the record, I consider EADDRINUSE a legit error on FreeBSD as when running running this test there could be another socket in the TIME_WAIT state whose connection properties (localAddress:localPort --> remoteAddress:remotePort) were the same as the ones in this test.

refack

Accept EADDRINUSE

refack · 2017-05-20T11:59:07Z

test/sequential/test-net-connect-local-error.js

-  port: common.PORT + 1,
-  localPort: common.PORT,
+  port: common.PORT,
+  localPort: common.PORT + 1,
  localAddress: common.localhostIPv4
 });

 client.on('error', common.mustCall(function onError(err) {
  assert.strictEqual(err.syscall, 'connect');
  assert.strictEqual(err.code, 'ECONNREFUSED');


@sebastianplesciuc IMHO @santigimeno it right, we should just add an if for EADDRINUSE (or err.code in ['ECONNREFUSED', 'EADDRINUSE'])
/cc @Trott @cjihrig

refack · 2017-05-20T11:59:15Z

test/sequential/test-net-connect-local-error.js

-    `${err.localAddress} !== ${common.localhostIPv4} in ${err}`
-  );
+  assert.strictEqual(err.localPort, common.PORT + 1);
+  assert.strictEqual(err.localAddress, common.localhostIPv4);
  assert.strictEqual(
    err.message,
    `connect ECONNREFUSED ${err.address}:${err.port} ` +


If we decide to accept EADDRINUSE this'll need to be adjusted too

refack · 2017-05-20T12:00:25Z

Not sure how I feel about this. I suspect NEVER_PORT needs either a different name or else a comment explaining what it's about. Is the idea that it should be a port we can reasonably rely on never listening? Seems like a code smell a bit, but not sure I have a better solution offhand.

/cc nodejs/testing for other opinions...

Ohh, definatly a code smell 👃...
It started with a test balloon I added, but since it worked I wanted feedback. I think I'll remove it.

Maybe I'm missing something but, considering that the purpose of this test is checking that the properties of the error object are properly set, why not consider EADDRINUSE a legit error and move on?

For the record, I consider EADDRINUSE a legit error on FreeBSD as when running running this test there could be another socket in the TIME_WAIT state whose connection properties (localAddress:localPort --> remoteAddress:remotePort) were the same as the ones in this test.

That's sounds reasonable.

Also in the near future (#13100) we'll fix test-dgram-bind-shared-ports to stop using common.PORT

sebastianplesciuc · 2017-05-20T17:43:27Z

@refack fixed. Let me know if it's ok.

refack

LGTM, let's see a CI and another stress

refack · 2017-05-20T19:33:41Z

PR CI: https://ci.nodejs.org/job/node-test-commit/10046/
Stress CI (50 x /sequential/ @ freeBSD): https://ci.nodejs.org/job/node-stress-single-test/nodes=freebsd10-64/1231/ - Pass

refack · 2017-05-20T21:50:37Z

Ping @Trott @cjihrig. Reverted code smell. Instead we allow two error codes. PTAL.

Trott · 2017-05-20T22:22:11Z

test/sequential/test-net-connect-local-error.js

  assert.strictEqual(
    err.message,
-    `connect ECONNREFUSED ${err.address}:${err.port} ` +
+    `connect ${err.code} ${err.address}:${err.port} ` +


Are we 100% sure that ECONNREFUSED and EADDRINUSE return the same exact error message other than the err.code and that therefore this line is correct?

AFACT they come from the same mechanism.
I tried to stress this, but didn't want to hog the freeBSD machine so only run 50 cycles.
Last time we saw 1 in a 100 give the EADDRINUSE.
So worst case we have a flake and some good comments and bread crumbs to dig further.

Trott · 2017-05-20T22:22:42Z

test/sequential/test-net-connect-local-error.js

@@ -3,28 +3,21 @@ const common = require('../common');
 const assert = require('assert');
 const net = require('net');

+const expectedErrorCodes = ['ECONNREFUSED', 'EADDRINUSE'];


Can we add a comment here explaining why EADDRINUSE is included? Otherwise, someone will come across the test in 8 months and remove EADDRINUSE and we'll be right back where we are. :-D

I need your advice on a proper comment for this. Is this ok?

// EADDRINUSE is expected to occur on FreeBSD // Please see https://github.com/nodejs/node/issues/13055 for more details

Not really sure if you guys approve of URLs in code comments.

Yeah that's fine @sebastianplesciuc . Full URL is great.

Format suggested is fine. EADDRINUSE doesn't occur every time, just sometimes, right? EADDRINUSE is expected makes it sound like it occurs every time. Maybe EADDRINUSE can occur on FreeBSD.?

Whoops, I'm too late! :-D Anyway, the comment is fine. What I suggested is just a nit. It can be ignored.

santigimeno

LGTM with #13064 (comment) addressed

Fixed sequential test-net-connect-local-error by swapping port and localPort in net.connect options. Fixes: #13055

refack · 2017-05-22T18:05:25Z

test/sequential/test-net-connect-local-error.js

@@ -3,28 +3,23 @@ const common = require('../common');
 const assert = require('assert');
 const net = require('net');

+// EADDRINUSE is expected to occur on FreeBSD
+// Please see https://github.com/nodejs/node/issues/13055 for more details


Change to:
Ref: https://github.com/nodejs/node/issues/13055

NM. I'll do it.

Fixed sequential test-net-connect-local-error by swapping port and localPort in net.connect options. PR-URL: nodejs#13064 Fixes: nodejs#13055 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

refack · 2017-05-22T18:12:51Z

Landed in 3429c90

refack · 2017-05-22T18:15:29Z

Post land CI: https://ci.nodejs.org/job/node-test-commit/10071/

Fixed sequential test-net-connect-local-error by swapping port and localPort in net.connect options. PR-URL: #13064 Fixes: #13055 Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

nodejs-github-bot added the test Issues and PRs related to the tests. label May 16, 2017

mscdex added the net Issues and PRs related to the net subsystem. label May 16, 2017

refack approved these changes May 16, 2017

View reviewed changes

refack reviewed May 16, 2017

View reviewed changes

Trott previously approved these changes May 16, 2017

View reviewed changes

cjihrig previously approved these changes May 16, 2017

View reviewed changes

refack approved these changes May 17, 2017

View reviewed changes

refack mentioned this pull request May 18, 2017

test,dgram: harden test-dgram-bind-shared-ports.js #13100

Merged

3 tasks

refack self-assigned this May 19, 2017

refack suggested changes May 19, 2017

View reviewed changes

refack reviewed May 20, 2017

View reviewed changes

refack approved these changes May 20, 2017

View reviewed changes

Trott reviewed May 20, 2017

View reviewed changes

cjihrig approved these changes May 21, 2017

View reviewed changes

santigimeno approved these changes May 21, 2017

View reviewed changes

test: fix sequential test-net-connect-local-error

cd2f002

Fixed sequential test-net-connect-local-error by swapping port and localPort in net.connect options. Fixes: #13055

refack reviewed May 22, 2017

View reviewed changes

refack closed this May 22, 2017

refack mentioned this pull request May 25, 2017

errors, repl: migrate to use internal/errors.js #11347

Closed

4 tasks

jasnell mentioned this pull request May 28, 2017

8.0.0 Release Proposal #12220

Closed

gibfahn mentioned this pull request Jun 15, 2017

Auditing for 6.11.1 nodejs/Release#230

Closed

3 tasks

MylesBorins added the land-on-v6.x label Jul 17, 2017

MylesBorins mentioned this pull request Jul 18, 2017

v6.11.2 proposal #14356

Merged

refack removed their assignment Oct 20, 2018

test: fix sequential test-net-connect-local-error #13064

test: fix sequential test-net-connect-local-error #13064

Conversation

sebastianplesciuc commented May 16, 2017

Checklist

Affected core subsystem(s)

Choose a reason for hiding this comment

refack commented May 16, 2017

Trott left a comment

Choose a reason for hiding this comment

sebastianplesciuc commented May 17, 2017

refack commented May 17, 2017

refack commented May 17, 2017 • edited Loading

sebastianplesciuc commented May 17, 2017

refack commented May 17, 2017 • edited Loading

sebastianplesciuc commented May 17, 2017

refack commented May 17, 2017

sebastianplesciuc commented May 17, 2017

refack commented May 17, 2017 • edited Loading

refack commented May 17, 2017 • edited Loading

refack commented May 18, 2017

santigimeno commented May 18, 2017

refack commented May 18, 2017

Trott commented May 18, 2017

refack commented May 18, 2017

refack commented May 19, 2017

refack left a comment • edited Loading

Choose a reason for hiding this comment

refack commented May 19, 2017

Trott commented May 20, 2017 • edited Loading

santigimeno commented May 20, 2017

refack left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack commented May 20, 2017

sebastianplesciuc commented May 20, 2017

refack left a comment

Choose a reason for hiding this comment

refack commented May 20, 2017 • edited Loading

refack commented May 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

santigimeno left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack commented May 22, 2017

refack commented May 22, 2017

refack commented May 17, 2017 •

edited

Loading

refack commented May 17, 2017 •

edited

Loading

refack commented May 17, 2017 •

edited

Loading

refack commented May 17, 2017 •

edited

Loading

refack left a comment •

edited

Loading

Trott commented May 20, 2017 •

edited

Loading

refack commented May 20, 2017 •

edited

Loading