Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate flaky test-gc-http-client-timeout #43638

Closed
F3n67u opened this issue Jul 1, 2022 · 5 comments · Fixed by #44146
Closed

Investigate flaky test-gc-http-client-timeout #43638

F3n67u opened this issue Jul 1, 2022 · 5 comments · Fixed by #44146
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI.

Comments

@F3n67u
Copy link
Member

F3n67u commented Jul 1, 2022

Test

test-gc-http-client-timeout

Platform

freebsd

Console output

not ok 873 parallel/test-gc-http-client-timeout
  ---
  duration_ms: 0.546
  severity: fail
  exitcode: 1
  stack: |-
    node:events:515
          throw er; // Unhandled 'error' event
          ^
    
    Error: connect ECONNRESET 127.0.0.1:52005 - Local (127.0.0.1:53767)
        at internalConnect (node:net:999:16)
        at defaultTriggerAsyncIdScope (node:internal/async_hooks:464:18)
        at GetAddrInfoReqWrap.emitLookup [as callback] (node:net:1145:9)
        at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:113:8)
    Emitted 'error' event on ClientRequest instance at:
        at Socket.socketErrorListener (node:_http_client:472:9)
        at Socket.emit (node:events:537:28)
        at emitErrorNT (node:internal/streams/destroy:151:8)
        at emitErrorCloseNT (node:internal/streams/destroy:116:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      errno: -54,
      code: 'ECONNRESET',
      syscall: 'connect',
      address: '127.0.0.1',
      or...

Build links

Additional information

No response

@F3n67u F3n67u added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Jul 1, 2022
@F3n67u

This comment was marked as outdated.

@tniessen
Copy link
Member

Related: #43956

@F3n67u
Copy link
Member Author

F3n67u commented Jul 24, 2022

Another failure

Build link: https://ci.nodejs.org/job/node-test-binary-armv7l/1095/RUN_SUBSET=js,label=debian10-armv7l/testReport/junit/(root)/test/parallel_test_gc_http_client_timeout/

Console output:

done/collected/total: 93/0/804
done/collected/total: 322/93/804
done/collected/total: 415/322/804
done/collected/total: 664/349/804
done/collected/total: 737/415/804
done/collected/total: 804/664/804
done/collected/total: 804/737/804
done/collected/total: 804/804/804
node:_http_server:498
    connections[i].socket.destroy();
                          ^

TypeError: Cannot read properties of null (reading 'destroy')
    at Server.closeIdleConnections (node:_http_server:498:27)
    at Server.close (node:_http_server:481:8)
    at Immediate.status (/home/iojs/build/workspace/node-test-binary-armv7l/test/parallel/test-gc-http-client-timeout.js:66:14)
    at process.processImmediate (node:internal/timers:471:21)

Node.js v19.0.0-pre

This failure is caused by #43522

This error case is fixed by #43949.

But the Error: connect ECONNRESET is still probably will happen, so I leave this issue open temporarily.

@theanarkh
Copy link
Contributor

I think Error: connect ECONNRESET is triggered when the number of parallel connection is too many. By controlling this may fix it.

@nicksia-vgw
Copy link
Contributor

nicksia-vgw commented Aug 5, 2022

I was able to replicate this issue by increasing the number of calls to getAll() as @theanarkh suggested, on MacOS 12.4.

Setting the number of calls to ~50 meant the test failed with Error: connect ECONNRESET almost 100% of the time.

Update: It's not just this test - when running tools/test.py sequential with just the set of test-gc-http-client*.js tests, its pretty random which test script fails.

The number of requests produced is non-deterministic, and will sometimes produce enough requests to trigger the error.

nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error cause by
too many concurrent connections

Fixes: nodejs#43638
nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error cause by
too many concurrent connections

Fixes: nodejs#43638

Limit calls in all paths
nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error cause by
too many concurrent connections

Fixes: nodejs#43638

Limit calls in all paths


fix linting
nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error cause by
too many concurrent connections

Fixes: nodejs#43638

Limit calls in all paths


fix linting


cleanup
nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error cause by
too many concurrent connections

Fixes: nodejs#43638
nicksia-vgw added a commit to nicksia-vgw/node that referenced this issue Aug 6, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: nodejs#43638
nodejs-github-bot pushed a commit that referenced this issue Aug 7, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
danielleadams pushed a commit that referenced this issue Aug 16, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
ruyadorno pushed a commit that referenced this issue Aug 23, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
targos pushed a commit that referenced this issue Sep 5, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Fyko pushed a commit to Fyko/node that referenced this issue Sep 15, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: nodejs#43638
PR-URL: nodejs#44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
juanarbol pushed a commit that referenced this issue Oct 10, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
juanarbol pushed a commit that referenced this issue Oct 11, 2022
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: #43638
PR-URL: #44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
guangwong pushed a commit to noslate-project/node that referenced this issue Jan 3, 2023
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: nodejs/node#43638
PR-URL: nodejs/node#44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
guangwong pushed a commit to noslate-project/node that referenced this issue Jan 3, 2023
sequential/test-gc-http-client tests were sometimes failing due to
a non-deterministic number of requests being created, causing the
test to fail on some systems with a "ECONNRESET" error caused by
too many concurrent connections

Fixes: nodejs/node#43638
PR-URL: nodejs/node#44146
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: theanarkh <theratliter@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants