-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Investigate flaky test-benchmark-misc #31372
Comments
Not flaky, flat-out broken, fails on master, in ubuntu:
|
(By the way, if this turns out to be a genuine bug in Node.js core, I never want to hear anyone ever say again "Why do we have tests for the benchmarks?") |
@Trott thanks I'm doing to do a manual bisect, but my underpowered machine is making slow progress. |
You're almost certainly ahead of me. I'm doing work while my machine compiles the master branch.... |
It's 539df73. The issue is the benchmark test. It has the type set to an empty string and there is no default value in the corresponding benchmark file. It passes The solution is to use a default value in the benchmark or to pass through a legit value. |
The way our benchmark tests work is less than ideal. I won't find time for it right away but I'll look into rewriting the implementation so that we do not have to maintain our tests anymore. |
Yeah, instead of having default values, it may be best if the test is smart enough to determine the possible options and pop the first value off the default list, or something like that. |
Quick fix: #31377 |
Exactly that 👍 |
This fixes a benchmark test that was recently broken by a breaking change on the master branch. Fixes: #31372 PR-URL: #31377 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This fixes a benchmark test that was recently broken by a breaking change on the master branch. Fixes: #31372 PR-URL: #31377 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This fixes a benchmark test that was recently broken by a breaking change on the master branch. Fixes: #31372 PR-URL: #31377 Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com>
This has been failing node-daily-master since 10-01-2020.
https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1802/
https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1803/
https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1804/
https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1805/
https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1806/
The text was updated successfully, but these errors were encountered: