test_runner: support custom node options in `run()` #55126

RedYetiDev · 2024-09-25T23:02:48Z

This PR adds support for specifying custom execution arguments and custom file arguments when using the run() API. This way, users have a greater control of how their tests are executing.

nodejs-github-bot · 2024-09-25T23:02:52Z

Review requested:

@nodejs/test_runner

codecov · 2024-09-26T01:07:47Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.26%. Comparing base (812806a) to head (e5c71b4).
Report is 18 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #55126      +/-   ##
==========================================
+ Coverage   88.24%   88.26%   +0.02%     
==========================================
  Files         651      651              
  Lines      183905   183916      +11     
  Branches    35864    35859       -5     
==========================================
+ Hits       162286   162334      +48     
+ Misses      14905    14885      -20     
+ Partials     6714     6697      -17

Files with missing lines	Coverage Δ
lib/internal/test_runner/runner.js	`87.14% <100.00%> (+0.29%)`	⬆️

... and 38 files with indirect coverage changes

atlowChemi · 2024-09-26T05:27:39Z

@RedYetiDev why is this related to the run function though?
I am -1 for adding handling for flags non test-runner specific into run

RedYetiDev · 2024-09-26T10:31:43Z

IMO, with the addition of experimental typescript support, users may begin writing their tests in typescript. This way, the project can ensure compatibility with these new tests via the run() API.

Currently, this feature is supported via --test, but not the run() API.

This wouldn't be the first time a non-test runner specific flag was added. The run() API supports inspection and watch mode.

mcollina

Instead of a typescript flag, run should have a nodeOptions option that sets various flags to the executed test files

RedYetiDev · 2024-09-26T13:24:46Z

SGTM, I'll add an option to specify arguments.

mcollina

lgtm

nodejs-github-bot · 2024-09-28T09:43:03Z

CI: https://ci.nodejs.org/job/node-test-pull-request/62816/

nodejs-github-bot · 2024-09-28T13:52:06Z

Landed in bf11e57

cjihrig · 2024-09-28T15:35:31Z

The execArgv option specifically should probably be documented with a big warning. A number of Node's CLI flags just won't work properly when used here, and users shouldn't be using this option unless they really know what they are doing. I also don't think we should accept bug reports about specific flags not working here because we are going to end up with a maintenance headache.

RedYetiDev · 2024-09-28T18:46:28Z

It’s a little late for this PR, but ditto, good idea.

PR-URL: #55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159 doc: * add abmusse to collaborators (Abdirahim Musse) #55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971 src: * mark node `--run` as stable (Yagiz Nizipli) #53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234 PR-URL: TODO

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159 doc: * add abmusse to collaborators (Abdirahim Musse) #55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971 src: * mark node `--run` as stable (Yagiz Nizipli) #53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234 PR-URL: #55343

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159 doc: * add abmusse to collaborators (Abdirahim Musse) #55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971 src: * mark `node --run` as stable (Yagiz Nizipli) #53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234 PR-URL: #55343

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159 doc: * add abmusse to collaborators (Abdirahim Musse) #55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971 process: * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241 src: * mark `node --run` as stable (Yagiz Nizipli) #53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234 PR-URL: #55343

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159 doc: * add abmusse to collaborators (Abdirahim Musse) #55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971 process: * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241 * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) #54295 src: * mark `node --run` as stable (Yagiz Nizipli) #53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234 PR-URL: #55343

PR-URL: nodejs#55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) nodejs#55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) nodejs#54159 doc: * add abmusse to collaborators (Abdirahim Musse) nodejs#55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) nodejs#54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) nodejs#54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) nodejs#54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) nodejs#54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) nodejs#54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) nodejs#54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) nodejs#54971 process: * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) nodejs#55241 * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) nodejs#54295 src: * mark `node --run` as stable (Yagiz Nizipli) nodejs#53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) nodejs#55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) nodejs#54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) nodejs#53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) nodejs#55234 PR-URL: nodejs#55343

PR-URL: nodejs#55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Notable changes: crypto: * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) nodejs#55262 * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) nodejs#54159 doc: * add abmusse to collaborators (Abdirahim Musse) nodejs#55086 http2: * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) nodejs#54875 lib: * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) nodejs#54826 module: * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) nodejs#54933 * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) nodejs#54648 * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) nodejs#54971 * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) nodejs#54971 * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) nodejs#54971 process: * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) nodejs#55241 * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) nodejs#54295 src: * mark `node --run` as stable (Yagiz Nizipli) nodejs#53763 test_runner: * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) nodejs#55126 * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) nodejs#54851 * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) nodejs#53937 worker: * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) nodejs#55234 PR-URL: nodejs#55343

nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Sep 25, 2024

RedYetiDev force-pushed the typescript-runner-run branch 2 times, most recently from 2ec5bc7 to 170cd84 Compare September 25, 2024 23:06

mcollina requested changes Sep 26, 2024

View reviewed changes

RedYetiDev marked this pull request as draft September 26, 2024 13:24

RedYetiDev changed the title ~~test_runner: support typescript in run~~ test_runner: support custom node options in run() Sep 26, 2024

RedYetiDev added wip Issues and PRs that are still a work in progress. and removed experimental Issues and PRs related to experimental features. strip-types Issues or PRs related to strip-types support labels Sep 26, 2024

test_runner: support custom arguments in run()

b77315a

RedYetiDev force-pushed the typescript-runner-run branch from 648de09 to b77315a Compare September 26, 2024 21:38

RedYetiDev requested a review from mcollina September 26, 2024 21:39

RedYetiDev removed the wip Issues and PRs that are still a work in progress. label Sep 26, 2024

RedYetiDev marked this pull request as ready for review September 26, 2024 21:39

mcollina approved these changes Sep 27, 2024

View reviewed changes

mcollina requested review from cjihrig and atlowChemi September 27, 2024 08:02

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 28, 2024

RedYetiDev added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Sep 28, 2024

aduh95 approved these changes Sep 28, 2024

View reviewed changes

aduh95 added the commit-queue Add this label to land a pull request using GitHub Actions. label Sep 28, 2024

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Sep 28, 2024

nodejs-github-bot merged commit bf11e57 into nodejs:main Sep 28, 2024
61 checks passed

RedYetiDev mentioned this pull request Sep 30, 2024

doc: add notice to execArgv #55177

Open

targos pushed a commit that referenced this pull request Oct 4, 2024

test_runner: support custom arguments in run()

edb7407

PR-URL: #55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

targos pushed a commit that referenced this pull request Oct 4, 2024

test_runner: support custom arguments in run()

86f7cb8

PR-URL: #55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

aduh95 mentioned this pull request Oct 9, 2024

V22.10.0 proposal #55343

Merged

RafaelGSS mentioned this pull request Oct 11, 2024

2024-10-16, Version 23.0.0 (Current) #55338

Merged

louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024

test_runner: support custom arguments in run()

dcab209

PR-URL: nodejs#55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024

test_runner: support custom arguments in run()

208e6cd

PR-URL: nodejs#55126 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_runner: support custom node options in `run()` #55126

test_runner: support custom node options in `run()` #55126

RedYetiDev commented Sep 25, 2024 •

edited

Loading

nodejs-github-bot commented Sep 25, 2024

codecov bot commented Sep 26, 2024 •

edited

Loading

atlowChemi commented Sep 26, 2024

RedYetiDev commented Sep 26, 2024 •

edited

Loading

mcollina left a comment

RedYetiDev commented Sep 26, 2024

mcollina left a comment

nodejs-github-bot commented Sep 28, 2024

nodejs-github-bot commented Sep 28, 2024

cjihrig commented Sep 28, 2024

RedYetiDev commented Sep 28, 2024

test_runner: support custom node options in run() #55126

test_runner: support custom node options in run() #55126

Conversation

RedYetiDev commented Sep 25, 2024 • edited Loading

nodejs-github-bot commented Sep 25, 2024

codecov bot commented Sep 26, 2024 • edited Loading

Codecov Report

atlowChemi commented Sep 26, 2024

RedYetiDev commented Sep 26, 2024 • edited Loading

mcollina left a comment

Choose a reason for hiding this comment

RedYetiDev commented Sep 26, 2024

mcollina left a comment

Choose a reason for hiding this comment

nodejs-github-bot commented Sep 28, 2024

nodejs-github-bot commented Sep 28, 2024

cjihrig commented Sep 28, 2024

RedYetiDev commented Sep 28, 2024

test_runner: support custom node options in `run()` #55126

test_runner: support custom node options in `run()` #55126

RedYetiDev commented Sep 25, 2024 •

edited

Loading

codecov bot commented Sep 26, 2024 •

edited

Loading

RedYetiDev commented Sep 26, 2024 •

edited

Loading