Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: implement flushCompileCache() #54971

Closed
wants to merge 3 commits into from

Conversation

joyeecheung
Copy link
Member

@joyeecheung joyeecheung commented Sep 16, 2024

module: write compile cache to temporary file and then rename it

This works better in terms of avoiding race conditions.

Drive-by: throw instead of abort when module.enableCompileCache()
gets passed a non-string argument.

module: implement flushCompileCache()

This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.

Fixes: #54770
Fixes: #54465

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Sep 16, 2024
src/compile_cache.cc Outdated Show resolved Hide resolved
src/compile_cache.cc Show resolved Hide resolved
src/env.cc Outdated Show resolved Hide resolved
@richardlau richardlau added the semver-minor PRs that contain new features and should be released in the next minor version. label Sep 16, 2024
doc/api/module.md Outdated Show resolved Hide resolved
Copy link

codecov bot commented Sep 16, 2024

Codecov Report

Attention: Patch coverage is 73.23944% with 19 lines in your changes missing coverage. Please review.

Project coverage is 88.24%. Comparing base (e35299a) to head (5d5b549).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
src/compile_cache.cc 65.00% 10 Missing and 4 partials ⚠️
src/node_modules.cc 81.25% 2 Missing and 1 partial ⚠️
src/env.cc 77.77% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #54971      +/-   ##
==========================================
+ Coverage   88.03%   88.24%   +0.21%     
==========================================
  Files         652      652              
  Lines      183797   183861      +64     
  Branches    35863    35855       -8     
==========================================
+ Hits       161804   162249     +445     
+ Misses      15242    14891     -351     
+ Partials     6751     6721      -30     
Files with missing lines Coverage Δ
lib/internal/modules/helpers.js 98.83% <100.00%> (+0.78%) ⬆️
lib/module.js 100.00% <100.00%> (ø)
src/compile_cache.h 100.00% <100.00%> (ø)
src/env.h 98.00% <ø> (ø)
src/env.cc 85.52% <77.77%> (-0.15%) ⬇️
src/node_modules.cc 78.10% <81.25%> (+0.44%) ⬆️
src/compile_cache.cc 77.54% <65.00%> (-2.46%) ⬇️

... and 66 files with indirect coverage changes

@RedYetiDev RedYetiDev added the module Issues and PRs related to the module subsystem. label Sep 16, 2024
@joyeecheung
Copy link
Member Author

joyeecheung commented Sep 17, 2024

Updated to remove the keepDeserializedCache option - we can add it back if someone actually has a use case for it. For now it doesn't seem to be actually useful to me - either you purge the module cache frequently then you probably already expect the code to change frequently and should opt out of the compile cache in the first place, or you don't purge the module cache then there's no point keeping the other tier of serialized code cache around. Might as well just remove it to keep things simple.

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 17, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 17, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung
Copy link
Member Author

@anonrig @nodejs/loaders @nodejs/cpp-reviewers can I get some reviews please?

@joyeecheung joyeecheung added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 19, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 19, 2024
@nodejs-github-bot
Copy link
Collaborator

@joyeecheung
Copy link
Member Author

It seems the clang-format errors were caused by what was fixed by #54994 - not sure why it is still faililng, it could be a GitHub action hiccup?

This works better in terms of avoiding race conditions.
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.
@joyeecheung
Copy link
Member Author

Trying to rebase again to see if it will make #54994 effective.

nodejs-github-bot pushed a commit that referenced this pull request Sep 20, 2024
This works better in terms of avoiding race conditions.

PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
nodejs-github-bot pushed a commit that referenced this pull request Sep 20, 2024
PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
nodejs-github-bot pushed a commit that referenced this pull request Sep 20, 2024
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.

PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
// of essential builtins that are loaded in the snapshot can have faster first
// invocation.
if (isolate_data->is_building_snapshot()) {
builtin_loader()->SetEagerCompile();
Copy link
Member Author

@joyeecheung joyeecheung Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

er, I meant to open a separate PR about this (this was why I opened #54633 which removed a coverage regression that made me not include this and left the TODO above in #51672), somehow this ended up in the checkout of this PR due to local git mess up 😅 but it was approved and landed anyway..

Copy link
Member Author

@joyeecheung joyeecheung Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity: the short summary of this small diff that accidentally get landed in this unrelated PR is that it speeds up core startup by not having to compile any inner functions in essential builtins. Previously, only the top-level functions were cached in the snapshot, so during bootstrap, when the inner functions were executed to do the loading, they still needed to be compiled without cache. This diff changed to cache the inner functions too, which adds ~630KB to the binary, in exchange of a slightly faster core startup (and also speeding up the first invocation of most functions provided by the essential builtins).

❯ hyperfine "./node_pr ./test/fixtures/semicolon.js" "./node_main ./test/fixtures/semicolon.js"
Benchmark 1: ./node_pr ./test/fixtures/semicolon.js
  Time (mean ± σ):      34.2 ms ±   1.5 ms    [User: 30.1 ms, System: 2.5 ms]
  Range (min … max):    33.1 ms …  43.7 ms    80 runs

Benchmark 2: ./node_main ./test/fixtures/semicolon.js
  Time (mean ± σ):      35.1 ms ±   1.2 ms    [User: 31.0 ms, System: 2.5 ms]
  Range (min … max):    34.1 ms …  42.6 ms    77 runs

Summary
  './node_pr ./test/fixtures/semicolon.js' ran
    1.03 ± 0.06 times faster than './node_main ./test/fixtures/semicolon.js'

targos pushed a commit that referenced this pull request Oct 4, 2024
This works better in terms of avoiding race conditions.

PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Oct 4, 2024
PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
targos pushed a commit that referenced this pull request Oct 4, 2024
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.

PR-URL: #54971
Fixes: #54770
Fixes: #54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
aduh95 added a commit that referenced this pull request Oct 9, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
src:
  * mark node `--run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: TODO
@aduh95 aduh95 mentioned this pull request Oct 9, 2024
aduh95 added a commit that referenced this pull request Oct 9, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
src:
  * mark node `--run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 9, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
src:
  * mark node `--run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 11, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
src:
  * mark node `--run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 11, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 11, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 11, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 13, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) #54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 13, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) #54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 16, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) #54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
aduh95 added a commit that referenced this pull request Oct 16, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) #55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) #54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) #55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) #54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) #54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) #54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) #54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) #54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) #54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) #55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) #54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) #53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) #55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) #54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) #53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) #55234

PR-URL: #55343
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
This works better in terms of avoiding race conditions.

PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.

PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) nodejs#55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) nodejs#54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) nodejs#55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) nodejs#54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) nodejs#54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) nodejs#54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) nodejs#54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) nodejs#54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) nodejs#54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) nodejs#54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) nodejs#55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) nodejs#54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) nodejs#53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) nodejs#55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) nodejs#54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) nodejs#53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) nodejs#55234

PR-URL: nodejs#55343
tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024
This works better in terms of avoiding race conditions.

PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024
PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024
This implements an API for users to intentionally flush the
accumulated compile cache instead of waiting until process
shutdown. It may be useful for application that loads dependencies
first and then either reload itself in other instances, or spawning
other instances that load an overlapping set of its dependencies -
in this case its useful to flush the cache early instead of waiting
until the shutdown of itself.

Currently flushing is triggered by either process
shutdown or user requests. In the future we should simply start the
writes right after module loading on a separate thread, and this method
only blocks until all the pending writes (if any) on the other thread
are finished. In that case, the off-thread writes should finish long
before any attempt of flushing is made so the method would then only
incur a negligible overhead from thread synchronization.

PR-URL: nodejs#54971
Fixes: nodejs#54770
Fixes: nodejs#54465
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
tpoisseau pushed a commit to tpoisseau/node that referenced this pull request Nov 21, 2024
Notable changes:

crypto:
  * (SEMVER-MINOR) add `KeyObject.prototype.toCryptoKey` (Filip Skokan) nodejs#55262
  * (SEMVER-MINOR) add Date fields for `validTo` and `validFrom` (Andrew Moon) nodejs#54159
doc:
  * add abmusse to collaborators (Abdirahim Musse) nodejs#55086
http2:
  * (SEMVER-MINOR) expose `nghttp2_option_set_stream_reset_rate_limit` as an option (Maël Nison) nodejs#54875
lib:
  * (SEMVER-MINOR) propagate aborted state to dependent signals before firing events (jazelly) nodejs#54826
module:
  * (SEMVER-MINOR) support loading entrypoint as url (RedYetiDev) nodejs#54933
  * (SEMVER-MINOR) implement the `"module-sync"` exports condition (Joyee Cheung) nodejs#54648
  * (SEMVER-MINOR) implement `flushCompileCache()` (Joyee Cheung) nodejs#54971
  * (SEMVER-MINOR) throw when invalid argument is passed to `enableCompileCache()` (Joyee Cheung) nodejs#54971
  * (SEMVER-MINOR) write compile cache to temporary file and then rename it (Joyee Cheung) nodejs#54971
process:
  * (SEMVER-MINOR) add `process.features.require_module` (Joyee Cheung) nodejs#55241
  * (SEMVER-MINOR) add `process.features.typescript` (Aviv Keller) nodejs#54295
src:
  * mark `node --run` as stable (Yagiz Nizipli) nodejs#53763
test_runner:
  * (SEMVER-MINOR) support custom arguments in `run()` (Aviv Keller) nodejs#55126
  * (SEMVER-MINOR) add `'test:summary'` event (Colin Ihrig) nodejs#54851
  * (SEMVER-MINOR) add support for coverage via `run()` (Chemi Atlow) nodejs#53937
worker:
  * (SEMVER-MINOR) add `markAsUncloneable` api (Jason Zhang) nodejs#55234

PR-URL: nodejs#55343
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. lib / src Issues and PRs related to general changes in the lib or src directory. module Issues and PRs related to the module subsystem. needs-ci PRs that need a full CI run. semver-minor PRs that contain new features and should be released in the next minor version.
Projects
None yet
6 participants