-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fs: add c++ fast path for writeFileSync string utf8 #49884
fs: add c++ fast path for writeFileSync string utf8 #49884
Conversation
The
notable-change
Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. |
98c5cd5
to
7ab6586
Compare
Rebased on latest main and addressed various review comments. Should be good for review fully now. |
@santigimeno and/or @bnoordhuis, would you mind checking/reviewing the latest revision? Uses a write loop like original JS source now. Thanks! |
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 deps: * add simdjson (Yagiz Nizipli) #50322 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * merge config with `package_json_reader` (Yagiz Nizipli) #50322 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * move package resolver to c++ (Yagiz Nizipli) #50322 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) create fs_dir per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) create worker per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 PR-URL: #50954
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) create fs_dir per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) create worker per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 PR-URL: #50954
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) create fs_dir per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) create worker per isolate properties (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 PR-URL: #50954
These test create temporary folders using test case titles. It started to fail recently likely due to newer NodeJS version 21.3.0. There are some changes around `fs.writeFileSync`. nodejs/node#49884 Same tests were passing on 21.2.0. This PR changes the test to use simpler names.
Some of our tests using Is this related? |
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: TODO
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
PR-URL: #49884 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Stephen Belanger <admin@stephenbelanger.com> Reviewed-By: James M Snell <jasnell@gmail.com>
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) #50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) #50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) #50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) #48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) #49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) #48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) #48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) #50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) #48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) #48655 stream: * use Array for Readable buffer (Robert Nagy) #50341 * optimize creation (Robert Nagy) #50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) #50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) #48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) #50443 PR-URL: #51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) nodejs#50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) nodejs#50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) nodejs#50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) nodejs#48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) nodejs#49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) nodejs#48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) nodejs#48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) nodejs#50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) nodejs#48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) nodejs#48655 stream: * use Array for Readable buffer (Robert Nagy) nodejs#50341 * optimize creation (Robert Nagy) nodejs#50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) nodejs#50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) nodejs#48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) nodejs#50443 PR-URL: nodejs#51124
Notable changes: crypto: * update root certificates to NSS 3.95 (Node.js GitHub Bot) nodejs#50805 doc: * add MrJithil to collaborators (Jithil P Ponnan) nodejs#50666 * add Ethan-Arrowood as a collaborator (Ethan Arrowood) nodejs#50393 esm: * (SEMVER-MINOR) add import.meta.dirname and import.meta.filename (James Sumners) nodejs#48740 fs: * add c++ fast path for writeFileSync utf8 (CanadaHonk) nodejs#49884 module: * (SEMVER-MINOR) remove useCustomLoadersIfPresent flag (Chengzhong Wu) nodejs#48655 * (SEMVER-MINOR) bootstrap module loaders in shadow realm (Chengzhong Wu) nodejs#48655 src: * (SEMVER-MINOR) add `--disable-warning` option (Ethan Arrowood) nodejs#50661 * (SEMVER-MINOR) create per isolate proxy env template (Chengzhong Wu) nodejs#48655 * (SEMVER-MINOR) make process binding data weak (Chengzhong Wu) nodejs#48655 stream: * use Array for Readable buffer (Robert Nagy) nodejs#50341 * optimize creation (Robert Nagy) nodejs#50337 test_runner: * (SEMVER-MINOR) adds built in lcov reporter (Phil Nash) nodejs#50018 * (SEMVER-MINOR) add Date to the supported mock APIs (Lucas Santos) nodejs#48638 test_runner, cli: * (SEMVER-MINOR) add --test-timeout flag (Shubham Pandey) nodejs#50443 PR-URL: nodejs#51124
Summary
Added fast path in almost entirely C++ for
writeFileSync
with UTF8 encoding and string data. Also improvesappendFileSync
as it just useswriteFileSync
under the hood. Only string data as Buffer seems questionable in benchmarks for this so I'll just leave it for strings for now.TL;DR: This makes
writeFileSync(path, data: string)
UTF8 up to ~2.5x faster depending on data size, especially when using file descriptorsBench results
Benchmark in this PR
Note: running locally on Linux/i9/SSD
Benchmark CI (old): https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1421/
Alternative benchmark
Alternative benchmark using Bun's bench for
fs.copyFileSync
(string data, using paths)Current (main)
This PR
Long ascii:
6.14µs
->3.46µs
(~1.8x speedup)Long utf8:
40.02µs
->18.25µs
(~2.2x speedup)