Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib,src: implement WebAssembly Web API #42701

Merged
merged 1 commit into from
Apr 23, 2022

Conversation

tniessen
Copy link
Member

Now that we have fetch(), we may as well also have WebAssembly.compileStreaming() and WebAssembly.instantiateStreaming().

Unfortunately, our fetch() implementation cannot be interacted with easily from C++, which is why part of this implementation ended up in JavaScript. The C++ glue code allows controlling the v8::WasmStreaming instance from JavaScript.

This attempts to be a spec-compliant implementation with no node-specific extensions.

Refs: #41749
Fixes: #21130

@tniessen tniessen added notable-change PRs with changes that should be highlighted in changelogs. wasm Issues and PRs related to WebAssembly. fetch Issues and PRs related to the Fetch API labels Apr 12, 2022
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/gyp
  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 12, 2022
new WasmStreamingObject(env, args.This());
}

void WasmStreamingObject::Push(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless i am mistaken, these methods do not allocate. would you be up for adding fast call wrappers? if not i can maybe add them in a followup pr.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try that tomorrow. So far, every time I tried using fast calls, I can into some V8 limitation, but Push might be a good candidate for once.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, given how web streams are performing so far and how slow everything is, that seems like premature optimization. I might create a follow-up PR to add that or I might push that as a separate commit later, but I'd like to get this landed rather sooner than later and not make it more complicated to review.

@targos targos added the semver-minor PRs that contain new features and should be released in the next minor version. label Apr 12, 2022
src/node_wasm_web_api.h Outdated Show resolved Hide resolved
src/node_wasm_web_api.h Outdated Show resolved Hide resolved
src/node_wasm_web_api.h Outdated Show resolved Hide resolved
src/node_wasm_web_api.cc Outdated Show resolved Hide resolved
src/node_wasm_web_api.cc Outdated Show resolved Hide resolved
src/node_wasm_web_api.h Show resolved Hide resolved
@tniessen tniessen force-pushed the wasm-web-api branch 4 times, most recently from f76bda6 to 8689723 Compare April 13, 2022 00:58
@tniessen tniessen added the review wanted PRs that need reviews. label Apr 17, 2022
@tniessen tniessen added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 17, 2022
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add WTP tests? Maybe https://github.com/web-platform-tests/wpt/tree/HEAD/wasm/webapi or a subset of it?

doc/api/errors.md Show resolved Hide resolved
@tniessen
Copy link
Member Author

Should we add WTP tests?

That's a good idea. It's not straightforward because many WPT tests for this API rely on fetch and we currently mock fetch in WPT.

Of the ones I can run easily, wasm/webapi/empty-body.any.js is failing, presumably due to nodejs/undici#1345.

@aduh95
Copy link
Contributor

aduh95 commented Apr 18, 2022

Should we add WTP tests?

That's a good idea. It's not straightforward because many WPT tests for this API rely on fetch and we currently mock fetch in WPT.

Of the ones I can run easily, wasm/webapi/empty-body.any.js is failing, presumably due to nodejs/undici#1345.

I think you can create a status file that lists all the failing tests, and that can be used as a TODO list for contributors who want to improve Node.js API compliance. But maybe that's a lot of work (I've never done that myself so not sure), and it probably should not block this from landing.

RafaelGSS pushed a commit that referenced this pull request May 10, 2022
Refs: #42701
Refs: nodejs/undici#1346
Refs: #42939

PR-URL: #42960
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
rekmarks added a commit to MetaMask/snaps that referenced this pull request May 14, 2022
Node landed `WebAssembly.compileStreaming` and `instantiateStreaming` in nodejs/node#42701 and made it available in Node 18, so we no longer need to exclude these properties from our WebAssembly endowment. This PR therefore removes the WebAssembly endowment factory, which will now be included in whatever form it exists on the execution environment root realm global.
@juanarbol
Copy link
Member

I think it's fine to land it on v16.x, because it's behind the --experimental-fetch flag.

Yeap, it does, It lands fine; except for the bootstrap, tried to fix this by adding the binding wasm_web_api to lib/internal/bootstrap/loaders.js with no sucess, I will label this to request backport:

make[1]: Leaving directory '/home/juanarbol/GitHub/node/test/node-api/test_worker_terminate_finalization/build'                                     [487/5756]
/usr/bin/python3.10 tools/test.py  --mode=release \                                                                                                           
         \                                                                                                                                                    
        --skip-tests= \                                                                                                                                       
        default \                                                                                                                                             
        addons js-native-api node-api                                                                                                                         
=== release test-bootstrap-modules ===                                                                                                                        
Path: parallel/test-bootstrap-modules                                                                                                                         
node:assert:123                                                                                                                                               
  throw new AssertionError(obj);                                                                                                                              
  ^                                                                                                                                                           
                                                                                                                                                              
AssertionError [ERR_ASSERTION]: These modules were not loaded:                                                                                                
  Internal Binding wasm_web_api                                                                                                                               
                                                                                                                                                              
    at Object.<anonymous> (/home/juanarbol/GitHub/node/test/parallel/test-bootstrap-modules.js:216:8)                                                         
    at Module._compile (node:internal/modules/cjs/loader:1105:14)                                                                                             
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)                                                                               
    at Module.load (node:internal/modules/cjs/loader:981:32)                                                                                                  
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)                                                                                        
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)                                                                     
    at node:internal/main/run_main_module:17:47 {

juanarbol pushed a commit that referenced this pull request May 31, 2022
Refs: #42660
Refs: #42701

PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
danielleadams pushed a commit that referenced this pull request Jun 27, 2022
Refs: #42660
Refs: #42701

PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Jul 12, 2022
Refs: #42660
Refs: #42701

PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
targos pushed a commit that referenced this pull request Jul 31, 2022
Refs: #42660
Refs: #42701

PR-URL: #42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
guangwong pushed a commit to noslate-project/node that referenced this pull request Oct 10, 2022
Refs: nodejs/node#42660
Refs: nodejs/node#42701

PR-URL: nodejs/node#42836
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
codebytere added a commit to electron/electron that referenced this pull request Oct 24, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 24, 2022
codebytere added a commit to electron/electron that referenced this pull request Oct 26, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 8, 2022
codebytere added a commit to electron/electron that referenced this pull request Nov 10, 2022
* chore: update to Node.js v18

* child_process: improve argument validation

nodejs/node#41305

* bootstrap: support configure-time user-land snapshot

nodejs/node#42466

* chore: update GN patch

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* src: use a typed array internally for process._exiting

nodejs/node#43883

* chore: lib/internal/bootstrap -> lib/internal/process

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* chore: remove redudant browserGlobals patch

* chore: update BoringSSL patch

* src: allow embedder-provided PageAllocator in NodePlatform

nodejs/node#38362

* chore: fixup Node.js crypto tests

- nodejs/node#44171
- nodejs/node#41600

* lib: add Promise methods to avoid-prototype-pollution lint rule

nodejs/node#43849

* deps: update V8 to 10.1

nodejs/node#42657

* src: add kNoBrowserGlobals flag for Environment

nodejs/node#40532

* chore: consolidate asar initialization patches

* deps: update V8 to 10.1

nodejs/node#42657

* deps: update V8 to 9.8

nodejs/node#41610

* src,crypto: remove AllocatedBuffers from crypto_spkac

nodejs/node#40752

* build: enable V8's shared read-only heap

nodejs/node#42809

* src: fix ssize_t error from nghttp2.h

nodejs/node#44393

* chore: fixup ESM patch

* chore: fixup patch indices

* src: merge NativeModuleEnv into NativeModuleLoader

nodejs/node#43824

* [API] Pass OOMDetails to OOMErrorCallback

https://chromium-review.googlesource.com/c/v8/v8/+/3647827

* src: iwyu in cleanup_queue.cc

* src: return Maybe from a couple of functions

nodejs/node#39603

* src: clean up embedder API

nodejs/node#35897

* src: refactor DH groups to delete crypto_groups.h

nodejs/node#43896

* deps,src: use SIMD for normal base64 encoding

nodejs/node#39775

* chore: remove deleted source file

* chore: update patches

* chore: remove deleted source file

* lib: add fetch

nodejs/node#41749

* chore: remove nonexistent node specs

* test: split report OOM tests

nodejs/node#44389

* src: trace fs async api

nodejs/node#44057

* http: trace http request / response

nodejs/node#44102

* test: split test-crypto-dh.js

nodejs/node#40451

* crypto: introduce X509Certificate API

nodejs/node#36804

* src: split property helpers from node::Environment

nodejs/node#44056

* nodejs/node#38905

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob

* lib,src: implement WebAssembly Web API

nodejs/node#42701

* fixup! deps,src: use SIMD for normal base64 encoding

* fixup! src: refactor DH groups to delete crypto_groups.h

* chore: fixup base64 GN file

* fix: check that node::InitializeContext() returns true

* chore: delete _noBrowserGlobals usage

* chore: disable fetch in renderer procceses

* dns: default to verbatim=true in dns.lookup()

nodejs/node#39987

Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
khalwa pushed a commit to solarwindscloud/electron that referenced this pull request Feb 22, 2023
* chore: update to Node.js v18

* child_process: improve argument validation

nodejs/node#41305

* bootstrap: support configure-time user-land snapshot

nodejs/node#42466

* chore: update GN patch

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* src: use a typed array internally for process._exiting

nodejs/node#43883

* chore: lib/internal/bootstrap -> lib/internal/process

* src: disambiguate terms used to refer to builtins and addons

nodejs/node#44135

* chore: remove redudant browserGlobals patch

* chore: update BoringSSL patch

* src: allow embedder-provided PageAllocator in NodePlatform

nodejs/node#38362

* chore: fixup Node.js crypto tests

- nodejs/node#44171
- nodejs/node#41600

* lib: add Promise methods to avoid-prototype-pollution lint rule

nodejs/node#43849

* deps: update V8 to 10.1

nodejs/node#42657

* src: add kNoBrowserGlobals flag for Environment

nodejs/node#40532

* chore: consolidate asar initialization patches

* deps: update V8 to 10.1

nodejs/node#42657

* deps: update V8 to 9.8

nodejs/node#41610

* src,crypto: remove AllocatedBuffers from crypto_spkac

nodejs/node#40752

* build: enable V8's shared read-only heap

nodejs/node#42809

* src: fix ssize_t error from nghttp2.h

nodejs/node#44393

* chore: fixup ESM patch

* chore: fixup patch indices

* src: merge NativeModuleEnv into NativeModuleLoader

nodejs/node#43824

* [API] Pass OOMDetails to OOMErrorCallback

https://chromium-review.googlesource.com/c/v8/v8/+/3647827

* src: iwyu in cleanup_queue.cc

* src: return Maybe from a couple of functions

nodejs/node#39603

* src: clean up embedder API

nodejs/node#35897

* src: refactor DH groups to delete crypto_groups.h

nodejs/node#43896

* deps,src: use SIMD for normal base64 encoding

nodejs/node#39775

* chore: remove deleted source file

* chore: update patches

* chore: remove deleted source file

* lib: add fetch

nodejs/node#41749

* chore: remove nonexistent node specs

* test: split report OOM tests

nodejs/node#44389

* src: trace fs async api

nodejs/node#44057

* http: trace http request / response

nodejs/node#44102

* test: split test-crypto-dh.js

nodejs/node#40451

* crypto: introduce X509Certificate API

nodejs/node#36804

* src: split property helpers from node::Environment

nodejs/node#44056

* nodejs/node#38905

bootstrap: implement run-time user-land snapshots via --build-snapshot and --snapshot-blob

* lib,src: implement WebAssembly Web API

nodejs/node#42701

* fixup! deps,src: use SIMD for normal base64 encoding

* fixup! src: refactor DH groups to delete crypto_groups.h

* chore: fixup base64 GN file

* fix: check that node::InitializeContext() returns true

* chore: delete _noBrowserGlobals usage

* chore: disable fetch in renderer procceses

* dns: default to verbatim=true in dns.lookup()

nodejs/node#39987

Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com>
nicolo-ribaudo added a commit to nicolo-ribaudo/browser-compat-data that referenced this pull request Sep 14, 2023
Implemented in nodejs/node#42701. Manually tested that it's present in Node.js 18.1 and not in 18.0.
queengooborg added a commit to mdn/browser-compat-data that referenced this pull request Sep 18, 2023
* Add data for `WebAssebly.*Streaming` in Node.js

Implemented in nodejs/node#42701. Manually tested that it's present in Node.js 18.1 and not in 18.0.

* Apply suggestions from code review

Co-authored-by: Queen Vinyl Da.i'gyu-Kazotetsu <vinyldarkscratch@gmail.com>

---------

Co-authored-by: Queen Vinyl Da.i'gyu-Kazotetsu <vinyldarkscratch@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. experimental Issues and PRs related to experimental features. fetch Issues and PRs related to the Fetch API lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version. wasm Issues and PRs related to WebAssembly.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support WebAssembly.instantiateStreaming
8 participants