Emit bundler-friendly URL locators #13571
Comments
|
To elaborate: in This pattern is increasingly commonly recognised by tooling, including major bundlers, and would allow to bundle Emscripten-generated JS more effortlessly. It's, of course, also a valid runtime expression to get a URL of the Wasm file relatively to the module, so it works directly in browsers too. |
RReverser
changed the title
|
I tried to take a stab at this, but ran into a blocker on the Closure side :( google/closure-compiler#3707 |
|
No, wait, I think it's on Terser side... |
|
Ah it's both. |
This adds the import meta rollup extension which will copy assets that use this pattern:
```js
new URL('./compiler.wasm', import.meta.url);
```
Among other things, this helps support for wasm when the above pattern is used.
* Rust now uses this pattern, so bundlers can include the wasm file. https://twitter.com/RReverser/status/1365342246653018115
* Emscripten (compiles C/C++) is working on support: emscripten-core/emscripten#13571 (comment)
This will also help with other types of static assets loaded this way, such as images.
This adds the import meta rollup extension which will copy assets that use this pattern:
```js
new URL('./compiler.wasm', import.meta.url);
```
Among other things, this helps support for wasm when the above pattern is used.
* Rust now uses this pattern, so bundlers can include the wasm file. https://twitter.com/RReverser/status/1365342246653018115
* Emscripten (compiles C/C++) is working on support: emscripten-core/emscripten#13571 (comment)
This will also help with other types of static assets loaded this way, such as images.
|
Perhaps this could be implemented in |
|
@kripken I actually worked around that issue, but then ran into Maybe if someone wants to tackle #11792, I'm happy to do just this one on top. |
|
Yes, unfortunately despite being there for years, EXPORT_ES6 is still fundamentally broken. :( |
|
I worked around the minifier support bug myself by instead using |
|
@matthewp Yeah I did something similar in Emscripten patch too; the Node issue / scriptDirectory still needs to be fixed though. |
|
I haven't ran into the |
|
@RReverser Ah yeah, I now see what you mean about the nested |
Yes, We use esbuild to bundle&minify wasm|js in production, which does not rely on terser or google closure compiler, I think wasm bundle strategy should be left to user land(so many strategies could be token),and do not think bound with terser or Closure is an good idea. |
|
@hardfist Not really; running Closure on practice is still useful regardless of the final minifier, because Closure can do a lot better minification using Emscripten-emitted annotations, whereas any regular minifier would just ignore those comments and would have less leeway in optimizing code, removing unused pieces, renaming properties and so on. |
|
any one working on this ? It's really annoying to deploy an universal wasm library without this feature. and we're ok with the unminify version of glue js. |
|
It looks like closure fixed this meanwhile. Perhaps terser has too? If so maybe we can update. If terser is a blocker here, we could consider adding an option to not run it even when optimizing. The results would be less optimal (e.g. metadce would not run), but it would unblock people? |
|
Yes, there is more than Closure though - I've actually made quite a bit of progress on this (again) today and can make a PR next week that will cover most common scenarios. (For now I've decided to treat Node.js + EXPORT_ES6 as simply unsupported combo, since it's already broken in other ways and it allows to unblock this issue otherwise.) |
This PR changes loading of main Wasm binary as well as helper Worker used by PThread integration to use a URL expression that can be both used directly by browsers as well as statically detected by bundlers like Webpack. The main caveats are: 1. Emscripten is very configurable, so some of the new conditions might look odd but are necessary to keep backward compatibility and allow overriding bundler-friendly URL with a custom one during runtime. 2. None of Closure, our fork of Terser, and `eval` (which is used by Emscripten's JS library preprocessing) support `import.meta` expressions without more work. While Closure seems to have _just_ implemented such support, and it wouldn't be too hard to add it to our Terser too, `eval` usage would still require a string replacement before execution (or complete revamp). To keep implementation simple, for now I went with just string replacement that covers all tools - this way, we replace `import.meta` -> `EMSCRIPTEN$IMPORT$META` only once per library when JS is added before any of this tooling is executed, and then replace back after everything is done right before the final emit. We might want to revisit this in future, but for now this works well and covers all the tooling incompatibilities together. 3. Webpack assumes that all modules are strict mode, so I updated `worker.js` correspondingly to avoid usages of global `this` (which is `undefined` in strict mode and breaks in bundled code) and instead using `self`; I've also updated the Node.js adapter code to satisfy strict requirements too and to be a bit simpler. 4. This won't work in Node.js, since it's not compatible with `EXPORT_ES6` in general yet. 5. I've only updated places for loading main Wasm binary and PThread code. This should cover majority of use-cases, but other external files like side modules, proxy-to-pthread, proxy-to-worker, external memory loading etc. are not covered by this PR and need to be updated separately if someone wants them to work with bundlers out of the box too. Fixes #13571.
|
Sent a PR at #14135. |
This PR changes loading of main Wasm binary as well as helper Worker used by PThread integration to use a URL expression that can be both used directly by browsers as well as statically detected by bundlers like Webpack. The main caveats are: 1. Emscripten is very configurable, so some of the new conditions might look odd but are necessary to keep backward compatibility and allow overriding bundler-friendly URL with a custom one during runtime. 2. None of Closure, our fork of Terser, and `eval` (which is used by Emscripten's JS library preprocessing) support `import.meta` expressions without more work. While Closure seems to have _just_ implemented such support, and it wouldn't be too hard to add it to our Terser too, `eval` usage would still require a string replacement before execution (or complete revamp). To keep implementation simple, for now I went with just string replacement that covers all tools - this way, we replace `import.meta` -> `EMSCRIPTEN$IMPORT$META` only once per library when JS is added before any of this tooling is executed, and then replace back after everything is done right before the final emit. We might want to revisit this in future, but for now this works well and covers all the tooling incompatibilities together. 3. Webpack assumes that all modules are strict mode, so I updated `worker.js` correspondingly to avoid usages of global `this` (which is `undefined` in strict mode and breaks in bundled code) and instead using `self`; I've also updated the Node.js adapter code to satisfy strict requirements too and to be a bit simpler. 4. This won't work in Node.js, since it's not compatible with `EXPORT_ES6` in general yet. 5. I've only updated places for loading main Wasm binary and PThread code. This should cover majority of use-cases, but other external files like side modules, proxy-to-pthread, proxy-to-worker, external memory loading etc. are not covered by this PR and need to be updated separately if someone wants them to work with bundlers out of the box too. Fixes #13571.
This PR changes loading of main Wasm binary as well as helper Worker used by PThread integration to use a URL expression that can be both used directly by browsers as well as statically detected by bundlers like Webpack. The main caveats are: 1. Emscripten is very configurable, so some of the new conditions might look odd but are necessary to keep backward compatibility and allow overriding bundler-friendly URL with a custom one during runtime. 2. None of Closure, our fork of Terser, or even `eval` (which is used by Emscripten's JS library preprocessing) support `import.meta` expressions without more work. While Closure seems to have _just_ implemented such support, and it wouldn't be too hard to add it to our Terser too, `eval` usage would still require a string replacement before execution (or complete revamp). To keep implementation simple, for now I went with just string replacement that covers all tools - this way, we replace `import.meta` -> `EMSCRIPTEN$IMPORT$META` only once when JS is added before any of this tooling is executed, and then replace back after everything is done right before the final emit. We might want to revisit this in future, but for now this works well and covers all the tooling incompatibilities together. 3. This won't work in Node.js, since it's not compatible with `EXPORT_ES6` in general yet. 4. I've only updated places for loading main Wasm binary and PThread code. This should cover majority of use-cases, but other external files like side modules, proxy-to-pthread, proxy-to-worker, external memory loading etc. are not covered by this PR and need to be updated separately if someone wants them to work with bundlers out of the box too. Fixes #13571.
according to https://twitter.com/RReverser/status/1365342246653018115 ,wasm-bindgen already support emit code import wasm using import, which is very helpful for bundler, I wonder whether emscripten could support the same feature?
The text was updated successfully, but these errors were encountered: