feat: support WebAssembly (Wasm) imports in ESM modules

[kachkaev]

Summary

Closes #11011

• import * as myWasm from './my-wasm.wasm';
• const myWasmModule = await import('./my-wasm.wasm');
• const myWasmModule = await import('data:application/wasm;base64,...');

Related links

• jestjs.io/docs/ecmascript-modules
• webassembly.org

Test plan

See newly added tests

[facebook-github-bot]

Hi @kachkaev!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[SimenB]

exciting stuff!

[kachkaev]

Interesting: CI has passed with and without --experimental-wasm-modules. I guess that in our case the usage of this flag is not necessary, because wasm imports are implemented via new SyntheticModule(...) (--experimental-vm-modules).

Let me try to get rid of the flag completely and merge native-esm-wasm with native-esm for simplicity.

[kachkaev]

Done in dcff11e. Because --experimental-wasm-modules is not necessary, seems like we don’t even have to update the docs! WASM just works™

Existing markdown still looks accurate: https://jestjs.io/docs/ecmascript-modules. It does not mention all supported cases, so not sure if wasm should be any special here.

[SimenB]

Awesome! It might make sense to mention that wasm doesn't need an extra flag (at least at the moment - node might decide to unflag vm modules and at that point require it for wasm? unsure)

[kachkaev]

Done in 27d73b6 (happy to improve the docs further based on your suggestions).

[kachkaev]

File source: https://github.com/hasharchives/wasm-ts-esm-in-node-jest-and-nextjs

Happy to replace this wasm with some canonical example file, but I don’t know any. Ideally, this file would export a function with arguments (mine is just getAnswer = () => 42).

[kachkaev]

Maybe something from https://github.com/mdn/webassembly-examples?

[SimenB]

that seems reasonable 👍

[kachkaev]

Done in 79e0f40, using add(i32, i32) from mdn/webassembly-examples.

[kachkaev]

Quite a few lines below are marked as changed because of extra indent. Hiding whitespace in diff viewer makes it a bit easier to see the real difference.

[SimenB]

very nice!

[SimenB]

we have a readFile helper which caches the result. we should do the same here. readFile currently reads as utf8, so maybe just a separate readFileBuffer?

[kachkaev]

Done in 0d4945e. I had to change the type for _cacheFS, which unfortunately spills outside jest-runtme. Happy to improve the implementation if you have any suggestions.

[SimenB]

should these be put into this. _esmoduleRegistry as well?

[kachkaev]

I’ve replaced linkAndEvaluateModule with loadEsmModule in 26db3fd. Not 100% sure if this is right, but AFAIU, loadEsmModule is a wrapper function that deals with _esmoduleRegistry. If that’s not the best solution, feel free to push to my branch if the required change is not trivial.

[kachkaev]

Hmmm I should have kept linkAndEvaluateModule. Using loadEsmModule here can result

TypeError: modulePath.endsWith is not a function
  at isWasm (../../../node_modules/jest-runtime/build/index.js:210:41)

This happens when Wasm imports refer back to a javascript file that calls it. E.g.:

imports: [
  {
    module: './index_bg.js',
    name: '__wbindgen_json_parse',
    kind: 'function'
  },
  {
    module: './index_bg.js',
    name: '__wbindgen_json_serialize',
    kind: 'function'
  },
  {
    module: './index_bg.js',
    name: '__wbindgen_throw',
    kind: 'function'
  }
]

loadEsmModule returns

modulePath: SourceTextModule {
  status: 'linking',
  identifier: '/path/to/index_bg.js',
  context: {
    clearInterval: [Function: clearInterval],
    clearTimeout: [Function: clearTimeout],
    setInterval: [Function: setInterval],
    setTimeout: [Function: setTimeout] {
      [Symbol(nodejs.util.promisify.custom)]: [Getter]
    },

which is then mistakenly passed to to isWasm instead of a string.

I can try submiting a follow-up PR with a fix in a few days.

[SimenB]

Thanks!

[kachkaev]

#13608

[mationorato]

I found this out because I am facing the same error. I have two libraries that make use of wasm under the hood waiting for proper testing. Looking forward to seeing this PR merged. If I can help with something just let me know!
Thanks!

[SimenB]

I don't think we should change this type.

I can tinker with this 👍

[SimenB]

Thanks!

[kachkaev]

@SimenB aren’t we doubling memory footprint by having these two caches? Most opened files will end up with an entry in both _cacheFSBuffer and _cacheFS, while all reads will be from _cacheFS except the first one.

The impact can be insignificant, but there is a small chance that some tests may now fail with JavaScript heap out of memory in a new version. WDYT?

[SimenB]

https://github.com/facebook/jest/releases/tag/v29.3.0

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.