[native_assets_cli] build.dart / link.dart web backends and WasmCode and JsCode assets

[dcharkes]

At some point, we'd like to output assets for web:

• DataAsset ([native_assets_cli] data assets #154)
• WasmCode (Compilation of native code to WASM #156)
• JSCode

And WasmCode and it should have a dialect field that is linear_memory, wasm_gc. (I'm assuming the linking might work differently for the two dialects.)

Besides asset types to output, we should also have the BuildConfig communicate that we're targeting "web" instead of "native".
The question is how we should surface this in the protocol and Dart API. And this depends on differences in target.

I believe build.dart and link.dart should consider dart compile js the same as flutter build web and dart compile wasm the same as flutter build web --wasm. I believe it would be an anti-pattern if build.dart or link.dart would branch on Dart vs Flutter, if we want packages to work both in Dart and Flutter. (It's fine to branch on web vs Android vs MacOS.) #946 (comment)

@mosuem suggested to extend the API with: OS.web, Architecture.js and Architecture.wasm (and extend the protocol ditto).

I don't know enough about dart2js/dart2wasm and Flutter for web to know if it's reasonable to let users branch on whether we're compiling dart to js or wasm.
I'm thinking that theoretically it might be enough to just introduce OS.web, and have the dart-js-interop and dart-wasm-interop work for any backend (js or wasm). However, that would require the dart code interoperating with js and wasm to work with both the js and wasm backends. In practice, I expect dart2js to not support wasm interop but to require a wrapper JS layer. Therefore the build.dart would need to output that wrapper when targeting Architecture.js. Hence, we need to introduce Architecture.js and Architecture.wasm alongside OS.web.

@eyebrowsoffire @mkustermann @osa1 Could you comment on how WASM and WASM GC code files are linked into a flutter build web --wasm / flutter build web / dart compile wasm application? (I'm aware that we can currently only have a single linear memory module, but conceptually how would this work?) Pointers to examples and the implementation in the Dart and Flutter SDK would be greatly appreciated. 🙏

@sigmundch Who is the right person to ask about the (new) JS interop? Does the (new) JS interop work seamlessly when compiling Dart to JS and compiling Dart to WASM? Does dart compile js have any support for including WASM code in a web app? (e.g. if Architecture.js should we allow outputting WasmCode assets at all?

Note, we don't need to answer all these questions now, or commit to supporting this. I just want to make sure that while designing the API we design it in a way so that we can support these use cases later.

[sigmundch]

Who is the right person to ask about the (new) JS interop?

@srujzs is our main point of contact 😃

Does the (new) JS interop work seamlessly when compiling Dart to JS and compiling Dart to WASM?

That's the goal, for the most part. There are some caveats/inconsistencies that we have to live with, though (e.g. like differences between null/undefined, different default conversions of arrays, among others). That aside, yes, the new dart:js_interop model is meant to work on both. This is what enables the future flutter web based on dart:js_interop and package:web (DOM APIs layered on top of this new interop) to work on both JS and WasmGC seamlessly.

That said, the JS interop story centers on accessing JavaScript from Dart and not on accessing Wasm modules. Today, we don't have much to support the latter. Users would need to manually create an export layer and expose it via JavaScript as another JS interop interface. We've considered that a JS-2-wasm and wasm-2-wasm interface is more aligned with a Dart-2-native interface, and as such, it we could potentially use dart:ffi's syntax in the future to support it.

Does dart compile js have any support for including WASM code in a web app?

Nope. Nothing is done automatically, if users manage their own assets and provide a JS interface, then that works.

It's probably worth noting that we also don't do any management for JS assets, either. Today most developers manage this by hand. Some include assets in their lib/ folders with the Dart code to allow sharing across packages. This is not ideal as it can interleave Dart and JS build toolchains unnecessarily. Internally, in bazel, we do have some BUILD macros that assist in the process and allows disentangling the dependencies. This is then used to link all js assets when creating application targets.

[mkustermann]

I believe build.dart and link.dart should consider dart compile js the same as flutter build web and dart compile wasm the same as flutter build web --wasm

dart compile wasm is similar to dart compile exe - it's mainly focused on compiling dart code itself.

Wouldn't dart build <js/exe/...> (the new command @dcharkes added under experimental flag) be the analogous to flutter build web?

I believe it would be an anti-pattern if build.dart or link.dart would branch on Dart vs Flutter, if we want packages to work both in Dart and Flutter.

Generally true, but the flutter build command may offer more functionality, more asset types, etc than dart build does. When building for Android it may e.g. allow building & including java code for example.

@eyebrowsoffire @mkustermann @osa1 Could you comment on how WASM and WASM GC code files are linked into a flutter build web --wasm / flutter build web / dart compile wasm application? (I'm aware that we can currently only have a single linear memory module, but conceptually how would this work?) Pointers to examples and the implementation in the Dart and Flutter SDK would be greatly appreciated. 🙏

You can call dart compile wasm hello.dart and see the output: It will produce an hello.mjs file that can e.g. be used by an index.html (like flutter does). This hello.mjs file has an exported instantiate function that can given the wasm module and things that module needs (via imports). It also exposes a invoke function that will invoke the main function.

So outside code (like flutter or non-flutter) is responsible for

• loading the hello.wasm file
• loading the hello.mjs file & calling instantiate and providing all necessary imports
• calling the invoke

Any used linear memory e.g. is provided to the wasm module via the imports (e.g. a WebAssembly.Memory). A wasm module could import functions that are exported by another wasm module. Then multiple wasm modules can operate on the same memory.

Though one may think about whether it would make sense to statically link wasm modules together.

Does the (new) JS interop work seamlessly when compiling Dart to JS and compiling Dart to WASM?

The new JS interop is designed for Dart code to interop with JavaScript. Yes, ignoring a few corner cases, code written using the new JS interop should behave the same whether compiled with dart2js or dart2wasm.

Though none of the two have an official interop with linear memory wasm (dart2wasm supports a subset of dart:ffi - sufficient enough to interact with skwasm/skia - but this isn't supported to 3rd party users yet).