Improve JS interop

[jmesserly]

We'd like to significantly improve Dart's ability to interoperate with JavaScript libraries and vice versa. This will build on existing capabilities (see examples of current JS interop).

The new design will take advantage of Dart 2's sound static typing to provide JS interop that is convenient, high performance, and with minimal code size impact.

Some of the tasks for this:

• Fully specify JS interop, including all details (Need spec and compile-time errors of incorrect JS interop usage #32929)
• Fix incompatibilities between dart2js and dartdevc
• Design it to optimize well in both dart web compilers
• Implement static analysis (errors/warnings/hints) to help use it correctly, and prevent using unsupported patterns
• Provide a convenient way of writing interface definitions (see firebase-dart for some examples--it's the best we can do with existing JS interop, but we can do better)
• Implement tools for automatic interface generation (e.g. from typescript files, web IDL)

Some of the things we'll need to address (details are subject to change):

• Support for renaming JS APIs (including instance members, see Allow @JS on class method #24779)
• Dart APIs can be exported to JS (retain them during tree-shaking and ensure they're available from a consistent location)
• Common Dart & JS types should interoperate well (e.g. Future/Promise, Iterables, Maps, Sets, DateTime, JS Objects used as maps through a JSMap type)
• When necessary, wrappers and coercions will be implicitly done (i.e. no hand written boilerplate for doing those), possibly guided by annotations in the interface definition (e.g. an annotation to convert a parameter from a Map to a raw JS object)
• Objects created in JS should be able to bypass Dart's reified generic checks (e.g. Arrays created in JS would not be subject to strict type checks, see @JS() doesn't support or fail on external List<NotDynamic> get #34195 for an example)
• JS interop types should not be subject to runtime cast failures (similar to the "anonymous" types currently supported)
• Ability to do JS dynamic dispatch distinct from Dart's dynamic (e.g. JSDynamic that dispatches directly to JS members)
• Dart classes should be able to extend a JS interop class (e.g. for custom elements), and vice versa if possible (probably opt-in from the Dart class, so it explicitly exports itself for JS subclassing)
• Ability to rename/hide JS APIs, including ones that start with "_"
• "extension method" APIs can be added to JS interop classes (note: also includes getters/setters)
• No metadata needs to be retained for JS types, and no RTTI needs to be computed at runtime

Other details:

• Fix this binding for tearoffs (DDC: JS-Interop tearoffs cause type errors #32370)
• Allow package:js to be imported on non-web platforms, but it can simply throw, similar to dart:io on the web (use Dart's conditional imports to implement it). This simplifies package dependencies/build rules.

Dynamic dispatch support:

We will likely want to keep opt-in dynamic dispatch support, similar to what "dart:html" currently provides, but well specified and available for use by any JS interop class. The classes must opt-in for this. This will cause extra data to be retained for use by runtime dispatch. Type checks are not required, however. The types will use the relaxed JS interop casting rules. Calling conventions in dart:html can likely be simplified (e.g. we do not require obj.foo(x) and obj.foo(x, null) to substitute undefined for null). Similar to JS Array and JS function types, tearoffs of JS methods will be untyped and not checked (they can be converted to any function type).

Because of these changes, we should be able to reduce the signature data stored for dart:html. These changes should be largely non-breaking, as code rarely relies on dart:html throwing type errors (indeed, such code will rarely work in production mode, which omits checks). It is also unlikely that any code is relying on dynamic tearoffs having a specific reified function type.

Note that any data for automatic conversions, or extension methods added to the types, will need to be stored so runtime dispatch can access it. Currently it is implementation defined how these methods are stored (dart2js uses interceptors , dartdevc adds symbolized members to the JS classes). If we want these methods to be accessible from JS (e.g. they are exported), we'll need to specify this precisely. A similar issue exists for the static extension methods (described above).

Longer term: we'd like to use the new JS interop to implement the DOM in a package, and then migrate from "dart:html". This will necessarily be a slow migration/deprecation process, since "dart:html" is necessary for all web apps today. It may involve automated refactoring tools or opt-in static analysis to assist the migration.

Due to these changes, migration to the new dart:html package should not be difficult (likely as simple as importing "package:html/html1.dart"). We will want a static-only version of dart:html, however (e.g. "package:html/html2.dart"). Types from those two HTML libraries should be compatible, so one can cast between them (after all, it is the same JS object underneath). At compile time, an explicit cast may be required (we could allow old types to be converted to new types, though, and you could always go in the other direction with as html1.Element because runtime casting will be allowed).

Ideally, we use re-export to declare the "dynamic version" of package:html:

// in file html1.dart
import "package:js/js.dart" as js;

@js.SupportsDynamicDispatch()
export "html2.dart";

That library simply indicates that dynamic dispatch data should be generated. If "html1.dart" is not imported, the runtime dispatch data would not be generated.

[jmesserly]

I've added a section about opt-in support for dynamic dispatch. Mainly this is intended to ease dart:html migration. But it's also a capability that we support in both dart4web compilers, so I believe we can offer it for broader use. It makes the interop a bit friendlier to use, at the cost of some extra runtime data. (probably not as much as dart:html currently retains, due to the elimination of checks for JS interop types)

[matanlurey]

Cool, thanks for adding that!

[jmesserly]

Very early draft (apologies, it may have lots of typos/bugs) is here: https://github.com/dart-lang/sdk/blob/js_interop/pkg/js/proposal.md ... I'm out of town for a week, but I'd like to do an edit pass when I get back.

[kevmoo]

@jmesserly – do you plan to merge that branch in? After feedback? Just curious...

[jmesserly]

do you plan to merge that branch in? After feedback? Just curious...

yes, there's a CL up for it, will send out after a bit more time to gather feedback

[eukreign]

@jmesserly any updates on this?

[jodinathan]

@jmesserly I have built a package that fully transpiles WebIDLs specs to Dart: https://pub.dev/packages/js_bindings.

Maybe it can help move this issue forward?

[vsmenon]

@srujzs is investigating this area right now.

[jodinathan]

@vsmenon

Thanks for the answer.

We are doing math here in what should be our next steps regarding Dart and web. Even if a step at all.
Our focus is basic document oriented web, so Flutter Web is not an option.

So could I kindly ask you what we can expect from Google efforts towards this issue and others related to HTML/JS interop?
It is ok to have a negative answer. After all, with FlutterWeb in stable channel it would be understandable the lack of resource to work with this.

However, it is important to the community to know what to expect.
There are many possibilities:
Dig dart2js and other packages to know how complex would be to fix those issues, generate a JS interop using dart:js instead of package:js, fork AngularDart and etc.

Can you clarify this please?

[srujzs]

Couple of comments because there's a lot of goals here. There have been several static checks added to ensure package:js is less thorny. There's also been work to fix some issues in the typing of package:js. I believe there's also been work to improving some of the components of package:js. I'm also not sure if we have a separate general overarching bug of what we're working on right now, so maybe updates are best suited for here currently.

Our current interop goal in order to resolve some of the above goals is to allow the use of a language feature in the shape of extension types (or views or zero-cost wrappers) for interop. You can see active discussion on Erik's proposal here: dart-lang/language#1426.

What this entails us to do is to treat JS objects as static extension types. In other words, we have some basic type for all JS objects which the user can then wrap with their own interface/extension type, similar to what we have today with package:js. The difference is that we won't have some of the limitations we have today in terms of annotations, only external members, or corner cases that come from using classes e.g.

extension type Foo on JavaScriptObject {
  external Promise get promise;
  Future future => promiseToFuture(promise);
}

  ...
  var foo = Foo(jsObj);
  await foo.future;

We don't do an underlying check on the JS type, so you can easily use some other wrapper on the same object. externals will be desugared into js_util calls which will then be optimized. This gets us much closer to the static dart:html which we want.

For dart:html, we want to move the classes we can to this new syntax and away from the @Native syntax. This will allow users to write their own interfaces for these objects (either with extension types or package:js). This also means that we can move to a dart:html that can be more easily contributed to and is much lighter-weight. There is still work to be done to figure out what classes we don't want to move away from @Native.

Still, we're thinking of ways that maybe we can avoid the collisions between interop and dart:html classes today. This is the issue that was being tracked in #39753.

So the following are what we are currently working on:

1. JS type relationships. What on-type we should expose, how it should behave with existing JS and native types, how reified generics work and should work, interceptors in DDC, and so forth.
2. Investigating usage.
3. Optimizing js_util calls such that they can be direct JS accesses/calls when possible.
4. Supporting external extension methods.
5. Identifying what classes can be moved away from @Native and potentially start migrating them to package:js + extension methods until we have extension types. This will unblock those who want to interface those classes.

Long term:

1. Figure out what to do to support mocking for these extension types.
2. Possibly a migration tool to use extension types instead of package:js to make the transition easier.
3. Migrate components of dart:html to use extension types. Users can then either modify these declarations or declare their own. Maybe an initial generation can be done based on TypeScript descriptor files.

What we haven't figured out yet or might not add support for currently:

1. Custom elements/web components.
2. Export Dart to JS besides what exists today in the form of allowInterop and such.
3. Dynamic dispatch.

I'm likely missing details, so I'll edit as I figure out what I missed.

[donny-dont]

Hi @srujzs glad to see someone working on this!

I had done a couple PRs against Jenny's original dart-custom-element-demo repository experimenting with getting Custom Elements up and running. I've dabbled some in trying to get an actual framework in Dart that's built on top of Web Components (I still hold onto the dream of Web Components in Dart and even used the web-ui package back in the day).

So rampage is my stab at making Web Components work in Dart. The example in rampage/web works in DDC and dartj2s.

In it there are more or less three layers.

The one closest to the "metal" is the js libraries. These are just extensions on JsObject which just expose the WebIDL definition for the dictionary or interface.

The highest level is just the interface. These are just abstract classes representing the hierarchy from the WebIDL definitions of the interfaces.

Between the two is the implementation. Primarily this just implements the interface by holding on to a JsObject and forwards the calls to the "js" layer. It also has the problem of wrapping and unwrapping the JsObject as necessary and making sure that two Dart Objects don't wrap the same JsObject.

Finally it also needs to provide for some quality of life features that are present in dart:html like getting a Stream for an event listener. Those are things that need to extend the interface outside of what the WebIDL exposes.

To actually do the Custom Element part there's a custom_element.js which proxies calls to the CustomElementsRegistry on window. An anonymous class gets created and captures the constructor for the Dart Object. When the actual JsObject calls the Dart constructor and assigns the Dart Object to a field on itself.

If I remember correctly I think this is the sort of stuff @sigmundch and @jakemac53 had to do back in the day with polymer.dart.

All the implementation in the library is hand rolled. I have a WebIDL parser that I am planning on using in conjunction with code_builder to generate these files now that I have a better handle on how things look.

I primarily wanted to give you another point of reference with this so maybe the Custom Element stuff could be figured out. Happy to assist in any way to help move this along quicker.

[a14n]

In google_maps package I already use extensions to dartify the underlying JS api and it works quite well. For instance LatLng is implemented like this:

@JS('google.maps.LatLng')
class LatLng {
  external LatLng(num? lat, num? lng, [bool? noWrap]);
  external bool? equals(LatLng? other);
  external String toString();
  external String? toUrlValue([num? precision]);
}
// `latLng.lat` in dart instead of `latLng.lat()` in JS
extension LatLng$Ext on LatLng {
  num get lat => _lat();
  num get lng => _lng();
  num _lat() => callMethod(this, 'lat', []);
  num _lng() => callMethod(this, 'lng', []);
}

What can be mapped directly to the JS api goes in the class and what need reshape goes in the extension.

Figure out what to do to support mocking for these extension types.

That has been a concern when google_maps_flutter migrated to google_maps-5.x (that used extensions)

If we go further with only extension types/views how will constructors be declared? Will it be possible to have static members?

[daniel-v]

Recently, we ran into wanting to quite a bit of JS interop, exposing business logic entrypoints to JS world. This issue was opened in 2018. Is this still on roadmap, is it likely to get some caring attention in the near future?

[srujzs]

@donny-dont, I believe the conversation around this is tied to #46248 so I'll keep it there instead. :)

---

If we go further with only extension types/views how will constructors be declared? Will it be possible to have static members?

The current plan is to enable factories (external or otherwise redirecting to a js_util call) in extension types/views. Yeah, like with extension members, I expect there to be static members in the extension type/views.

Mocking will be interesting, and something still in our roadmap to figure out with mockito. The current intuitive approach is mock the JS object itself and not the view, because the static semantics disallows the ability to mock the extension methods. What has been done with that project to work around static semantics?

---

Is this still on roadmap, is it likely to get some caring attention in the near future?

Yes, there's still ongoing effort here, but some of the goals have changed as I mention above. Are you specifically interested in exposing Dart classes to JS? This is a more complicated goal and one that would be in the further future after some of other interop goals mentioned above.

[WorikQCI]

I am wrestling with this, and I have a simple question: Is it possible to call Flutter/Dart code from JavaScript?

I have done a lot of digging and there is a lot of contradictory information, which is to be expected for a moving target like this.

I was sent here from https://dart.dev/web/js-interop which talks of Dart 2.19.

I am:

dart --version
Dart SDK version: 3.0.6 (stable) (Tue Jul 11 18:49:07 2023 +0000) on "macos_arm64"

so that document feels rather old.

Is it worth pursuing this as a developer or should I look for a different approach?

(I am successfully calling into Javascript from Dart)

[sigmundch]

I am wrestling with this, and I have a simple question: Is it possible to call Flutter/Dart code from JavaScript?

Yes! it's possible - but you have to explicitly export the API you want to call. Right now, that means storing callbacks (like you do with allowInterop today when you pass a callback to JS) somewhere where the JS code can discover them.

Recently @srujzs addded support for @JSExport and createDartExport to make it a bit simpler to export an entire object. You can find some documentation for it here: https://pub.dev/packages/js#jsexport-and-js_utilcreatedartexport

... so that document feels rather old.

That document is fairly recent. Note that 2.19 was the last SDK version before the jump to 3.0, so it's only 1 version ago :). That page doesn't discuss exports in detail, but it also has a link to the JSExport docs I mentioned above.

Hope you find this helpful!

[mit-mit]

We launched a first step toward this next-gen JS interop in Dart 3.3. For an introduction, see the blog post:
https://medium.com/dartlang/dart-3-3-325bf2bf6c13

We also have some documentation available:

• Low-level JS interop with static types
• package:web
• Migration from older JS interop

[sigmundch]

Indeed! This new release brings up to address many of the goals initially stated at the beginning of this issue. In particular:

• Fully specify JS interop, including all details

Yes! Some of the documentation linked by @mit-mit above provides more details. Today dart2js, ddc, and dart2wasm share a compiler pass that checks for valid uses of JS-interop APIs and provides compile-time errors for invalid usage. Some of it was added to old interop APIs, but going forward the main support lives with dart:js_interop.

• Fix incompatibilities between dart2js and dartdevc

Yes! Most inconsistencies were fixed by adding a shared checker to all compilers.

• Design it to optimize well in both dart web compilers

Yes! Some optimizations still need to be implemented, but the design supports it.

• Implement static analysis (errors/warnings/hints) to help use it correctly, and prevent using unsupported patterns

Yes! Same comment as earlier above.

• Provide a convenient way of writing interface definitions

With extension-types this has really improved!

• Implement tools for automatic interface generation (e.g. from typescript files, web IDL)

We have explored tools like js_facade_gen in the past, but recently some of our community members have created new alternatives like package:typings that really help here!

• Support for renaming JS APIs (including instance members, see Allow @JS on class method #24779)

Yes! This is fully supported now.

• Dart APIs can be exported to JS (retain them during tree-shaking and ensure they're available from a consistent location)

Yes. This still requires a step to progamatically expose them, but it is much easier than in the past. Check out createJSInteropWrapper

• Common Dart & JS types should interoperate well (e.g. Future/Promise, Iterables, Maps, Sets, DateTime, JS Objects used as maps through a JSMap type)

Many of these types now have direct conversion functions (e.g. Future/Promise).

• When necessary, wrappers and coercions will be implicitly done (i.e. no hand written boilerplate for doing those), possibly guided by annotations in the interface definition (e.g. an annotation to convert a parameter from a Map to a raw JS object)

Not yet. This was not prioritized. We only do implicit conversions for primitive values.

• Objects created in JS should be able to bypass Dart's reified generic checks (e.g. Arrays created in JS would not be subject to strict type checks, see @JS() doesn't support or fail on external List<NotDynamic> get #34195 for an example)

Yes, the new JS-types make clear how these types get interepreted in Dart.

• JS interop types should not be subject to runtime cast failures (similar to the "anonymous" types currently supported)

Check! The new JS-interop treats all JS objects as external entities.

• Ability to do JS dynamic dispatch distinct from Dart's dynamic (e.g. JSDynamic that dispatches directly to JS members)

This is now a non-goal.

• Dart classes should be able to extend a JS interop class (e.g. for custom elements), and vice versa if possible (probably opt-in from the Dart class, so it explicitly exports itself for JS subclassing)

We decided to take a different approach here, and not support mixing class hierarchies. Especially with the recent requirements to support Wasm.

• Ability to rename/hide JS APIs, including ones that start with "_"
  "extension method" APIs can be added to JS interop classes (note: also includes getters/setters)

Yes! This is now very natural to do with the extension types syntax.

• No metadata needs to be retained for JS types, and no RTTI needs to be computed at runtime

Yes! All interop types are now extension types, which are a compile-time only feature. THey all get erased to a simple type representing all interop objects.

• Fix this binding for tearoffs (DDC: JS-Interop tearoffs cause type errors #32370)

This was reevaluated. Instead, tearoffs of external members are statically banned. It is not as convenient, but was done to ensure consistency. In doing so, we prevent nuances with binding of this and avoid semantic changes with optional arguments being passed implicitly as null.

• Allow package:js to be imported on non-web platforms, but it can simply throw, similar to dart:io on the web (use Dart's conditional imports to implement it). This simplifies package dependencies/build rules.

This was treated as a non-goal now that conditional imports are part of the language and packages can hide uses of JS-interop in the same way they hide access to other web-specific APIs.

---

It's worth highlighting that the new Dart/JS interop introduced JS types to clearly denote the Dart/JS boundary in the type system, it is sound, and it is supported in Dart2wasm.

As such, I think it is fair to close this issue as complete 🎉

@srujzs - would you do the honors?

[donny-dont]

This is great stuff!

• Dart classes should be able to extend a JS interop class (e.g. for custom elements), and vice versa if possible (probably opt-in from the Dart class, so it explicitly exports itself for JS subclassing)

We decided to take a different approach here, and not support mixing class hierarchies. Especially with the recent requirements to support Wasm.

Just wanted to confirm with you both @sigmundch and @srujzs if one wanted to do Custom Elements you'd have to have a Dart object that wraps the js element. I've already gotten this to work with the new dart:js_interop just wasn't sure if this would be the way.

[sigmundch]

Just wanted to confirm with you both @sigmundch and @srujzs if one wanted to do Custom Elements you'd have to have a Dart object that wraps the js element. I've already gotten this to work with the new dart:js_interop just wasn't sure if this would be the way.

Hi @donny-dont! Indeed, I believe that's the best approach at the moment. It's possible that some of the new helpers (like @JSExport) could simplify some aspects of the process, but the general technique remains the same.

[hlynurl]

This is a great new feature and a step forward.

However what is currently missing is a concrete example of how to call a function in Dart from JS. All the examples are focused on Dart -> JS but not the other way around.

Is there any way to get a concrete example of how to call Dart code from JS?

[srujzs]

If I understand your ask correctly, you can convert Dart functions to a value that can be passed to JS using toJS (like allowInterop from the older dart:js_util) e.g.

@JS('some_library')
library;

import 'dart:js_interop';

// Some `external` member in order to pass the function to JS.
@JS()
external JSFunction jsFunction;

void printString(String s) {
  print(s);
}

void main() {
  // Has to be a function that only uses types that are allowed. `String` and `void` are okay.
  jsFunction = printString.toJS;
}

and then you can call it in JS:

  some_library.jsFunction('hello world');

There's some ongoing work to further document some of the conversions (as well as migrations from other forms of interop) that should make operations like the above a bit more obvious. Let me know if you still have more questions.