Extension types page

[MaryaBelanger]

Fixes #4177

---

The first commit includes a "Use cases" section. I removed this from the second commit as I felt like the information covered there is more generally covered across the rest of the page where syntax and usage are discussed.

But, if anyone thinks the use cases are valuable I am happy to include them (I didn't fully flesh out those sections because I was leaning towards removing them already, but if we want to keep them I will improve/refine them)

Link to page: https://dart-dev--pr5508-extension-types-6op3ayna.web.app/language/extension-types

---

TODO:

• After some offline feedback from @domesticmouse, justify the functionality so readers aren't left asking "why?"

  • make it clearer up front that this feature's main purpose is enabling JS interop,
  • maybe use interop code as the top sample / maybe even have all the samples in a JS context?

• Maybe take the "Use cases" from the first commit and write a blog post?

  • Erik also gave me an interesting scenario about wrapping objects from a typeless tree data structure and the performance benefits extension types would have, which could be another interesting scenario to pad out the blog post

[dart-github-bot]

Visit the preview URL for this PR (updated for commit 7558e2d):

https://dart-dev--pr5508-extension-types-6op3ayna.web.app

[lrhn]

Maybe also show:

  int i = 2;
  if (i is NumberE) print("It is");
  if (i case NumberE v) print("value: ${v.value}");
  switch (i) {
    case NumberE(:var value): print("value: $value");
  }

which shows that the static type of the matched value is NumberE here.

[MaryaBelanger]

I added a lead-in to this example saying:

Similarly, the static type of the matched value is that of the extension type
in this example:

But I don't think that's makes sense and I don't really understand the example. What are we showing here? Is it just another way to show that NumberE is int at run time? It doesn't seem like "the static type of the matched value (is that i?) is NumberE" to me, but rather that the runtime type of NumberE is int

[eernstg]

Looks great! I added a bunch of comments.

[atsansone]

LGTM!

Would recommend some language refactoring, but nothing to block this great piece of work!

[eernstg]

make it clearer up front that this feature's main purpose is enabling JS interop

This characterization of the feature is limiting in several ways.

It is true that JS interop is a use case that has been on the agenda for a long time. However, the core nature of the extension type mechanism is that it allows the interface of an existing type to be modified (with "extend the interface" and "replace the interface" as important special cases), without incurring the cost of an actual wrapper object, and that's much broader than JS interop.

With JS interop, this means that we can perform native JS method invocations using standard Dart syntax and static type checking, and a similar description would fit other kinds of interop. With JS interop, almost all member declarations are external, and they are given special treatment by the compiler (such that we get the effect of performing a native JS invocation at each call site). You could say that each external member of a JS interop extension type is syntactic sugar for the low-level code that performs the native JS invocation, plus an assumption that every invocation will be inlined. In any case, JS interop uses extension types in a rather specialized manner, and they don't show many of the elements of the feature. For example, members almost never have a body, and there are few if any references to the representation object. That's basically unthinkable for an extension type that doesn't get special treatment by the compiler, and this means that JS interop may not be the perfect domain for introductory examples. It's still fine to mention JS interop as an important use case, but it is probably not important to explain the intricacies of JS-interop-extension-types here, as opposed to extension types in general.

Another use case sets out from a weakly typed object structure (say, a combination of Map<String, dynamic>, List<dynamic>, String, etc., created by a JSON parser). We can then use a schema describing the expected structure of that weakly typed data to create extension types (probably using code generation, but it could of course also be written manually, especially when the schema is simple). This would allow us to navigate any of these weakly typed object graphs safely, as long as they follow the schema. This use case does not rely on special treatment by the compiler, but an example might need to be somewhat large in order to make sense.

A very different case is 'newtype', where we introduce a static type distinction between values with the same representation. For example, we may wish to distinguish a value of type Centimeter from a value of type Inch, yet using a plain int as the representation. The IdNumber example is also in this category: The purpose is, among other things, to make sure that we don't consider an arbitrary int to be an IdNumber, and vice versa.

Another example is union types. This repo shows how extension types can be used to emulate the core properties of union types. The emulation is not nearly as convenient as a real language mechanism. Consider, say, the union of the types int and String. The emulation of int | String isn't equal to String | int, and there's a need to taken an explicit step in order to obtain an expression of type int | String from an expression of type int. However, this emulation will allow a developer to define an API where a method accepts an argument of type int or String, reporting a compile-time error for all other types, and it can be done today using extension types.

Another example is that it can be used to modify the signature of an existing method. The method Future.then accepts an argument named onError whose type is Function. This allows the onError argument to be any function whatsoever, which is not safe (it is actually required to be something much more specific, but something which cannot be expressed as a Dart type). So the following small program fails at run time (because print certainly won't work):

void main() async {
  var fut = Future<int>.error("Failed!");
  await fut.then((_) => 42, onError: print); // No compile-time error; throws.
}

Here's a variant that uses an extension type to make the signature of Future.then safer:

import 'safe_future.dart';

void main() async {
  var fut = Future<int>.error("Failed!");
  // await fut.then((_) => 42, onError: print); // Compile-time error.
  int i = await fut.then((_) => 42, onError: (o) => 24); // OK!
}

As you can see, the unsafe invocation is now a compile-time error. The extension type looks as follows (of course, it's just an example, we'd do several additional things in order to make it realistic):

// ----- Library 'safe_future.dart'.
import 'dart:async' hide Future;
import 'dart:async' as async show Future;

extension type Future<T>._(async.Future<T> _it) implements async.Future<T> {
  Future.error(Object error, [StackTrace? stackTrace])
      : this._(async.Future.error(error, stackTrace));
  // Other constructors of `Future` are replicated similarly.
  
  Future<R> then<R>(
    FutureOr<R> onValue(T value), {
    T Function(Object)? onError,
  }) =>
      Future._(_it.then(onValue, onError: onError));

  Future<R> thenWithStack<R>(
    FutureOr<R> onValue(T value), {
    required T Function(Object, StackTrace) onError,
  }) =>
      Future._(_it.then(onValue, onError: onError));
}

Yet another example is that it can be used to emulate invariance of type variables. Due to dynamically checked covariance, we can easily write an example where there are no compile-time errors, and yet there's a run-time type error:

void main() {
  List<int> xs = [1];
  xs.add(1); // OK at compile time and run time.
  List<num> ys = xs; // Also OK all the way.
  ys.add(1.5); // OK at compile time, throws a type error at run time!
}

We could add statically checked variance (dart-lang/language#524) to the language, but we can also handle the issue today using an extension type:

// Library 'safe_list.dart'.
import 'dart:core' as core show List;
import 'dart:core' hide List;

typedef Inv<X> = X Function(X);
typedef List<X> = _List<X, Inv<X>>;

extension type const _List<X, I extends Inv<X>>(core.List<X> _it)
    implements core.List<X> {}

// Library 'my_library.dart'.
import 'safe_list.dart';

void main() {
  List<int> xs = List([1]);
  xs.add(1); // OK at compile time and run time.
  List<num> ys = xs; // Compile-time error.
  ys.add(1.5); // Irrelevant, declaration of `ys` is an error.
}

This illustrates yet again that extension types can be used to make adjustments to the given static typing properties of existing types, like a small type system extension engine. We may well wish to add some of those enhancements of the type system as proper language features, but surely some such cases will be application domain specific or narrow in scope, in which case they will probably never be supported as language mechanisms. In any case, it does matter that we can start using an emulation of such type system enhancements already today.

Extension types is a language mechanism, and it can be used for many purposes. I'm sure we will develop more ways to use it over time, just like any other language mechanism worth its salt. It shouldn't be impossible to answer the 'why' question.

[eernstg]

Added a long comment in order to hint at answers to the 'why' question.

[kevmoo]

Ugh! I hate to pile on last minute.

Should we discuss private constructors here?

extension type MyConsole._(JSObject thing) implements JSObject {
  static MyConsole get instance => MyConsole._(globalContext['console'] as JSObject);
  
  external void log(JSAny? value);
  external void debug(JSAny? value);
  external void info(JSAny? value);
  external void warn(JSAny? value);
}

[MaryaBelanger]

Should we discuss private constructors here?

@kevmoo Would that be better discussed in the JS docs, maybe under Usage > Interop type members > Constructors (bullet)

[kevmoo]

The fact that one can COMPLETELY hide the constructor (instead of just defining a new one) is...interesting, I think. Like the case I provide. It doesn't have to be in the context of JS interop.

[MaryaBelanger]

The fact that one can COMPLETELY hide the constructor (instead of just defining a new one) is...interesting, I think. Like the case I provide. It doesn't have to be in the context of JS interop.

Oh, interesting! I'll push my current WIP commit and then see about adding something about that (I wasn't aware of the concept until now). At the very least I'll add it in a follow up PR if this one gets down to the wire

Edit: @kevmoo I added it in 92d57ca, but I used your simpler example, not the JS interop one. Not for any major reason, just because it was clearer for me. If you think it should really be the JS interop one, just lmk and I'll change it!

[MaryaBelanger]

This characterization of the feature is limiting in several ways.

@eernstg Thanks for that illuminating write up. I've mentioned before I am hoping to write a blog post that dives into all these interesting use cases for extension types, and this is great information for that!

I had added a sentence saying that extension types were "specifically implemented to enable static JS interop", but I've changed that now to better emphasize that they enable static js interop because of beneficial characteristics.

Also, I unfortunately might have to merge this before you'll have time for another look, but please rereview the most recent version as I made a lot of changes based on the feedback! Thank you so much.