Static Extension Types

[lrhn]

Solution for #40. This is not a full proposal, just a primer for discussion.

Scoped static extension methods (#41) are useful, but risk naming conflicts and makes it hard to see where a function comes from.

An alternative is to not add the methods directly on the existing type, but instead introduce a new type alias for the original type, and then only attach the new methods to that alias.
The alias can expose the original object's members, or it can hide them and expose other methods instead.

Example:

typedef Stringy extends String {
  String double() => this + this;
}

This declares a new type alias Stringy which extends the type String with a new method double.
Any String value can be cast to and from Stringy.
Example:

Stringy x = "42";
print(x.double());

Just like extension methods, the methods are really static, just with a special binding for this.
The Stringy type does not exist at run-time. Effectively it's like normal static extension methods that are only declared on static-only type aliases.

This prevents naming conflict, the user picks the extensions that they want on a case-by-case basis. It's entirely opt-in, there is no effect-at-a-distance where one import can add extension methods to another import's class - you have to choose to use the extension.

If the static extension type wants to replace functionality, it can choose to not expose the original object's API:

typedef Vector on Point<num> {  // not "extends"
  Vector operator+(Vector other) => Point(super.x + other.x, super.y + other.y);
  Vector operator*(double scale) => Point(super.x * scale, super.y * scale);
  double get magnitude => math.sqrt(super.x * super.x + super.y * super.y);
  double get angle => ...
}

This provides a different view on an existing object. It uses super to access the member of the adapted object, rather the vector's own API.

Maybe we can add multiple on constraints, like on the mixin declaration. Then using this type will be similar to using a statically resolved mixin application.

It's possible to add constructors like:

typedef Vector on Point<num> {
  Vector(double angle, double magnitude) => 
      Point(math.cos(angle) * magnitude, math.sin(angle) * magnitude);
  ...

(which would disable the default assignability from Point to Vector?)

Again, we the typedef can be generic:

typedef Vector<T extends num> on Point<T> { 
  ...
}

and if we can deconstruct types, it might also apply here:

typedef Vector on Point<var T> { 
  ...
}

giving the extension methods access to the reified generic type argument of the receiver

Static extension types have a drawback compared to extension methods like #41.
In a chain of operations, you need to do an explicit cast to enable the new functionality, e.g.:

(foo.something as HelperType).somethingElse

where the extension method can be put directly on the type returned by foo.something.

Static extension types like these may be useful as simple data types if Dart adds simple tuple types. Then we could declare something like:

typedef complex on (num, num) {
  num get r => super[0];
  num get i => super[1];
  complex operator+(complex other) => (r + other.r, i + other.i);
  complex operator*(complex other) => (r * other.r - i * other.i, r * other.i, i * other.r);
}

This is a simple structural data-type with extra functionality, implemented on top of tuples and static extension types.

[zoechi]

This prevents naming conflict, the user picks the extensions that they want on a case-by-case basis. It's entirely opt-in, there is no effect-at-a-distance where one import can add extension methods to another import's class

wouldn't show/hide be enough for that already?

[lrhn]

It should definitely be possible to hide extension methods (hide String.foo), but that only works if you are aware of it. Since extension methods can shadow instance methods, it's possible to change behavior of an invocation without anyone noticing (until things crash). It also doesn't allow you to use both extensions in the same library, just different ones at different times, since a hide/show would affect the entire library.

[zoechi]

but that only works if you are aware of it.

That's one of the things I like about #43 because it's more explicit

extension methods can shadow instance methods

I haven't seen anything in the proposal that would hint at that.
Using import as prefixes is probably not easy to make useable in a good way.
That would be another advantage of #43

It also doesn't allow you to use both extensions in the same library

also dependent on being able to use prefixes

This extension types approach seems to introduce a lot of new concepts not too easy to understand.

[lrhn]

I personally think extension types introduce fewer concepts than extension methods. The methods are the same, but the resolution process is much simpler. (Especially if we assume that we already have type aliases anyway, like typedef StringList = List<String>;, then this is just type aliases with extra static extension methods).

I have added text to #41 say that there is potential conflict between instance members and static extension methods, and that I think the sanest way to resolve it is to always pick the extension method. And that prefixes do nothing. There is no syntactic place to use them.

[ds84182]

I like the thought of static extension types more than static extension methods and abusing the pipeline operator for various reasons.

Static extension methods

Static extension methods in Kotlin work great because of explicit imports AND function overloads. But a large problem is discoverability, and an even larger problem is libraries that feel that its necessary to add extension methods to all types, which gets annoying during auto completion.

Pipeline operator

While I see the pipeline operator being used for cleanly composing operations, I don't see it as a clean way to add extended functionality to an object, and it's limiting in certain aspects (for instance, how do I define an operator?).

Static extension types

Reminds me of Kotlin's inline class, which has static dispatch on "virtual types" and avoids creating expensive wrappers for primitive types.

[munificent]

An alternative is to not add the methods directly on the existing type, but instead introduce a new type alias for the original type, and then only attach the new methods to that alias.

I think this a major flaw, unfortunately. In your motivating example, you say:

Example: helper(foo.something).somethingElse is harder to read and understand the sequencing of than foo.something.helper().somethingElse.

But, unless you control foo.something, this proposal doesn't enable the latter syntax. You instead have to do:

HelperThing helper = foo.something;
helper.somethingElse;

That's even worse than using a top level function. You could maybe use as:

(foo.something as HelperThing).somethingElse;

But, again, it's not clear that that's an improvement over a function.

Method chains are incredibly common, and this problem area is directly about allowing more operations to use method chain syntax. So an extension method system that doesn't let you hang off an existing method chain that calls methods you don't control is going to be really limited.

When I was programming in C#, extension methods on IEnumerable were some of the most common ones I defined (example). It would have been maddening if I couldn't call those methods on the result of other methods that return IEnumerable, including the many higher-order sequence transformation methods.

[leafpetersen]

Solution for #40. This is not a full proposal, just a primer for discussion.

I think this is a partial (but not complete as @munificent observes) solution for #40, but I don't think #40 is the main (or at least not the only) problem that this mechanism would target. The main thing this gives you is a way of producing:

• new types with additional functionality, defined in terms of other types
• that don't incur a representation overhead
• and that don't have all the problems that subclassing gives you (name clashes, etc)

In other words, I don't think there's necessarily anything (or much) that you can do with this that you can't do with delegation already in Dart. But it gives you a much more convenient syntax for it, and a much more efficient (lightweight) representation. In turn, you give up various capabilities that you would get with the class wrapper/delegation approach:

• subclass/implement/mixin
• dynamic dispatch
• strong encapsulation

[lrhn]

I actually do think

(foo.something as HelperThing).somethingElse;

is better than the static helper function. It preserves "flow-order", which is also the thought-order I use when understanding what is going on.
That said, both cast and await are annoyances in that respect, and I'd love to have properly strongly-associating suffix versions of those. Perhaps foo.something.as<HelperThing>.somethingElse.await (or some other syntax ... foo.something@HelperThing.something! to go wild :).

I've added the "needs a cast" caveat to the initial post.

[willlarche]

I don't prefer this for 2 reasons:

1. It needs a cast which is intimidating for the person consuming the API.
2. It's essentially light-weight subclassing which is the annoyance I wanted to avoid. I'd like my users to not have to learn new typedefs in addition to the new methods.

[yjbanov]

This looks close to Golang's defined types but without the same level of utility. Go's defined types get the efficiency from compact object representation and create a new namespace for adding extra methods on top of the underlying types, but they are still distinct from their underlying types (e.g. you can do a runtime type check is Stringy).

I think #41 is a better solution for #40. However, I wouldn't say "no" to defined types in Dart, for example for units, such as pixel, meter, Celsius, without the overhead of wrapping objects within objects.

[jmesserly]

For what it's worth: in JS interop, I can see both #41 and this proposal (#42) being very useful.

What excites me about this proposal is the typedef ... on ... syntax. Being able to replace the API would give users a much nicer means of expressing renames/hides of JS members, rather than using a bunch of JS interop annotations.

As others noted, I feel like the two proposals attack two different and valid use cases. Extension members are perfect for adding helpers, and they're great for having an interface (e.g. Iterable) and getting lots of useful functionality "for free" in an extensible way. They're also easy to use: no need to change static types around, break up method chains, or learn new type names and how they relate to each other. Just import the extensions into your library, and you're good to go!

Extension types seem excellent for reshaping APIs. That's one thing extension methods don't do well: they can only add things. With this proposal you can remove and rename things too.

BTW, one thing I don't especially like:

(foo.something as HelperThing).somethingElse;

When I see as, I generally assume there's a runtime cast, and try to avoid those when I can. It's a bit unfortunate that these casts are totally safe and compile-time checked, but they look like they could be unsafe. Depending on the type returned by foo.something, it might actually be unsafe, but as a reader I have no way of knowing without jumping to the definition of foo.something and HelperThing. Can we make that better, somehow?

[ds84182]

Instead of a cast, what about a constructor, like Kotlin's inline classes?

From Kotlin's docs on Inline Classes:

inline class Name(val s: String) {
    val length: Int
        get() = s.length

    fun greet() {
        println("Hello, $s")
    }
}    

fun main() {
    val name = Name("Kotlin")
    name.greet() // method `greet` is called as a static method
    println(name.length) // property getter is called as a static method
}

Makes it easier to convert actual uses of "extension classes" to extension types in the Future.

[lrhn]

I am actually considering constructors (and perhaps even deconstructors) for extension types.

Extension types are new nominal types with the same underlying implementation as an existing type.
The current proposal assumes assignability between the extension type and its representation type.

That is both simple and useful. You can change only the static type of a variable and get all the extra features of MySuperList<T> extends List<T>.

However, it does not allow you to restrict assignability or do any conversion.

So, we can also allow you to declare "constructors" on the extension type. Those are necessarily factory constructors, and they must return the representation type. Example:

class Point<T> { final T x, y; const Point(this.x, this.y); }

typedef Complex on Point<num> {
  Complex(num r, num c) => Point(r, c);
  Complex.angular(num theta, num r) => Point(r * cos(theta), r * sin(theta));
  num get r => super.x;
  num get c => super.y;
  Complex operator +(Complex other) => Complex(r + other.r, c + other.c);
  Complex operator *(Complex other) => Complex(r * other.r - c * other.c, r * other.c + c * other.r);
  Point<num> toPoint() => super;
}

If an extension type has a constructor, then there is no automatic assignability in either direction. You always have to add use the constructors to create an "instance" of the extension type. The constructor can then validate inputs or convert them.

Is it too strict to disallow assignment if there is any constructor?
Should we allow it anyway if there is an unnamed constructor taking exactly the representation type as argument (which would then be the default constructor added if there is no constructor)?

What about destructing/destructuring/projecting the representation back out then? If we allow assignability in one direction, it's hard to not allow it in the other direction.

I do want the simple assignability. I also do want the constructors. I'm not sure they should be mutually exclusive.

Maybe this ability should not be guided by constructors, but by a manual flag:

sealed typedef Complex on Point<num> { .... }

means there is not assignability between Complex and Point<num> (since "sealed" doesn't mean anything for non-classes anyway, even if we add it).

WDYT?

[ds84182]

I like the idea of sealed! Here's an example I can think of from one of my projects:

sealed typedef FNV32Hash on int {
  FNV32Hash.fromRaw(int hash) => hash & 0xFFFFFFFF;
  FNV32Hash.of(String str) => ...;
  int get raw => super;
  // Interesting issue, is this valid? Can you override Object methods?
  String toString() => "FNV32Hash: " + super.toRadixString(16).padLeft('0', 8);
}

sealed typedef ClassKey on int {
  ClassKey.fromRaw(int hash) => hash & 0xFFFFFFFF;
  ClassKey.fromFNV32(FNV32Hash hash) => hash.raw;
  ClassKey.of(String str) => FNV32Hash.of(str).raw;
  int get raw => super;
  String toString() => "ClassKey: " + super.toRadixString(16).padLeft('0', 8);
}

// C+P ClassKey for CollectionKey, etc.

Several questions I can think of:

• Can you override Object members like toString, hashcode, and operator==?
• Can typedefs share common code? For the example above I'd like to share the implementations of raw and toString to reduce code duplication.
• Can you create a typedef on another typedef? So typedef ClassKey on FNV32Hash?
• If you have a container of, for example, int (like a Uint32List), can you cast that to a List<FNV32Hash>, even if it's sealed? Or would you have to make another typedef for it (typedef FNV32List on List<int>)?
• If you wanted to provide a specific interface, could you do typedef FNV32List on List<int> implements List<FNV32Hash>?

[Hixie]

I'm not sure sealed is the right word for this, but I do agree that it would be nice to be able to make new types in this way. One question would be around what "runtimeType" does for these types. Some classes use the runtimeType (e.g. it's pretty much required to use it in operator ==), and it's not clear to me what it would mean in these new types.

FWIW, modern Pascal variants support this kind of thing specifically for what we would call "value types", like int or structs (record in Pascal). You can declare a type as being another type in two ways:

type
   TFoo = TBar; // essentially an alias, useful for generics and the like
   TBaz = type TBar; // a new type that happens to be identical to the other

This is useful e.g. to have a different type for "meters" and for "inches", or for "celsius" and for "fahrenheit", so that you can't accidentally assign one to the other. (You can still cast from one to the other, which is essentially free since it's just a way to disable the type checking.)

[leafpetersen]

One question would be around what "runtimeType" does for these types. Some classes use the runtimeType (e.g. it's pretty much required to use it in operator ==), and it's not clear to me what it would mean in these new types.

Yes, this is something we've been thinking about. One of my goals for this class of features is to provide some way to support close to zero-cost abstractions, which means no boxing. That is, I can say something like:

typedef Idx on int {... }

and be confident that the representation of Idx is a flat int, no wrapper object. However, as you note, this means that if I allow you to assign this to a variable of type Object and then ask for its .runtimeType, then you will get int.

• One possibility is to define these to autobox, but it's not clear that gives you much benefit over just saying that they are always boxed and the compiler can try to unbox them (at least if you give them the right identity semantics).
• Another possibility is to put it under the programmer's control: you can choose to give them a boxed semantics in which case you get a wrapped object which has a nominal type and possibly even a vtable, but you can also choose to make them unboxed.

[rrousselGit]

I don't quite see how this is related to #40. More specifically the:

helper(foo.something).somethingElse

as this proposal requires a casting, which makes the syntax worse.

---

This ultimately is syntax sugar over wrapper classes, such as this one:

class Foo implements Bar {
   final Bar _wrapped;

   Foo(int value): _wrapped = Bar(value);

   int get value => _wrapped.value;

   void newFeature() {
      // TODO: do something cool
   }
}

Which can already be solved using mixins shorthand syntax:

mixin _Foo on Bar {
  void newFeature() {
    // TODO: do something cool
  }
}

class Foo = Bar with _Foo;

---

Maybe a good middleground would be to enhance that syntax sugar with the ability to override some things without having to rewrite the entire class:

class Foo = Bar with Mixin {
  @override
  void methodOverrideExample() { }
}

[lrhn]

This is indeed syntactic sugar for something equivalent to wrapper classes. The idea is that it is based entirely on static types, so you don't need to allocate a wrapper object. That makes it possible to implement it as a low-cost aliasing operation rather than an actual wrapper class. Even if it is implemented as a wrapper class, it provides a way to easily define such a wrapper class without having to forward all the methods.

typedef Lister<E> extends List<E> {
  Lister<E> repeat(int count) => List<E>.generate(this.length * count, (i) => this[i%this.length]);
  // Forwards all List methods to super.
}

It is, however, not the same as a mixin. A mixin creates a new class with the original class as super-class. It doesn't allow you to wrap existing objects, only create new objects. Wrappers allow you to dynamically wrap an existing object that was created by someone oblivious to your type.

[rrousselGit]

I'd say this is covered by #65
Combined with #41

And then mixins makes up for the sealed version

[lrhn]

The more I think about this, it seems to me that there are two different features to consider:

Transparent Extension Types: A type alias for another type wrt. subtyping, but with different members. Only exists at compile-time, it is entirely static. Basically we introduce an extension type, which is not an interface type. We treat member access on that type differently from member lookups on interface types. The extension type is otherwise equivalent to its on type (sub- and super-type of it), and is therefore assignable in both directions. At run-time, the extension type is erased and becomes the on type.
This means that you can cast a List<String> to and from List<MyString>. The only difference is that list[0].myMethod works on the latter.

We could even use the existing extension method definition as the transparent extension type definition. The extension name becomes the extension type name.

Opaque Extension Types: Introduces a new type which is completely unrelated to all other types except Object. You cannot cast to the extension type (no downcast from Object!) because the type may have constraints that are not type based. The only way to create an instance of the extension type is to use a constructor. The constructor must create an object of the on type, and designate it as the value. Maybe as simple as extension class Foo on int { Foo(int x) : super(x); }, but the syntax is an open question. Basically, we guard entry into the type by requireing it to go through a constructor, allowing us to ensure some invariants, but since we don't actually have a run-time representation on the object, we have to prevent any reinterpreting casts, otherwise we cannot ensure the invariants.
You can cast the extesion type to the on type by casting through Object, but not vice versa, because you can't cast from Object to the extension type.
The type is reified, so a List<MyString> is unrelated to List<String> at run-time.
Maybe you can cast it to List<Object>, but that is problematic because it requires the add method to do a MyString check, which will always fail (but we don't want that check to fail when we use the list as List<MyString>.
If the opaque extension type is generic, it must probably be invariant.
The type cannot have subtypes, so the only cast to be worried about is from Object.
It might not even be nullable, unless we require the on type to be non-nullable.

(Definitely some unclear issues here).

[Cat-sushi]

What is the benefit of transparency extension types?
In general, subtypes of existing types can be defined.
Of course, int can't be extended, then should we make it open/ unsealed class?
If we shouldn't, typedef MyInt extends int is acceptable for other reasons?
You said "it is entirely static", but isn't mutual assignability at runtime by default confusing?

[Cat-sushi]

I guess, int has a special internal representation for performance reasons, so we CAN'T extend it with additional field.
It is the reason why we must have separate syntax from subtyping.
Am I correct?
Is prohibiting additional field to subclass of int insufficient?

[eernstg]

@Cat-sushi wrote:

What is the benefit of transparency extension types?

Transparent extension types allow us to impose a special discipline on a given object graph. The typical example could be a Map<String, dynamic> and a tree of objects reachable from there which are intended to model a JSON value with a specific schema. The Dart type system doesn't know it, but that tree is highly regular (assuming that it's not buggy), and we'd expect to find specific kinds of objects at specific locations in the tree, and we could use that to navigate into the JSON value using conceptually meaningful names, and with a reasonable level of type safety.

You cannot "declare a subtype" of such an object graph that contains a similar level of discipline. You could transform the whole object graph to a tree of typed Dart objects, but that's much more costly in terms of time and space.

[lrhn]

You can't implement int because it's a system type that we need to pass to the underlying system. We can't send your int subclass to a native OS function, but we know how to handle the platform integers. We'd need to call toInt on your int to get a real int? Not going to work.

You can't extend int because it's so highly optimized that not all integers are real objects. So yes, internal representation would make it harder, but we could probably provide a default implementation that can be extended. The real problem is that all our integer optimizations would fail when a non-native integer value appears, which would make things much slower.
(We'd also have to update the spec to say that constant operations on integers are only valid for native integers.)

Creating a subtype of int by extension type is potentially viable if all the values are actual integers. All our run-time optimizations are still valid. So,

extension EvenInt on int {
  EvenInt(int value) => value.isEven ? value : throw ArgumentError.value(value, "value", "Is not even");
  EvenInt._unverified(int value) => value;
  EvenInt operator +(EvenInt other) => EvenInt._unverified(this + other.toInt());
  EvenInt operator -(EvenInt other) => EvenInt._unverified(this - other.toInt());
  EvenInt operator *(EvenInt other) => EvenInt._unverified(this * other.toInt());
  int toInt() => this;
}

This could be a valid subtype of int.

[leafpetersen]

I think this is covered by extension types.