Implicit constructors in receiver/wrapper classes

[eernstg]

In response to #107 and #40, and as a foundation for solutions similar to #41 and #42, this issue proposes implicit constructors in a class which is either a receiver class or a wrapper class. The modifier receiver respectively wrapper on the class has just one effect: It controls the situations where an implicit constructor can be invoked.

An implicit constructor must be a generative constructor. It only differs from other generative constructors by having the modifier implicit.

The difference between implicit constructors and other constructors only arises at locations where they are invoked: Constructors must in general be denoted explicitly (e.g., C() or C.name() may create a new instance of the class C, and the syntax explicitly allows us to look up the constructor named C respectively C.name and see that this will create a new object). Implicit constructors can also be invoked explicitly.

However, an implicit constructor invocation can also arise because a certain situation exists for an expression e, and that expression is then transformed into an expression e2 which differs from e by invoking an implicit constructor with e as an argument. For instance e.foo() might be transformed into C<int>.name(e).foo() where C.name is an implicit constructor.

The experience from Scala suggests that it is prudent to be cautious when introducing a mechanism that is capable of implicitly transforming objects of one type into objects of another type (see, for instance, this comment). With that in mind, this proposal only allows for creation of new objects by wrapping an existing object in a new one, in the sense that the existing object is passed as a constructor argument when the new one is created (and it would be surprising if it were ever useful to ignore that argument, so it's probably going to get "wrapped" in the new object in some sense).

However, the basic idea is similar: During static analysis it may be the case that an expression e of type T is used in some context where an instance of T cannot be used, and e may then be implicitly replaced by an instance creation C<T1..Tk>(e) which will work in that context.

One such situation is simply when e occurs in a context where a type S is expected; this situation is handled with a wrapper class, as described below, and in this situation C<T1..Tk>(e) is a subtype of S. The notion of a wrapper class can be considered as a foundation for (and generalization of) the notion of static extension types.

Another situation is when e is used for a member access, like e.m(...), but the static type T of e has no member named m. In that situation we may again replace this expression by C<T1..Tk>(e).m(...), where C does have a member named m. This can occur when the class C is a receiver class. The notion of receiver classes can be considered to be a foundation for (and generalization of) the notion of static extension methods.

There may be additional invocation criteria of interest, but this proposal starts off focusing on just these two: Wrapper classes and receiver classes.

The two proposals in the following can trivially be combined, this just amounts to allowing both class modifiers in the same program, and potentially on the same class. All rules about such classes will coexist without conflict.

Note that these proposals include the static class proposal (#308), because it is important for a receiver class and for a wrapper class to be able to be static, in order to ensure that extension methods can be invoked with just the cost of a top-level function invocation, and a static extension type can be used on a local variable without allocating a wrapper object for the target.

Proposal, Shared Part

Syntax

The grammar is adjusted as follows:

<constructorSignature> ::=
    'implicit'? <constructorName> <formalParameterList>

The only change is that it is possible to use the modifier implicit on a constructor.

Static Analysis

An implicit constructor is a constructor whose declaration includes the modifier implicit.

It is a compile-time error for a constructor to be implicit, unless it is a generative constructor.

It is a compile-time error for a generative constructor to be implicit, except in the cases that are explicitly allowed in the proposals below.

The static analysis of an implicit constructor proceeds identically to the static analysis of the same constructor with no implicit modifier.

An implicit constructor only differs from the same constructor without implicit by being invoked implicitly in certain locations in code. This is specified as a source code transformation, and the normal static analysis is then applied to the transformed code.

Dynamic Semantics

Implicit constructors have the same dynamic semantics as other constructors.

Again, call sites may be affected by the source code transformation mentioned above, but the dynamic semantics is unchanged when considering the code after this transformation.

Proposal for Wrapper classes

Syntax

The grammar is adjusted as follows:

<classDeclaration> ::= // Modified
    'abstract'? 'static'? 'wrapper'? 'class' <typeApplication>
    (<superclass> <mixins>?)? <interfaces>?
    '{' (<metadata> <classMemberDefinition>)* '}'
  | 'abstract'? 'static'? 'wrapper'? 'class' <mixinApplicationClass>

The only change is that the modifier wrapper is allowed on a class declaration.

Static Analysis

A wrapper class is a class whose declaration includes the modifier wrapper.

The static analysis of a wrapper class is identical to the static analysis of other classes.

Expressions in the program are affected by the existence of wrapper classes:

Consider an expression e with static type T that occurs with context type S, and assume that T is not assignable to S (without this proposal, that is a compile-time error).

Assume that the set of wrapper classes in scope is C1 .. Ck. Let c1 .. cm be the implicit constructors of C1 .. Ck accepting one positional argument (which means that cj is of the form SomeClass or SomeClass.someName for each j in 1 .. m), and assume that inference on cj(e) with context type S succeeds and yields type arguments Ujs (that is a list of actual type arguments, and the type list U1s may have a different length than the type list U2s, etc), for j in 1 .. n.

There may be fewer than m of these (because inference and/or type checking fails with some constructors), hence we go up to n rather than m. We assume that the constructors have been ordered such that the failing ones are the ones with the highest numbers.

If exactly one of cj<Ujs> is such that the type of its parameter is a subtype of all of those of c1<U1s> .. cn<Uns> then e is replaced by cj<Ujs>(e). Otherwise a compile-time error occurs.

That is, it is an error whenever multiple wrapper classes can be used (or even just multiple constructors from the same wrapper class), but none of them gives the target a "better type" than all the others.

Example

class A {}

wrapper class C1<X extends num> implements A {
  final Iterable<X> target;
  implicit C1(this.target);
}

wrapper class C2 implements A {
  final List<num> target;
  implicit C2(this.target);
}


main() {
  List<num> xs = [];
  A a1 = xs; // OK, desugars into `A a1 = C2(xs);`.

  List<int> ys = [];
  A a2 = ys; // Error.
}

In the first case we infer C1<num>(xs) with context type A, and that yields the parameter type Iterable<num>; with C2(xs) (which is non-generic, so inference is a no-op), the parameter type is List<num>. List<num> <: Iterable<num> so C2 wins.

Note that a class C3 with a constructor C3(int i) would be ignored, because it would be an error to pass xs as the argument to that constructor. So C3 is out because the type check failed, and similarly an implicit wrapper constructor could be out because inference failed.

In the second case we infer C1<int>(ys), yielding parameter type Iterable<int>, and C2(ys) again yields parameter type List<num>. But none of those parameter types is most specific, so a compile-time error occurs.

Of course, it is always possible for a developer to write C1<int>(ys) or C2(ys) explicitly, thus eliminating the error by making that disambiguation decision that the compiler should not.

Proposal for Receiver Classes

The grammar is adjusted as follows:

<classDeclaration> ::= // Modified
    'abstract'? 'static'? 'receiver'? 'class' <typeApplication>
    (<superclass> <mixins>?)? <interfaces>?
    '{' (<metadata> <classMemberDefinition>)* '}'
  | 'abstract'? 'static'? 'receiver'? 'class' <mixinApplicationClass>

The only change is that the modifier receiver is allowed on a class declaration.

Static Analysis

A receiver class is a class whose declaration includes the modifier receiver.

The static analysis of a receiver class is identical to the static analysis of other classes.

Expressions in the program are affected by the existence of wrapper classes:

Consider an expression e with static type T which is subject to a member lookup for a member m, which we will indicate as e.m... below. (*For instance, it could be e.m(42) or e..m2()..m), and assume that the interface of T does not have a member with the name m. (Without this proposal, that is a compile-time error.)

Assume that the set of receiver classes declaring a member named m in scope is C1 .. Ck. Let c1 .. cm be the implicit constructors of C1 .. Ck accepting one positional argument (which means that cj is of the form SomeClass or SomeClass.someName for each j in 1 .. m), and assume that inference on cj(e).m... with context type S succeeds and yields type arguments Ujs (that is a list of actual type arguments, and the type list U1s may have a different length than the type list U2s, etc), for j in 1 .. n.

There may be fewer than m of these (because inference and/or type checking fails with some constructors), hence we go up to n rather than m. We assume that the constructors have been ordered such that the failing ones are the ones with the highest numbers.

If exactly one of cj<Ujs> is such that the type of its parameter is a subtype of all of those of c1<U1s> .. cn<Uns> then e is replaced by cj<Ujs>(e). Otherwise a compile-time error occurs.

Example

class A {}

receiver class C1<X extends num> implements A {
  final Iterable<X> target;
  implicit C1(this.target);
  X foo() => target.first;
}

receiver class C2 implements A {
  final List<num> target;
  implicit C2(this.target);
  num foo() => 3.2;
}


main() {
  List<num> xs = [];
  num n = xs.foo(); // OK, desugars into `num n = C2(xs).foo();`.

  List<int> ys = [];
  num i = ys.foo(); // Error.
}

In the first case we infer C1<num>(xs).foo() with context type num, and that gives the parameter the type Iterable<num>; with C2(xs) (which is non-generic, so inference is a no-op), the parameter type is List<num>. List<num> <: Iterable<num> so C2 wins.

In the second case we infer C1<int>(ys), yielding parameter type Iterable<int>, and C2(ys) again yields parameter type List<num>. But none of those parameter types is most specific, so a compile-time error occurs.

Note that a receiver class C3 that does not have a member named foo is ignored, and so is a receiver class C4 that does not have an implicit constructor with one argument whose argument type is such that the type of xs or ys is assignable to it.

Of course, it is again possible for a developer to disambiguate the situation by writing the invocation of one of the implicit constructors explicitly.

Static Variants

In the case where a wrapper class or a receiver class is static (cf. #308), the output from the code transformation associated with receiver respectively wrapper classes is such that the implicitly created object does not need to be allocated.

This makes it possible for static extension methods (#41) to be emulated by receiver classes:

extension E on T {
  <memberDeclarations>
}

// Desugars to the following:
static receiver class E {
  final T this;
  E(this.this);
  <memberDeclarations>
}

In this desugaring, we rely on the ability of certain declarations to have the name this, which makes it possible to implicitly access members of the value of this (just like we can access members of the current instance of the enclosing class in an instance method).

In the case where the receiver class is not declared static, it is allowed for the "receiver object" to carry its own mutable state and to use references to itself in arbitrary ways (that is, all expressions are allowed rather than just the non-leaking ones, cf. #308). Similarly, it is possible for a receiver class to declare any number of constructors, implicit or not, and they can have superinterfaces and use mixins like any other class. These things make receiver classes a non-trivial generalization of static extension methods.

Similarly, it is possible for an extension type (#42) to be emulated by a wrapper class:

typedef E on T {
  <memberDeclarations>
}

// Desugars as follows
static wrapper class E {
  final T this;
  E(this.this);
  <memberDeclarations>
}

At call sites, a wrapped object for a static wrapper class can be accessed through desugared top-level functions taking the wrappee (and any instance variables of the wrapper) as actual arguments:

main() {
  E x = e; // Some expression of type `T`.
  x.someMemberOfE();
  requireAnActualWrapper(x);
  requireAnActualWrapper(x);
}

// Desugars as follows:
main() {
  T _x = e;
  lazy E x = E(_x);
  desugared_E_someMemberOfE(_x, other, instance, variables);
  requireAnActualWrapper(x); // The initializer of `x` is now executed.
  requireAnActualWrapper(x); // Then: `x` denotes same wrapper each time.
}

There may be small discrepancies. For instance, the use of super to invoke a method on the target object—the original receiver for a receiver class and the wrapped object for the wrapper class—does not work the same when using a receiver/wrapper class, because that will simply be an invocation of a method from a superclass of that receiver/wrapper class, but this.foo() would always call a foo method on the original receiver/wrappee (rather than on the enclosing instance of the receiver/wrapper class), and a plain foo() would be used to get the one from the enclosing scope (and not from the original receiver/wrappee).

Discussion

The purpose of considering receiver and wrapper classes as a foundation of extension methods and extension types is to express the desired semantics of such mechanism using the smallest possible increments:

Static classes are used in order to ensure that performance corresponds to the straightforward implementation strategies for extension methods (just desugar them to statically resolved function calls) and extension types (where the "view" provided by the extension type is applied to the target object without changing the representation of that target at all, that is, without having a wrapper: just call static methods that desugar the static wrapper class, and pass the wrappee as well as any "instance variables of the wrapper" as actual arguments to the static functions that are desugared versions of the wrapper class methods).

If we consider the ability to eliminate instances of static classes as a given, then we may think of other properties of these mechanism as if they always have those objects which are created by the code which is the output of the transformations.

So whenever a method is executed on an instance of a receiver class, it's just a completely normal method invocation and it is in principle unimportant that the original receiver is the value of a field in that receiver class instance.

Similarly, if an object is wrapped by an instance of a wrapper class and used in a context which is "leaking" (that is, it is not non-leaking, cf. #308) then we will obtain an actual instance of the wrapper class, and that object will implement the superinterfaces from the declaration of the wrapper class, which means that it is a full-fledged object of the desired type. So in the case where we actually get a wrapper object, it serves as a mechanism which provides the "view" of the extension type on the given wrappee object, and this property will hold in all context, e.g., also for dynamic invocations.

Comparing with #107 (and more general implicit conversion mechanisms), this proposal only addresses the situation where we can write a constructor for the target class (B in #107 lingo). For instance, it does not address the situation where we want to transform a String into an int, because we cannot make int a wrapper class.

In relation to #108, this proposal also uses the modifier implicit on certain constructors. The obvious difference is that the situations where an implicit constructor can be applied is parceled out into the receiver case and the wrapper case; I believe that #108 is mainly focused on the latter.

The current proposal for wrapper classes does not support the "inverse conversion": We may implicitly wrap an A and get a B, but there is no standardized way to obtain the original A again from that B, say, if the B is passed around and an A is needed somewhere. Indeed, nobody enforces that the construction of the B even uses the argument of type A that it receives. It might be useful to have a standardized member in wrapper classes which are used to perform this kind of "unwrapping".

[lrhn]

One issue that needs to be addressed is cascades:

(e1..foo()..bar()).baz()

Here e1 is the receiver of three member invocations.

We usually say that we evaluate e1 to a value, then perform normal member invocation on the value multiple times. If the wrapping occurs independently for each invocation, then it means that there cannot be shared state between the invocations. On the other hand, each individual invocation can be wrapped or non-wrapped independently of the other, so if the type of e1 has a bar member, and two different static receiver classes have the foo and baz members, then this code still works.

If we require the cascade receiver itself to be wrapped only once, then we have to either make an arbitrary choice of which single invocation to use for the receiver class wrapping, or to check for a single receiver class which satisfies all invocations (and fail if there is none).

I think the former behavior is more reasonable. An extension method should act as if it was on the object, so it should be possible to mix it in between actual class methods.

That also suggests that keeping updatable state between invocations is not something we can rely on, and not something we should be designing this feature around.

[eernstg]

@lrhn wrote:

One issue that needs to be addressed is cascades

Right, I hadn't spelled that out in detail.

The specification of cascade execution actually describes it in such a way that we could claim that it is already being desugared as follows:

e..suffix -->
let t = e, _ = t.suffix in t

which will allow us to apply a separate implicit wrapping operation on each method invocation in a cascade (noting that t.suffix will unfold similarly until we have covered all sections).

This would mean that there is no shared state across the sections of a cascade. I believe that this would in fact be the most practical and comprehensible semantics.

make an arbitrary choice of which single invocation to use for the receiver class wrapping
[.. or ..]
check for a single receiver class which satisfies all invocations

I think it would be really confusing for developers if a cascade were to prevent regular instance methods from being invoked in later sections if the first one calls an extension method, and vice versa.

And the above desugaring actually allows them to be mixed freely (instance methods and extension methods, from the same or from different extensions). Everything happens one cascade section at a time—this is both possible to understand and (presumably) useful in practice.

keeping updatable state between invocations is not something we can rely on

Right, we wouldn't and shouldn't have that for a cascade.

On the other hand, I don't see any problems in having mutable state in a wrapper object in general. For instance, one extension method could give rise to a computation where any number of other methods on the same wrapper object would be executed, and they would be able to use the state of that wrapper object because it's in scope.

For a developer who is writing a receiver class R, the relevant way to think about mutable state is that "this class will be used as in new R(r).foo()", and then all the issues about how the mutable state is initialized and how long that R instance will live are just standard programming considerations. If we can write a class that works correctly when used like that explicitly, it will also work correctly when it is used implicitly in the same manner.

[vishnuagbly]

Can anyone tell me, about the progress made so far on this issue? This will be an exciting update for me.

[Levi-Lesches]

To add to that, what's the status of this vs extensions (and the proposed static extension types)?

[eernstg]

Static classes are probably not going to be added to the language. I think it would be a useful generalization of extension declarations. But we have extension declarations today, and it's not likely that they will undergo a substantial generalization like that.

Implicit constructors themselves are mentioned now and then, also in connection with views (which is the new and better name for static extension types ;-).

I think there is some support for the usefulness of the basic idea that we can mark a constructor as implicit, and it can then be used in a desugaring step where some hint is available (say, we encounter e.foo(), but the type of e doesn't have a foo, or we encounter foo(e), but foo doesn't accept an argument with that type). This part of the proposal has substantial overlaps with #108 as well.