Add AsyncFn family of traits

[compiler-errors]

I'm proposing to add a new family of asynchronous Fn-like traits to the standard library for experimentation purposes.

Why do we need new traits?

On the user side, it is useful to be able to express AsyncFn trait bounds natively via the parenthesized sugar syntax, i.e. x: impl AsyncFn(&str) -> String when experimenting with async-closure code.

This also does not preclude AsyncFn becoming something else like a trait alias if a more fundamental desugaring (which can take many¹ different² forms) comes around. I think we should be able to play around with AsyncFn well before that, though.

I'm also not proposing stabilization of these trait names any time soon (we may even want to instead express them via new syntax, like async Fn() -> ..), but I also don't think we need to introduce an obtuse bikeshedding name, since AsyncFn just makes sense.

The lending problem: why not add a more fundamental primitive of LendingFn/LendingFnMut?

Firstly, for async closures to be as flexible as possible, they must be allowed to return futures which borrow from the async closure's captures. This can be done by introducing LendingFn/LendingFnMut traits, or (equivalently) by adding a new generic associated type to FnMut which allows the return type to capture lifetimes from the &mut self argument of the trait. This was proposed in one of Niko's blog posts.

Upon further experimentation, for the purposes of closure type- and borrow-checking, I've come to the conclusion that it's significantly harder to teach the compiler how to handle general lending closures which may borrow from their captures. This is, because unlike Fn/FnMut, the LendingFn/LendingFnMut traits don't form a simple "inheritance" hierarchy whose top trait is FnOnce.

flowchart LR
    Fn
    FnMut
    FnOnce
    LendingFn
    LendingFnMut
    
    Fn -- isa --> FnMut
    FnMut -- isa --> FnOnce
    
    LendingFn -- isa --> LendingFnMut
    
    Fn -- isa --> LendingFn
    FnMut -- isa --> LendingFnMut

Loading

For example:

fn main() {
  let s = String::from("hello, world");
  let f = move || &s;
  let x = f(); // This borrows `f` for some lifetime `'1` and returns `&'1 String`.

That trait hierarchy means that in general for "lending" closures, like f above, there's not really a meaningful return type for <typeof(f) as FnOnce>::Output -- it can't return &'static str, for example.

Special-casing this problem:

By splitting out these traits manually, and making sure that each trait has its own associated future type, we side-step the issue of having to answer the questions of a general LendingFn/LendingFnMut implementation, since the compiler knows how to generate built-in implementations for first-class constructs like async closures, including the required future types for the (by-move) AsyncFnOnce and (by-ref) AsyncFnMut/AsyncFn trait implementations.

Footnotes

1. For example, with trait transformers, we may eventually be able to write: trait AsyncFn = async Fn; ↩

2. For example, via the introduction of a more fundamental "LendingFn" trait, plus a special desugaring with augmented trait aliases. ↩

[rustbot]

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

[compiler-errors]

I'm nominating this for T-libs-api discussion -- not sure if what process is needed to introduce these traits as unstable :)

r? libs-api

[compiler-errors]

also cc @rust-lang/wg-async

[traviscross]

Exciting work. Big thanks to @compiler-errors for pushing this forward.

@rustbot labels +I-lang-nominated

Nominating this for T-lang. At the end of the day T-lang will need to be satisfied with the design here, so we should discuss it at this point at least for visibility.

And given that this is adding new lang items, we may want to charter this as a T-lang experiment.

@rustbot labels +I-async-nominated

Nominating this for WG-async. Async closures have been a recent topic of discussion for WG-async and have been identified as a priority. The async WG will at the end of the day want to be happy with the design, so we too should discuss it.

[nikomatsakis]

Discussed in the @rust-lang/lang meeting and we agreed to accept this as an experiment (I will server as the liaison). All things being equal the goal we would like as much interop and generalization between "async functions" and "functions returning traits" as we can get, but I clarified that @compiler-errors and I were already aligned on that but that there are some interesting complications that make that non-trivial, and that the intent is to make concrete progress towards an MVP, with an eye for future compatibility.

[nikomatsakis]

@rustbot labels -I-lang-nominated

[nikomatsakis]

Hey @rust-lang/libs, I did not remove the I-libs-api-nominated tag. My take is that this is more of a lang question than a libs one, even though it involves a trait, and I would expect lang to be primary decision maker with libs-api in a "consulting role" (i.e., free to raise objections, but not requiring active checkboxes). If you all would like a more active role, though, happy to discuss.

[traviscross]

Regarding the I-libs-api-nominated tag, I removed that one because T-libs-api had not nominated it themselves; it had just been nominated for them while we worked out who would charter this work. As discussed above, T-lang is claiming this as an experiment, so there's probably not any remaining nominated question for T-libs-api. But as nikomatsakis suggested, of course, please renominate if there are open questions that T-libs-api should discuss.

[compiler-errors]

I'm gonna pause this for a second -- still trying to decide whether this is better than just introducing LendingFn and LendingFnMut traits.

@rustbot author

[compiler-errors]

After a lot of thought and experimentation, I've decided it's best to first introduce AsyncFn traits here, and then work on generalizing the behavior of async closures to work with the other "coroutine transform" flavors (gen and async gen) via a desugaring that decouples asyncness from the fact that these closures enable self-borrows.

A desugaring that is backed by LendingFn is more powerful, but it has some difficult interactions with the new trait solver I'm interested in writing up sometime soon. Given that @nikomatsakis has agreed to serve as liaison for this lang team experiment, and the interesting changes here are all in the type system, I'll r? oli-obk.

@rustbot ready

[oli-obk]

@bors r+ rollup

[bors]

📌 Commit 17b4333 has been approved by oli-obk

It is now in the queue for this repository.