Allow impl Fn() -> impl Trait

[WaffleLapkin]

This allows writing the following function signatures:

fn f0() -> impl Fn() -> impl Trait;
fn f1(_: impl Fn() -> impl Trait);
fn f2<F: Fn() -> impl Trait>(_: F);
fn f3() -> &'static dyn Fn() -> impl Trait;
fn f4(_: &dyn Fn() -> impl Trait);

All of the above are already allowed with common traits and associated
types, there is no reason why Fn* traits should be special in this
regard.

There even is a test that f0 compiles:

rust/src/test/ui/impl-trait/nested_impl_trait.rs

Lines 25 to 28 in 2f004d2

	fn allowed_in_ret_type() -> impl Fn() -> impl Into<u32> {
	//~^ `impl Trait` not allowed
	|| 5
	}

But it was changed in PR 48084 (lines) to test the opposite, probably unintentionally given PR 48084 (lines).

r? @oli-obk

---

This limitation is especially annoying with async code, since it forces one to write this:

trait AsyncFn3<A, B, C>: Fn(A, B, C) -> <Self as AsyncFn3<A, B, C>>::Future {
    type Future: Future<Output = Self::Out>;

    type Out;
}

impl<A, B, C, Fut, F> AsyncFn3<A, B, C> for F
where
    F: Fn(A, B, C) -> Fut,
    Fut: Future,
{
    type Future = Fut;

    type Out = Fut::Output;
}

fn async_closure() -> impl AsyncFn3<i32, i32, i32, Out = u32> {
    |a, b, c| async move { (a + b + c) as u32 }
}

Instead of:

fn async_closure() -> impl Fn(i32, i32, i32) -> impl Future<Output = u32> {
    |a, b, c| async move { (a + b + c) as u32 }
}

[oli-obk]

cc @rust-lang/lang this seems like a very straight-forward stabilization, especially considering there are workarounds that allow the same thing.

[cramertj]

Eventually I'd like this to work:

fn print_some_things_times_two(times_two: impl Fn(impl Add) -> impl Debug) {
    println!("{:?}", times_two(1u8));
    println!("{:?}", times_two(1u32));
}

print_some_things_times_two(|x| x + x);

Note that the return impl Debug type from the Fn depends on the type of the impl Add. Today, impl Trait in the return position of Fn signatures would desugar to a single type parameter on the top-level function, which isn't, I think, the behavior we want long-term. It might be that there's no way to observe this without also allowing impl Trait in the argument position of Fn signatures, in which case this stabilization is forwards-compatible with such a change. However, I want to be pretty confident that we can evolve towards this in the future before landing this stabilization.

[WaffleLapkin]

@cramertj shouldn't the type be impl Fn(impl Add<Output = impl Debug> + Copy) -> impl Debug for this to potentially work?

Note that this actually requires type-HRTB (or is it HKT?...) (as well as generic closures, oh wow):

for<T>
where
    T: Add<Output = impl Debug> + Copy
impl Fn(T) -> impl Debug

---

Personally, I think that

fn f(_: impl Fn(impl Tr)) {}

Should desugar to

fn f<F, A>(_: F)
where
    F: Fn(A),
    A: Tr,
{}

As this is more consistent with Ty<impl Tr>.

But I believe that this PR is compatible with both options.

[cramertj]

@WaffleLapkin It's exactly type-HRTB + generic closures, yes.

[joshtriplett]

So, if we decide to do this, we're deciding on an associativity for -> ... -> ... ->.

For instance, what happens if you write fn func() -> impl Fn() -> impl Fn() -> impl Debug? What precisely does that mean?

I would expect it to mean fn func() -> impl (Fn() -> impl (Fn() -> impl Debug)), which implies that -> is right-associative and binds from right to left. I think that's the correct answer; I don't think it would make sense for it to associate the other way.

Could you please add a test case demonstrating that associativity? Given such a test case, I'd be happy to propose this for FCP.

[joshtriplett]

(Removing T-compiler in order to do a T-lang FCP)

[joshtriplett]

@WaffleLapkin Thanks, that test case looks great!

@rust-lang/lang: Do we want to commit to right-associativity (rather than non-associativity) for ->, such that fn func() -> impl Fn() -> impl Fn() -> impl Debug works?

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @cramertj
• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm

Concerns:

• forward-compatibility-with-apit-type-hrtb (Allow impl Fn() -> impl Trait #93082 (comment))
• interaction-of-parens (Allow impl Fn() -> impl Trait #93082 (comment))
• niko-wants-to-do-a-good-review resolved by Allow impl Fn() -> impl Trait #93082 (comment)
• what-should-return-in-arg-mean (Allow impl Fn() -> impl Trait #93082 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[ehuss]

Would it be possible to update the documentation at https://github.com/rust-lang/reference/blob/master/src/types/impl-trait.md to make it clear this is allowed (and any other details that seem worthy to add)?

[veber-alex]

Will this allow:

fn new() -> impl for<'a> FnOnce(&'a i32) -> (impl Future<Output = &'a i32> + 'a) {
    |x| async move { x }
}

instead of the current:

fn old() -> impl for<'a> FnOnce(&'a i32) -> Box<dyn Future<Output = &'a i32> + 'a> {
    |x| Box::new(async move { x })
}

?

[cramertj]

@rfcbot concern forward-compatibility-with-apit-type-hrtb

See #93082 (comment)

[WaffleLapkin]

@ehuss it should be possible to update the reference, yes (it also doesn't seem to mention impl Iterator<Item = impl Debug> kind of things).

@veber-alex this will allow the following:

fn new() -> impl FnOnce(i32) -> impl Future<Output = i32> {
    |x| async move { x }
}

Your example with lifetime currently segfaults compiler, see #88236. But when this bug will be fixed, your example will be fine too.

[WaffleLapkin]

Actually I'm now thinking, when #88236 is fixed, will impl for<'a> Tr<'a, A = impl B + 'a> work as expected? ie will <T as Tr<'x>>::A and <T as Tr<'x>>::B be different types? 🤔

[nikomatsakis]

We have been specifically ruling this out because we wanted a clear determination of what it means to use impl Trait in all places.

I do want to permit this, but I want to incorporate it into https://rust-lang.github.io/impl-trait-initiative/ with a clearly documented set of rules. We should also permit T: impl Foo<Output = impl Bar>, if we don't already, and also make sure to test lifetime erasure interactions.

@rfcbot concern niko-wants-to-do-a-good-review

[WaffleLapkin]

@nikomatsakis impl Foo<Output = impl Bar> is already permitted. And impl Fn() -> impl Bar should follow the same rules.

If I understand correctly, in Assoc = impl Trait, impl Trait means the same as the "outer impl Trait".

For example param: impl Tr<A = impl Trait> means two generics and -> impl Tr<A = impl Trait> means two existential types.

[nikomatsakis]

@WaffleLapkin ok, +1, and I agree they should have the same meaning.

[fredlahde]

I ran into a segfault when trying to return very deeply nested function compositions (with a depth of 560). I filed #93237 for that.

[ericsampson]

Has anyone nailed down the answer to Josh’s question about the desired spec for associativity?

[WaffleLapkin]

I've created a PR to the reference, that clarifies existing and added in this PR rules on where impl Trait can be used: rust-lang/reference#1144

[scottmcm]

impl Foo<Output = impl Bar> is already permitted. And impl Fn() -> impl Bar should follow the same rules.

That argument is persuasive to me.

Just to confirm, it's only about output position here? It's not enabling -> impl Bar<impl Hello, impl World>?

[nikomatsakis]

@rfcbot resolve niko-wants-to-do-a-good-review

@rfcbot concern interaction-of-parens

Discussing in the @rust-lang/lang meeting and came up with some concerns. I am trending against stabilizing now, based on these. Let me try to spell it out.

The first concern is this: I have generally wanted '_ and impl Trait to act in analogous ways, but this proposal diverges from that. In particular, with impl Fn(&u32) -> Foo<'_>, the '_ refers back to something from the arguments, which is bound at the Fn level. But impl Fn() -> impl Debug introduces a binder at the surrounding fn item, which means that e.g. impl Fn(&u32) -> impl PartialEq<&u32> behaves in some very weird way.

[nikomatsakis]

@rfcbot concern what-should-return-in-arg-mean

We discussed what fn foo(x: impl Fn() -> impl Debug) would ideally mean and realized that it's rather unclear. The following seem not great:

• The proposal fn foo<T: Debug>(x: impl Fn() -> T) means that T cannot reference higher-bound regions -- and I eventually expect there to be higher-ranked types too that cannot be named (or captured!).
• The proposal fn foo(x: impl for<T: Debug> Fn() -> T) is very rarely what you want.

@cramertj pointed out that what you really want is probably something like fn foo(x: impl exists<T: Debug> Fn() -> T), which might be something we can express at some point (they wrote it as fn foo(x: impl Fn<(), Output: Debug>), essentially, I'm not sure what our current implementation of Foo: Bar bounds does in that situation....

I can whip up some tests here.

[nikomatsakis]

BTW, @cramertj pointed out that this particular case would be ok:

fn foo() -> impl Fn() -> impl Debug

That seems probably true, although it is still somewhat more limited than what it could eventually support.

[WaffleLapkin]

Just to confirm, it's only about output position here? It's not enabling -> impl Bar<impl Hello, impl World>?

@scottmcm Yes, here is the test for this:

rust/src/test/ui/impl-trait/nested_impl_trait.rs

Line 12 in 3ed486b

fn bad_in_arg_position(_: impl Into<impl Debug>) { }

[WaffleLapkin]

The proposal fn foo<T: Debug>(x: impl Fn() -> T) means that T cannot reference higher-bound regions -- and I eventually expect there to be higher-ranked types too that cannot be named (or captured!).

Well, it's actually pretty easy to come up with an example, where we want higher-bound regions -- |x: &_| move async { x }. That said, I still think that this (foo<T: Debug>) is the right choice here, because having an inconsistency between Fn() -> impl Trait and Fn<(), Output = impl Trait> seems very unpleasant.

The proposal fn foo(x: impl for<T: Debug> Fn() -> T) is very rarely what you want.

I don't even think that we can currently create a type that would satisfy such bound, at least on stable.

@cramertj pointed out that what you really want is probably something like fn foo(x: impl exists<T: Debug> Fn() -> T), which might be something we can express at some point (they wrote it as fn foo(x: impl Fn<(), Output: Debug>), essentially, I'm not sure what our current implementation of Foo: Bar bounds does in that situation....

• impl exists<T: Debug> Fn() -> T seems weird to me, because I've always thought that exists<T: Debug> implies that there is a single type that is known as T, but we obviously (?) want to accept functions that return different types, so this is not an option, is it. Am I missing something?...
• impl Fn<(), Output: Debug> if I'm not mistaken this should desugar to foo<F: Fn<()>> where F::Output: Debug which is practically equivalent to foo<F: Fn<(), Output = O>, O> where O: Debug (I'm not very familiar with this feature though, so don't take my word for it, but the RFC seems to suggest the same)

---

TIL that you can't write arg: impl for<'a> Add<&'a u32, Output = impl PartialEq<&'a u32>> uh oh (also the error isn't great for this, "use of undeclared lifetime name 'a")

[nikomatsakis]

>  because having an inconsistency between Fn() -> impl Trait and Fn<(), Output = impl Trait> seems very unpleasant.

I thought that at first, but then I remembered that -- indeed -- that is part of the point of () syntax, that it is different from <> with respect to how higher-ranked regions are bound. For example, parameters like impl Fn(&u8) and impl Foo<&u8> are not the same. The former is for<'a> Fn(&'a u8) and the latter is Foo<&'a u8> where 'a is bound on the enclosing function.

[WaffleLapkin]

So, it seems like we don't want impl Fn() -> impl Trait in argument position to mean what this PR makes it, or at least, not right now. Since you can already write practically the same on stable (f<T: Trait>(_: impl Fn() -> T)) there is no need for this either.

If I understood the conversation right, return position impl Fn is not as controversial (and it's also impossible to write it on stable).

So, am I right, that the way forward is to update this PR to only allow return-position-impl Fn?

[nikomatsakis]

@rfcbot fcp cancel

We discussed this in our @rust-lang/lang meeting today. The conclusion was that we would like to do two things:

• To preserve the insights that we gained here, in terms of potential interactions, on the https://rust-lang.github.io/impl-trait-initiative/ repository (that's on me!)
• To update the PR to covert "RPITIRPIT", i.e., return-position impl Trait in return position impl trait. i.e., fn foo() -> impl Fn() -> impl Debug, specifically.

@WaffleLapkin if you are up to do the latter, that would be great! 💜 I'm going to close this PR for now, though, so that we can have a fresh discussion once you've got that ready.

[cramertj]

fn foo() -> impl Fn() -> impl Debug

One potential source of weirdness is the possible ambiguity created by lifetimes in the impl Debug (the RPITIRPIT) type. I assume we want it to be able to capture lifetimes introduced by the Fn. That is, -> for<'a> Fn(&'a u8) -> impl Debug + 'a. Can one write this as -> Fn(&u8) -> impl Debug + '_? it seems at least visually ambiguous whether the + '_ should apply to the impl Fn or the impl Debug-- presumably we want to just error in the case that elided lifetimes are introduced both in the top-level fn arguments as well as in the impl Fn?

[WaffleLapkin]

@nikomatsakis sure, I'll try to update the PR to allow specifically RPITIRPIT :)

[Mathspy]

RPITRPIT

The return frog joins the turbo fish in guiding the young crab in this new exciting adventure of finding the never type

[WaffleLapkin]

New PR that only allows impl Fn() -> impl Trait in return position is up: #93582!

[kevincox]

Is there a tracking issue for this in an argument position? I specifically want to accept something like

fn foo(
    f: impl for<'a> FnOnce(&'a Args) -> impl 'a + std::future::Future<Res>,
)

[WaffleLapkin]

@kevincox as far as I know, there isn't, and it would be hard to allow what you want, since impl 'a + Future<Res> here is a """family""" of types (a different type for each possible 'a) and Rust's type system just straight up does not support equating associated types to """families""". (I'm not sure if I used the correct terminology)

[Jules-Bertholet]

@kevincox the following works on Nightly with -Ztrait-solver=next:

#![feature(unboxed_closures)]

use std::future::Future;

struct Args;
struct Res;

fn foo<F>(f: F)
where
    F: for<'a> FnOnce<(&'a Args,)>,
    for<'a> <F as FnOnce<(&'a Args,)>>::Output: Future<Output = Res>,
{
    todo!()
}

fn main() {
    foo(|_: &Args| async { Res })
}

[nikomatsakis]

Filed rust-lang/impl-trait-initiative#15 to track impl Fn() -> impl Trait and rust-lang/impl-trait-initiative#16 for impl Fn(impl Trait) (cc @kevincox)