Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inherit lifetimes for async fn instead of duplicating them. #91403

Merged
merged 5 commits into from
Feb 13, 2022

Conversation

cjgillot
Copy link
Contributor

The current desugaring of async fn foo<'a>(&usize) -> &u8 is equivalent to

fn foo<'a, '0>(&'0 usize) -> foo<'static, 'static>::Opaque<'a, '0, '_>;
type foo<'_a, '_0>::Opaque<'a, '0, '1> = impl Future<Output = &'1 u8>;

following the RPIT model.

Duplicating all the inherited lifetime parameters and setting the inherited version to 'static makes lowering more complex and causes issues like #61949. This PR removes the duplication of inherited lifetimes to directly use

fn foo<'a, '0>(&'0 usize) -> foo<'a, '0>::Opaque<'_>;
type foo<'a, '0>::Opaque<'1> = impl Future<Output = &'1 u8>;

following the TAIT model.

Fixes #61949

@rust-highfive
Copy link
Collaborator

r? @nagisa

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 30, 2021
@rust-log-analyzer

This comment has been minimized.

@cjgillot
Copy link
Contributor Author

cjgillot commented Dec 1, 2021

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Dec 1, 2021
@bors
Copy link
Contributor

bors commented Dec 1, 2021

⌛ Trying commit 88af837 with merge 5dc1f10578886a598f5fc415475961d726194f80...

@bors
Copy link
Contributor

bors commented Dec 1, 2021

☀️ Try build successful - checks-actions
Build commit: 5dc1f10578886a598f5fc415475961d726194f80 (5dc1f10578886a598f5fc415475961d726194f80)

@rust-timer
Copy link
Collaborator

Queued 5dc1f10578886a598f5fc415475961d726194f80 with parent f04a2f4, future comparison URL.

@apiraino apiraino added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Dec 1, 2021
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (5dc1f10578886a598f5fc415475961d726194f80): comparison url.

Summary: This change led to moderate relevant mixed results 🤷 in compiler performance.

  • Small improvement in instruction counts (up to -0.6% on incr-unchanged builds of deeply-nested-async)
  • Moderate regression in instruction counts (up to 1.1% on full builds of issue-88862)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Dec 2, 2021
@oli-obk
Copy link
Contributor

oli-obk commented Dec 3, 2021

r? @oli-obk

@rust-log-analyzer

This comment has been minimized.

Copy link
Member

@spastorino spastorino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, there are some good cleanups. There rust-analyzer tests failing. On the other hand there are some things being done that are going in the opposite direction. In particular, not inheriting things at all.

@spastorino
Copy link
Member

Having seconds thoughts now, can you maybe extract the cleanup commits that doesn't change functionality into a separate PR?. That part can easily be merged and then we can continue discussing this with @oli-obk and @nikomatsakis

@spastorino spastorino self-requested a review December 6, 2021 17:01
@bors

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 8, 2021
Simplify collection of in-band lifetimes

Split from rust-lang#91403

r? `@oli-obk`
@oli-obk
Copy link
Contributor

oli-obk commented Feb 12, 2022

@bors r+

No matter what final design we come up with, as mentioned previously, it reduces complexity

@bors
Copy link
Contributor

bors commented Feb 12, 2022

📌 Commit 10cf626 has been approved by oli-obk

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Feb 12, 2022
@bors
Copy link
Contributor

bors commented Feb 12, 2022

⌛ Testing commit 10cf626 with merge 3cfa4de...

@bors
Copy link
Contributor

bors commented Feb 13, 2022

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing 3cfa4de to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 13, 2022
@bors bors merged commit 3cfa4de into rust-lang:master Feb 13, 2022
@rustbot rustbot added this to the 1.60.0 milestone Feb 13, 2022
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (3cfa4de): comparison url.

Summary: This benchmark run shows 2 relevant regressions 😿 to instruction counts.

  • Average relevant regression: 1.7%
  • Largest regression in instruction counts: 2.0% on full builds of issue-88862 check

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression

@cjgillot cjgillot deleted the inherit-async branch February 13, 2022 09:26
@nikomatsakis
Copy link
Contributor

We should definitely talk this out -- I don't think I want to avoid lowering where clauses twice, so that's worth talking over. =)

@pnkfelix
Copy link
Member

pnkfelix commented Feb 16, 2022

Finished benchmarking commit (3cfa4de): comparison url.

Summary: This benchmark run shows 2 relevant regressions 😿 to instruction counts.

* Average relevant regression: 1.7%

* Largest regression in instruction counts: 2.0% on `full` builds of `issue-88862 check`

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression

It looks to me like the regression identified by the rust-timer run from 4 days ago (i.e. when this landed) is exactly the same as the regression identified back when the PR was first filed.

But, I do not see any discussion of that performance regression. 😞

Part of the reason to do the timer runs is to encourage PR authors and reviewers to acknowledge the fallout of the changes they are making, and to answer questions like:

  • Was this performance regression anticipated?
  • Is it justified based on the benefit of the PR?
  • Is there a plan (or even vague ideas) to address the regression, or is it expected to be a cost we will just pay?

oli-obk added a commit to oli-obk/rust that referenced this pull request Feb 17, 2022
…-obk"

This reverts commit 3cfa4de, reversing
changes made to 5d8767c.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 17, 2022
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 18, 2022
flip1995 pushed a commit to flip1995/rust that referenced this pull request Feb 24, 2022
…-obk"

This reverts commit 3cfa4de, reversing
changes made to 5d8767c.
flip1995 pushed a commit to flip1995/rust that referenced this pull request Feb 24, 2022
@tmandry
Copy link
Member

tmandry commented May 3, 2022

Status: Reverted in #94088

bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 28, 2023
Stabilize `impl_trait_projections`

Closes rust-lang#115659

## TL;DR:

This allows us to mention `Self` and `T::Assoc` in async fn and return-position `impl Trait`, as you would expect you'd be able to.

Some examples:
```rust
#![feature(return_position_impl_trait_in_trait, async_fn_in_trait)]
// (just needed for final tests below)

// ---------------------------------------- //

struct Wrapper<'a, T>(&'a T);

impl Wrapper<'_, ()> {
    async fn async_fn() -> Self {
        //^ Previously rejected because it returns `-> Self`, not `-> Wrapper<'_, ()>`.
        Wrapper(&())
    }

    fn impl_trait() -> impl Iterator<Item = Self> {
        //^ Previously rejected because it mentions `Self`, not `Wrapper<'_, ()>`.
        std::iter::once(Wrapper(&()))
    }
}

// ---------------------------------------- //

trait Trait<'a> {
    type Assoc;
    fn new() -> Self::Assoc;
}
impl Trait<'_> for () {
    type Assoc = ();
    fn new() {}
}

impl<'a, T: Trait<'a>> Wrapper<'a, T> {
    async fn mk_assoc() -> T::Assoc {
        //^ Previously rejected because `T::Assoc` doesn't mention `'a` in the HIR,
        //  but ends up resolving to `<T as Trait<'a>>::Assoc`, which does rely on `'a`.
        // That's the important part -- the elided trait.
        T::new()
    }

    fn a_few_assocs() -> impl Iterator<Item = T::Assoc> {
        //^ Previously rejected for the same reason
        [T::new(), T::new(), T::new()].into_iter()
    }
}

// ---------------------------------------- //

trait InTrait {
    async fn async_fn() -> Self;

    fn impl_trait() -> impl Iterator<Item = Self>;
}

impl InTrait for &() {
    async fn async_fn() -> Self { &() }
    //^ Previously rejected just like inherent impls

    fn impl_trait() -> impl Iterator<Item = Self> {
        //^ Previously rejected just like inherent impls
        [&()].into_iter()
    }
}
```

## Technical:

Lifetimes in return-position `impl Trait` (and `async fn`) are duplicated as early-bound generics local to the opaque in order to make sure we are able to substitute any late-bound lifetimes from the function in the opaque's hidden type. (The [dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html#aside-opaque-lifetime-duplication) has a small section about why this is necessary -- this was written for RPITITs, but it applies to all RPITs)

Prior to rust-lang#103491, all of the early-bound lifetimes not local to the opaque were replaced with `'static` to avoid issues where relating opaques caused their *non-captured* lifetimes to be related. This `'static` replacement led to strange and possibly unsound behaviors (rust-lang#61949 (comment)) (rust-lang#53613) when referencing the `Self` type alias in an impl or indirectly referencing a lifetime parameter via a projection type (via a `T::Assoc` projection without an explicit trait), since lifetime resolution is performed on the HIR, when neither `T::Assoc`-style projections or `Self` in impls are expanded.

Therefore an error was implemented in rust-lang#62849 to deny this subtle behavior as a known limitation of the compiler. It was attempted by `@cjgillot` to fix this in rust-lang#91403, which was subsequently unlanded. Then it was re-attempted to much success (🎉) in rust-lang#103491, which is where we currently are in the compiler.

The PR above (rust-lang#103491) fixed this issue technically by *not* replacing the opaque's parent lifetimes with `'static`, but instead using variance to properly track which lifetimes are captured and are not. The PR gated any of the "side-effects" of the PR behind a feature gate (`impl_trait_projections`) presumably to avoid having to involve T-lang or T-types in the PR as well. `@cjgillot` can clarify this if I'm misunderstanding what their intention was with the feature gate.

Since we're not replacing (possibly *invariant*!) lifetimes with `'static` anymore, there are no more soundness concerns here. Therefore, this PR removes the feature gate.

Tests:
* `tests/ui/async-await/feature-self-return-type.rs`
* `tests/ui/impl-trait/feature-self-return-type.rs`
* `tests/ui/async-await/issues/issue-78600.rs`
* `tests/ui/impl-trait/capture-lifetime-not-in-hir.rs`

---

r? cjgillot on the impl (not much, just removing the feature gate)

I'm gonna mark this as FCP for T-lang and T-types.
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 28, 2023
Stabilize `impl_trait_projections`

Closes rust-lang#115659

## TL;DR:

This allows us to mention `Self` and `T::Assoc` in async fn and return-position `impl Trait`, as you would expect you'd be able to.

Some examples:
```rust
#![feature(return_position_impl_trait_in_trait, async_fn_in_trait)]
// (just needed for final tests below)

// ---------------------------------------- //

struct Wrapper<'a, T>(&'a T);

impl Wrapper<'_, ()> {
    async fn async_fn() -> Self {
        //^ Previously rejected because it returns `-> Self`, not `-> Wrapper<'_, ()>`.
        Wrapper(&())
    }

    fn impl_trait() -> impl Iterator<Item = Self> {
        //^ Previously rejected because it mentions `Self`, not `Wrapper<'_, ()>`.
        std::iter::once(Wrapper(&()))
    }
}

// ---------------------------------------- //

trait Trait<'a> {
    type Assoc;
    fn new() -> Self::Assoc;
}
impl Trait<'_> for () {
    type Assoc = ();
    fn new() {}
}

impl<'a, T: Trait<'a>> Wrapper<'a, T> {
    async fn mk_assoc() -> T::Assoc {
        //^ Previously rejected because `T::Assoc` doesn't mention `'a` in the HIR,
        //  but ends up resolving to `<T as Trait<'a>>::Assoc`, which does rely on `'a`.
        // That's the important part -- the elided trait.
        T::new()
    }

    fn a_few_assocs() -> impl Iterator<Item = T::Assoc> {
        //^ Previously rejected for the same reason
        [T::new(), T::new(), T::new()].into_iter()
    }
}

// ---------------------------------------- //

trait InTrait {
    async fn async_fn() -> Self;

    fn impl_trait() -> impl Iterator<Item = Self>;
}

impl InTrait for &() {
    async fn async_fn() -> Self { &() }
    //^ Previously rejected just like inherent impls

    fn impl_trait() -> impl Iterator<Item = Self> {
        //^ Previously rejected just like inherent impls
        [&()].into_iter()
    }
}
```

## Technical:

Lifetimes in return-position `impl Trait` (and `async fn`) are duplicated as early-bound generics local to the opaque in order to make sure we are able to substitute any late-bound lifetimes from the function in the opaque's hidden type. (The [dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html#aside-opaque-lifetime-duplication) has a small section about why this is necessary -- this was written for RPITITs, but it applies to all RPITs)

Prior to rust-lang#103491, all of the early-bound lifetimes not local to the opaque were replaced with `'static` to avoid issues where relating opaques caused their *non-captured* lifetimes to be related. This `'static` replacement led to strange and possibly unsound behaviors (rust-lang#61949 (comment)) (rust-lang#53613) when referencing the `Self` type alias in an impl or indirectly referencing a lifetime parameter via a projection type (via a `T::Assoc` projection without an explicit trait), since lifetime resolution is performed on the HIR, when neither `T::Assoc`-style projections or `Self` in impls are expanded.

Therefore an error was implemented in rust-lang#62849 to deny this subtle behavior as a known limitation of the compiler. It was attempted by `@cjgillot` to fix this in rust-lang#91403, which was subsequently unlanded. Then it was re-attempted to much success (🎉) in rust-lang#103491, which is where we currently are in the compiler.

The PR above (rust-lang#103491) fixed this issue technically by *not* replacing the opaque's parent lifetimes with `'static`, but instead using variance to properly track which lifetimes are captured and are not. The PR gated any of the "side-effects" of the PR behind a feature gate (`impl_trait_projections`) presumably to avoid having to involve T-lang or T-types in the PR as well. `@cjgillot` can clarify this if I'm misunderstanding what their intention was with the feature gate.

Since we're not replacing (possibly *invariant*!) lifetimes with `'static` anymore, there are no more soundness concerns here. Therefore, this PR removes the feature gate.

Tests:
* `tests/ui/async-await/feature-self-return-type.rs`
* `tests/ui/impl-trait/feature-self-return-type.rs`
* `tests/ui/async-await/issues/issue-78600.rs`
* `tests/ui/impl-trait/capture-lifetime-not-in-hir.rs`

---

r? cjgillot on the impl (not much, just removing the feature gate)

I'm gonna mark this as FCP for T-lang and T-types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. perf-regression Performance regression. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inherent async fn returning Self treats type's lifetime parameters as 'static