Inherent async fn returning `Self` treats type's lifetime parameters as `'static` #61949

Arnavion · 2019-06-19T04:11:56Z

rustc 1.37.0-nightly (2887008e0 2019-06-12) and playground's 2019-06-17 b25ee644971a168287ee

#![feature(async_await)]

pub struct Foo<'a> {
    pub bar: &'a i32,
}

impl<'a> Foo<'a> {
    pub async fn new(bar: &'a i32) -> Self {
        Foo {
            bar
        }
    }
}

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
 --> src/lib.rs:8:22
  |
8 |     pub async fn new(bar: &'a i32) -> Self {
  |                      ^^^
  |
note: first, the lifetime cannot outlive the lifetime 'a as defined on the impl at 7:6...
 --> src/lib.rs:7:6
  |
7 | impl<'a> Foo<'a> {
  |      ^^
  = note: ...so that the expression is assignable:
          expected &i32
             found &'a i32
  = note: but, the lifetime must be valid for the static lifetime...
  = note: ...so that the types are compatible:
          expected Foo<'_>
             found Foo<'static>

It works to either change the signature to take bar: &'static i32, or to change the body of the fn to use a static borrow like bar: &5. So the compiler really does want the function to return a Foo<'static>, even though Self is a Foo<'a>

The workaround is to not use Self:

-    pub async fn new(bar: &'a i32) -> Self {
+    pub async fn new(bar: &'a i32) -> Foo<'a> {

Mentoring notes: See notes here on Zulip.

The text was updated successfully, but these errors were encountered:

Arnavion · 2019-06-19T04:18:59Z

jebrosen pointed out on IRC that it also happens with RPIT in general, so this might be the same as #53613 (from async fn expanding to -> impl Future<Output = ...>)

nikomatsakis · 2019-07-05T18:36:23Z

As I pointed out in #53613, there is a more dangerous variant of this involving projections:

#![feature(async_await)]

pub trait HasItem<'a> {
    type Item;
}

// Does not compile:
async fn example1<'a, T: HasItem<'a>>(item: T::Item) -> T::Item {
    item
}


fn main() { }

Note that this version compiles just fine

async fn example1<'a, T: HasItem<'a>>(item: T::Item) -> <T as HasItem<'a>>::Item {
    item
}

nikomatsakis · 2019-07-05T18:38:27Z

There are some forward compatibility concerns here. In particular, callers can (somewhat incorrectly) assume that the returned futures are 'static. In other words, the callee is committing to returning a future that does not capture 'a in these examples -- right now, that typically means callee doesn't compile, but it could happen that the callee happens to compile, and then callers would be able to use this future in places where 'a is out of scope. If we later fix this bug, those callers will stop compiling.

nikomatsakis · 2019-07-05T18:40:37Z

The bug involving Self is rather straightforward (but grungy) to fix. It's also possible for us to, temporarily, just create an error if you return a type mentioning Self in an async fn.

But fixing the T::Item case is rather more complex. We don't know until type-checking that T::Item resolves to <T as HasItem<'a>>::Item. And, if I had my druthers, we might not know until even later than that. I suppose we could do a forward compatibility restriction of some kind.

nikomatsakis · 2019-07-09T17:53:03Z

An example of something which compiles now but I think should not (and would not, if this were fixed):

#![feature(async_await)]

pub struct Foo<'a> {
    pub bar: &'a i32,
}

impl<'a> Foo<'a> {
    pub async fn new(_bar: &'a i32) -> Self {
        Foo {
            bar: &22
        }
    }
}

async fn foo() {
  let x = {
    let bar = 22;
    Foo::new(&bar).await
  };
  drop(x);
}

fn main() { }

the problem here is that the type of x should be Foo<'bar> -- i.e., it should be constrained to the lifetime of bar, but it is actually Foo<'static>.

nikomatsakis · 2019-07-09T18:24:15Z

I left some notes on how to fix this on Zulip

davidtwco · 2019-07-09T18:37:38Z

@rustbot claim

nikomatsakis · 2019-07-09T20:36:43Z

OK so having thought this over I have come to the conclusion that the right behavior here is to give an error if people use Self or T::Foo projections from within an impl Trait such that they wind up "inheriting lifetimes". It's not an ideal solution but it'll permit many uses while forbidding the cases that are causing trouble and giving us time to fix properly.

The way I think we should do this:

During type checking, there is a check_opaque function that currently checks for cyclic opaque types.

In the case of existential types whose origin is either ReturnImplTrait or AsyncFn, we want to enforce this condition.

What we would do is to access the tcx.predicates_of the existential def-id. The predicates field of the result will contain the "bounds" of the impl trait (with the self-type being an opaque type). (Note that we don't want to inspect the parent, which will contain predicates defined in the enclosing scope.)

Actually, we may want to introduce a new query (existential_bounds_of or something) that just returns those predicates that tcx.predicates_of would return..? I guess it's fine to invoke tcx.predicates_of, it just feels a touch fragile. Ah well, that's what tests are for. =)

Anyway, we basically iterate over that predicates field using a type visitor. In there, we look for references to regions that are ReEarlyBound but where the index indicates it is coming from the parent. We would report an error if we found any such region.

One thing to watch out for though while we are visiting the type. If we have something like impl Foo, that will be represented by a predicate O: Foo where O is the opaque identity type. Such a type is constructed here:

rust/src/librustc_typeck/collect.rs

Lines 1924 to 1925 in b8ec4c4

    
           let substs = InternalSubsts::identity_for_item(tcx, def_id); 
        
           let opaque_ty = tcx.mk_opaque(def_id, substs);

Such a type will contain the illegal regions and we need to ignore it. We basically want to ignore instances of the self-type when visiting -- i.e., we can construct the opaque identity type (call it opaque_identity_ty) and override visit_ty to skip visiting the contents if ty == opaque_identity_ty.

@nikomatsakis

typeck: Prohibit RPIT types that inherit lifetimes Part of #61949. This PR prohibits return position `impl Trait` types that "inherit lifetimes" from the parent scope. The intent is to forbid cases that are challenging until they can be addressed properly. cc @nikomatsakis

@nikomatsakis

typeck: Prohibit RPIT types that inherit lifetimes Part of #61949. This PR prohibits return position `impl Trait` types that "inherit lifetimes" from the parent scope. The intent is to forbid cases that are challenging until they can be addressed properly. cc @nikomatsakis

Centril · 2019-08-08T19:20:19Z

From Lang team meeting: @cramertj will do a more targeted fix for async fn based on #62849. If crater completes in time then we can generalize #62849.

@nikomatsakis

…imes, r=nikomatsakis typeck: Prohibit RPIT types that inherit lifetimes Part of rust-lang#61949. This PR prohibits return position `impl Trait` types that "inherit lifetimes" from the parent scope. The intent is to forbid cases that are challenging until they can be addressed properly. cc @nikomatsakis

@nikomatsakis

…imes, r=nikomatsakis typeck: Prohibit RPIT types that inherit lifetimes Part of rust-lang#61949. This PR prohibits return position `impl Trait` types that "inherit lifetimes" from the parent scope. The intent is to forbid cases that are challenging until they can be addressed properly. cc @nikomatsakis

@nikomatsakis

…imes, r=nikomatsakis typeck: Prohibit RPIT types that inherit lifetimes Part of rust-lang#61949. This PR prohibits return position `impl Trait` types that "inherit lifetimes" from the parent scope. The intent is to forbid cases that are challenging until they can be addressed properly. cc @nikomatsakis

tmandry · 2020-02-12T03:14:01Z

I did some investigating, leaving notes here.

The reason the example above does not emit the new error message is because the compile fails during type collection and never makes it to type checking. This is a bug fixed by #68884.

With the above patch applied, we now get the new error message that was introduced by #62849. We also get the more confusing error message about inferring appropriate lifetimes (this happens while checking the function body), but I'm not sure if we can avoid that without a lot of effort.

Agreed that the error message could be better. I think we should give this error a hint ("try spelling out the type instead of using Self"), along with an error code / longer description saying that this is a limitation of the compiler.

tmandry · 2020-02-18T18:16:19Z

Hello from triage.. downgrading from Focus since there is no forward-compat issue. We may keep this as OnDeck for now to track the diagnostics improvement.

tmandry · 2020-02-19T00:25:39Z

Opened #69276 to track the diagnostics improvement separately.

Kimundi · 2021-03-03T12:01:48Z

The situation here is still confusing, as the explanation for the error message, https://doc.rust-lang.org/nightly/error-index.html#E0760, talks about async, but the issue also arised with non-async code:

pub struct Foo<'a>(&'a ());

impl<'a> Foo<'a> {
    pub fn works<T>(self) -> impl FnOnce(T) -> Foo<'a> {
        move |_| todo!()
    }
    pub fn errors<T>(self) -> impl FnOnce(T) -> Self {
        move |_| todo!()
    }
}

EDIT: Actually, it talks about both async and impl Trait, it just a very terse sentence that makes it easy to glance over that, while only having example code that uses async.

Xiretza · 2022-04-24T11:56:28Z

#91403 was reverted (by #94088), I think this should be reopened.

Stabilize `impl_trait_projections` Closes rust-lang#115659 ## TL;DR: This allows us to mention `Self` and `T::Assoc` in async fn and return-position `impl Trait`, as you would expect you'd be able to. Some examples: ```rust #![feature(return_position_impl_trait_in_trait, async_fn_in_trait)] // (just needed for final tests below) // ---------------------------------------- // struct Wrapper<'a, T>(&'a T); impl Wrapper<'_, ()> { async fn async_fn() -> Self { //^ Previously rejected because it returns `-> Self`, not `-> Wrapper<'_, ()>`. Wrapper(&()) } fn impl_trait() -> impl Iterator<Item = Self> { //^ Previously rejected because it mentions `Self`, not `Wrapper<'_, ()>`. std::iter::once(Wrapper(&())) } } // ---------------------------------------- // trait Trait<'a> { type Assoc; fn new() -> Self::Assoc; } impl Trait<'_> for () { type Assoc = (); fn new() {} } impl<'a, T: Trait<'a>> Wrapper<'a, T> { async fn mk_assoc() -> T::Assoc { //^ Previously rejected because `T::Assoc` doesn't mention `'a` in the HIR, // but ends up resolving to `<T as Trait<'a>>::Assoc`, which does rely on `'a`. // That's the important part -- the elided trait. T::new() } fn a_few_assocs() -> impl Iterator<Item = T::Assoc> { //^ Previously rejected for the same reason [T::new(), T::new(), T::new()].into_iter() } } // ---------------------------------------- // trait InTrait { async fn async_fn() -> Self; fn impl_trait() -> impl Iterator<Item = Self>; } impl InTrait for &() { async fn async_fn() -> Self { &() } //^ Previously rejected just like inherent impls fn impl_trait() -> impl Iterator<Item = Self> { //^ Previously rejected just like inherent impls [&()].into_iter() } } ``` ## Technical: Lifetimes in return-position `impl Trait` (and `async fn`) are duplicated as early-bound generics local to the opaque in order to make sure we are able to substitute any late-bound lifetimes from the function in the opaque's hidden type. (The [dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html#aside-opaque-lifetime-duplication) has a small section about why this is necessary -- this was written for RPITITs, but it applies to all RPITs) Prior to rust-lang#103491, all of the early-bound lifetimes not local to the opaque were replaced with `'static` to avoid issues where relating opaques caused their *non-captured* lifetimes to be related. This `'static` replacement led to strange and possibly unsound behaviors (rust-lang#61949 (comment)) (rust-lang#53613) when referencing the `Self` type alias in an impl or indirectly referencing a lifetime parameter via a projection type (via a `T::Assoc` projection without an explicit trait), since lifetime resolution is performed on the HIR, when neither `T::Assoc`-style projections or `Self` in impls are expanded. Therefore an error was implemented in rust-lang#62849 to deny this subtle behavior as a known limitation of the compiler. It was attempted by `@cjgillot` to fix this in rust-lang#91403, which was subsequently unlanded. Then it was re-attempted to much success (🎉) in rust-lang#103491, which is where we currently are in the compiler. The PR above (rust-lang#103491) fixed this issue technically by *not* replacing the opaque's parent lifetimes with `'static`, but instead using variance to properly track which lifetimes are captured and are not. The PR gated any of the "side-effects" of the PR behind a feature gate (`impl_trait_projections`) presumably to avoid having to involve T-lang or T-types in the PR as well. `@cjgillot` can clarify this if I'm misunderstanding what their intention was with the feature gate. Since we're not replacing (possibly *invariant*!) lifetimes with `'static` anymore, there are no more soundness concerns here. Therefore, this PR removes the feature gate. Tests: * `tests/ui/async-await/feature-self-return-type.rs` * `tests/ui/impl-trait/feature-self-return-type.rs` * `tests/ui/async-await/issues/issue-78600.rs` * `tests/ui/impl-trait/capture-lifetime-not-in-hir.rs` --- r? cjgillot on the impl (not much, just removing the feature gate) I'm gonna mark this as FCP for T-lang and T-types.

Stabilize `impl_trait_projections` Closes #115659 ## TL;DR: This allows us to mention `Self` and `T::Assoc` in async fn and return-position `impl Trait`, as you would expect you'd be able to. Some examples: ```rust #![feature(return_position_impl_trait_in_trait, async_fn_in_trait)] // (just needed for final tests below) // ---------------------------------------- // struct Wrapper<'a, T>(&'a T); impl Wrapper<'_, ()> { async fn async_fn() -> Self { //^ Previously rejected because it returns `-> Self`, not `-> Wrapper<'_, ()>`. Wrapper(&()) } fn impl_trait() -> impl Iterator<Item = Self> { //^ Previously rejected because it mentions `Self`, not `Wrapper<'_, ()>`. std::iter::once(Wrapper(&())) } } // ---------------------------------------- // trait Trait<'a> { type Assoc; fn new() -> Self::Assoc; } impl Trait<'_> for () { type Assoc = (); fn new() {} } impl<'a, T: Trait<'a>> Wrapper<'a, T> { async fn mk_assoc() -> T::Assoc { //^ Previously rejected because `T::Assoc` doesn't mention `'a` in the HIR, // but ends up resolving to `<T as Trait<'a>>::Assoc`, which does rely on `'a`. // That's the important part -- the elided trait. T::new() } fn a_few_assocs() -> impl Iterator<Item = T::Assoc> { //^ Previously rejected for the same reason [T::new(), T::new(), T::new()].into_iter() } } // ---------------------------------------- // trait InTrait { async fn async_fn() -> Self; fn impl_trait() -> impl Iterator<Item = Self>; } impl InTrait for &() { async fn async_fn() -> Self { &() } //^ Previously rejected just like inherent impls fn impl_trait() -> impl Iterator<Item = Self> { //^ Previously rejected just like inherent impls [&()].into_iter() } } ``` ## Technical: Lifetimes in return-position `impl Trait` (and `async fn`) are duplicated as early-bound generics local to the opaque in order to make sure we are able to substitute any late-bound lifetimes from the function in the opaque's hidden type. (The [dev guide](https://rustc-dev-guide.rust-lang.org/return-position-impl-trait-in-trait.html#aside-opaque-lifetime-duplication) has a small section about why this is necessary -- this was written for RPITITs, but it applies to all RPITs) Prior to #103491, all of the early-bound lifetimes not local to the opaque were replaced with `'static` to avoid issues where relating opaques caused their *non-captured* lifetimes to be related. This `'static` replacement led to strange and possibly unsound behaviors (rust-lang/rust#61949 (comment)) (rust-lang/rust#53613) when referencing the `Self` type alias in an impl or indirectly referencing a lifetime parameter via a projection type (via a `T::Assoc` projection without an explicit trait), since lifetime resolution is performed on the HIR, when neither `T::Assoc`-style projections or `Self` in impls are expanded. Therefore an error was implemented in #62849 to deny this subtle behavior as a known limitation of the compiler. It was attempted by `@cjgillot` to fix this in #91403, which was subsequently unlanded. Then it was re-attempted to much success (🎉) in #103491, which is where we currently are in the compiler. The PR above (#103491) fixed this issue technically by *not* replacing the opaque's parent lifetimes with `'static`, but instead using variance to properly track which lifetimes are captured and are not. The PR gated any of the "side-effects" of the PR behind a feature gate (`impl_trait_projections`) presumably to avoid having to involve T-lang or T-types in the PR as well. `@cjgillot` can clarify this if I'm misunderstanding what their intention was with the feature gate. Since we're not replacing (possibly *invariant*!) lifetimes with `'static` anymore, there are no more soundness concerns here. Therefore, this PR removes the feature gate. Tests: * `tests/ui/async-await/feature-self-return-type.rs` * `tests/ui/impl-trait/feature-self-return-type.rs` * `tests/ui/async-await/issues/issue-78600.rs` * `tests/ui/impl-trait/capture-lifetime-not-in-hir.rs` --- r? cjgillot on the impl (not much, just removing the feature gate) I'm gonna mark this as FCP for T-lang and T-types.

jonas-schievink added A-async-await Area: Async & Await C-bug Category: This is a bug. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jun 19, 2019

Centril added the AsyncAwait-Unclear label Jun 19, 2019

nikomatsakis added AsyncAwait-Polish Async-await issues that are part of the "polish" area and removed AsyncAwait-Unclear labels Jul 2, 2019

nikomatsakis added E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. and removed E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. labels Jul 9, 2019

rustbot assigned davidtwco Jul 9, 2019

davidtwco mentioned this issue Jul 21, 2019

typeck: Prohibit RPIT types that inherit lifetimes #62849

Merged

Centril added the F-async_await label Jul 28, 2019

This was referenced Aug 1, 2019

[Stabilization] async/await MVP #62149

Closed

Stabilize async_await in Rust 1.39.0 #63209

Merged

Centril assigned cramertj Aug 8, 2019

tmandry added AsyncAwait-OnDeck and removed AsyncAwait-Polish Async-await issues that are part of the "polish" area P-high High priority labels Feb 18, 2020

tmandry moved this from In progress to To do in wg-async work Feb 18, 2020

tmandry mentioned this issue Feb 19, 2020

Improve error message for "return type cannot contain a projection or Self..." #69276

Closed

tmandry removed the AsyncAwait-OnDeck label Feb 19, 2020

tmandry added the P-medium Medium priority label Mar 3, 2020

tmandry removed this from To do in wg-async work Mar 3, 2020

gliderkite mentioned this issue Oct 31, 2020

Improve error message for "return type cannot contain a projection or Self..." #78600

Closed

jonas-schievink removed the F-async_await label Aug 29, 2021

tmandry removed their assignment Sep 16, 2021

cjgillot mentioned this issue Nov 30, 2021

Inherit lifetimes for async fn instead of duplicating them. #91403

Merged

ibraheemdev mentioned this issue Dec 29, 2021

Implementation of trait is not general enough when boxing future returning <T as Foo<'a>>::Foo #92415

Open

bors closed this as completed in 3cfa4de Feb 13, 2022

tmandry reopened this May 3, 2022

compiler-errors mentioned this issue Sep 8, 2023

Stabilize impl_trait_projections #115659

Merged

github-actions bot mentioned this issue Apr 9, 2024

rust/1.77.2 package update wolfi-dev/os#16567

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inherent async fn returning `Self` treats type's lifetime parameters as `'static` #61949

Inherent async fn returning `Self` treats type's lifetime parameters as `'static` #61949

Arnavion commented Jun 19, 2019 •

edited by rustbot

Arnavion commented Jun 19, 2019 •

edited

nikomatsakis commented Jul 5, 2019 •

edited

nikomatsakis commented Jul 5, 2019

nikomatsakis commented Jul 5, 2019

nikomatsakis commented Jul 9, 2019

nikomatsakis commented Jul 9, 2019

davidtwco commented Jul 9, 2019

nikomatsakis commented Jul 9, 2019

Centril commented Aug 8, 2019

tmandry commented Feb 12, 2020

tmandry commented Feb 18, 2020

tmandry commented Feb 19, 2020

Kimundi commented Mar 3, 2021 •

edited

Xiretza commented Apr 24, 2022 •

edited

Inherent async fn returning Self treats type's lifetime parameters as 'static #61949

Inherent async fn returning Self treats type's lifetime parameters as 'static #61949

Comments

Arnavion commented Jun 19, 2019 • edited by rustbot

Arnavion commented Jun 19, 2019 • edited

nikomatsakis commented Jul 5, 2019 • edited

nikomatsakis commented Jul 5, 2019

nikomatsakis commented Jul 5, 2019

nikomatsakis commented Jul 9, 2019

nikomatsakis commented Jul 9, 2019

davidtwco commented Jul 9, 2019

nikomatsakis commented Jul 9, 2019

Centril commented Aug 8, 2019

tmandry commented Feb 12, 2020

tmandry commented Feb 18, 2020

tmandry commented Feb 19, 2020

Kimundi commented Mar 3, 2021 • edited

Xiretza commented Apr 24, 2022 • edited

Inherent async fn returning `Self` treats type's lifetime parameters as `'static` #61949

Inherent async fn returning `Self` treats type's lifetime parameters as `'static` #61949

Arnavion commented Jun 19, 2019 •

edited by rustbot

Arnavion commented Jun 19, 2019 •

edited

nikomatsakis commented Jul 5, 2019 •

edited

Kimundi commented Mar 3, 2021 •

edited

Xiretza commented Apr 24, 2022 •

edited