Stabilize derive(CoercePointee)

[dingxiangfei2009]

What is being proposed for stabilization

This stabilization report concerns the macro specified by RFC3621, now called CoercePointee.

This macro enables the derivation of trait implementation of Unsize, CoerceUnsize and DispatchFromDyn traits, both of which are unstable at the moment, under a well-defined and structured schema, so that the users who would like to allow their custom type to admit unsizing coercion and use of dyn DST in one of its generic type parameter in a safe way. The greatest use case of this macro is to allow users to equip their types with this behaviour without dependence on alloc crate, or when alloc is not available and a custom implementation of pointer-like data structure in the likes of Rc and Arc is highly desirable. The most prominent example is the kernel crate by Rust-for-Linux, which would greatly benefit from this macro due to reduction of boilerplate, and reduce the project's dependence on unstable features behind Unsize, CoerceUnsize and DispatchFromDyn.

In a nutshell, derive(CoercePointee) derives the implementation of CoerceUnsize and DispatchFromDyn traits on the target struct definition with all the necessary trait bounds. It identifies one generic type variable in the generic list of the target struct, that is either uniquely annotated with the #[pointee] attribute or is the only type variable among the generics. This identified generic, which is called source type in this document, will be treated as the target of unsizing operation and dyn trait object dispatch. Correspondingly, the resultant type after the unsizing coercion will be called target type in this document. In case additional trait bounds applies on the source type, the said trait bounds will be migrated to both the source type and the target type in the trait implementation.

Out of necessity of unsizing coercion, the macro requires the struct to adopt the repr(transparent) layout. The only data field in the struct definition, which is required by this layout, should also implement CoerceUnsize and DispatchFromDyn at the moment, or at least conform to requirement of the unsizing coercion and trait object dispatching protocol.

As an example, here is how the generated code would look like after expanding the macro. The following is a user source that invokes this macro.

pub trait MyTrait<T: ?Sized> {}

#[derive(core::marker::CoercePointee)]
#[repr(transparent)]
pub struct MyPointer2<'a, Y, Z: MyTrait<T>, #[pointee] T: ?Sized + MyTrait<T>, X: MyTrait<T> = ()>
where
    Y: MyTrait<T>,
{
    data: &'a mut T,
    x: core::marker::PhantomData<X>,
}

This is the expansion result. The source type T after the unsizing coercion is assumed to be __S and the trait bounds requested by the original definition are migrated for __S as well throughtout.

#[repr(transparent)]
pub struct MyPointer2<
    'a, Y, Z: MyTrait<T>,
    #[pointee] T: ?Sized + MyTrait<T>,
    X: MyTrait<T> = ()
>
where Y: MyTrait<T>
{
    data: &'a mut T,
    x: core::marker::PhantomData<X>,
}

#[automatically_derived]
impl<
    'a, Y,
    Z: MyTrait<T> + MyTrait<__S>,
    T: ?Sized + MyTrait<T> + ::core::marker::Unsize<__S>,
    __S: ?Sized + MyTrait<__S>,
    X: MyTrait<T> + MyTrait<__S>
>
::core::ops::DispatchFromDyn<MyPointer2<'a, Y, Z, __S, X>> for MyPointer2<'a, Y, Z, T, X>
where Y: MyTrait<T>, Y: MyTrait<__S> {
}

#[automatically_derived]
impl<
    'a, Y,
    Z: MyTrait<T> + MyTrait<__S>,
    T: ?Sized + MyTrait<T> + ::core::marker::Unsize<__S>,
    __S: ?Sized + MyTrait<__S>,
    X: MyTrait<T> + MyTrait<__S>
>
::core::ops::CoerceUnsized<MyPointer2<'a, Y, Z, __S, X>> for MyPointer2<'a, Y, Z, T, X>
where Y: MyTrait<T>, Y: MyTrait<__S> {
}

Future interaction

This stabilization of this macro does not require the stabilization of any of the unstable traits mentioned. This macro aims at stabilising a subset of features thereof, which has widely proliferated in the ecosystem, without blocking further development of DSTs and unsizing coercion. In case of change in the design of these language features, as long as basic provision of the coercion operation remains the same, the macro should be updated to use the new facility inside its implementation when necessary, to preserve only the semantics and capabilities that this macro exposes users to.

Tracking:

• Tracking issue for RFC 3621: derive(CoercePointee) #123430

cc @rust-lang/lang

[rustbot]

r? @joboet

rustbot has assigned @joboet.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[dingxiangfei2009]

@rustbot ready

• Documentation is updated to reflect the requirements as laid out by the RFC

[dingxiangfei2009]

@rustbot label +F-derive_smart_pointer

[dingxiangfei2009]

Question:

Do we need to condition on bootstrap? No standard library item depends on this macro yet.

[Darksonn]

The RFC for this feature said that it depends on the arbitrary self types feature, but it is not a full blocker to stabilizing this feature. There are two things that the macro enables:

The first thing is coercions from MySmartPtr<MyType> to MySmartPtr<dyn MyTrait>.

The second thing is making traits such as this one object safe:

trait MyTrait {
    fn func(self: MySmartPtr<Self>);
}

Without arbitrary self types, it's not legal to declare this trait because MySmartPtr is not a valid self type. However, the coercions from MySmartPtr<MyType> to MySmartPtr<dyn MyTrait> still work, and custom smart pointers are still useful, e.g. given an MyArc<dyn MyTrait> that derefs to &dyn MyTrait, you can call trait methods on MyTrait that take &self just fine.

So if we stabilize the macro now, then we gain the ability to make coercions and call trait methods via deref, but we don't gain the ability to declare traits that take the smart pointer without a deref coercion. We would gain that ability once arbitrary self types are stabilized.

[traviscross]

@rfcbot fcp merge

Thanks @dingxiangfei2009. We talked about this in our triage call today. Let's ship it.

[rfcbot]

Team member @traviscross has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

Given the discussion here, it seems safe to say that we should re-FCP this with t-types. I'll recheck the boxes of everyone on lang.

@rustbot labels -finished-final-comment-period +T-types

[traviscross]

@rfcbot fcp merge

[compiler-errors]

@rfcbot fcp cancel

[rfcbot]

@compiler-errors proposal cancelled.

[compiler-errors]

@rfcbot fcp merge

[rfcbot]

Team member @compiler-errors has proposed to merge this. The next step is review by the rest of the tagged team members:

• @BoxyUwU
• @compiler-errors
• @jackh726
• @joshtriplett
• @lcnr
• @nikomatsakis
• @oli-obk
• @scottmcm
• @spastorino
• @tmandry
• @traviscross

Concerns:

• CoerceUnsized builtin checks do not handle aliases (Stabilize derive(CoercePointee) #133820 (comment))
• DispatchFromDyn builtin checks are weaker than CoerceUnsized (Stabilize derive(CoercePointee) #133820 (comment))
• leaking unstable Unsize impls (Stabilize derive(CoercePointee) #133820 (comment))
• not properly checking repr(transparent) CoercePointee doesn't tightly enforce the restriction to #[repr(transparent)]-only wrappers, if other proc-macros are involved #135206 (Stabilize derive(CoercePointee) #133820 (comment))

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

@rfcbot reviewed

[compiler-errors]

(Noting here that I started the merge so my box is checked and rfcbot doesn't give a way to uncheck my box afaict, but I'd still like to see the type system and compiler side of this stabilization -- not just the macro itself but the traits that it exposes indirectly -- noted clearly in the stabilization report before this merges. I won't file this as a concern just yet, because I don't want to immediately block this, but if anyone else on T-types feels more strongly they can file a concern before me.)

[lcnr]

rfcbot doesn't give a way to uncheck my box afaict

I'd expect just manually editing its comment to work 🤔 😁

[lcnr]

Lookiong at the unstable emitted impls, we've got:

CoerceUnsized

query coerce_unsized_info checks all requirements of that trait

rust/compiler/rustc_hir_analysis/src/coherence/builtin.rs

Line 342 in 66bb586

pub(crate) fn coerce_unsized_info<'tcx>(

DispatchFromDyn

Again, the requirements are checked in

rust/compiler/rustc_hir_analysis/src/coherence/builtin.rs

Line 199 in 66bb586

fn visit_implementation_of_dispatch_from_dyn(checker: &Checker<'_>) -> Result<(), ErrorGuaranteed> {

---

Now, given that all invariants of the trait are actually checked by the compiler, this derive is sound as long as these two traits are :3

Both traits don't have builtin impls, so there's no worry about incorrect overlap. They simply recursively walk ADTs until they reach a ptr, at which point we rely on Unsize (at least for CoerceUnsized) which checks the pointee. The Unsize check does not depend on CoerceUnsized.

I assume that an impl which soundly works for Arc will also soundly work for other user-defined types.

One concern is that we add a requirement T: Unsize<U> even if the pointer is not necessarily pointing to T directly, e.g. we leak some of the existing Unsize impls

#![feature(derive_coerce_pointee)]
use std::marker::PhantomData;
use std::marker::CoercePointee; 
#[derive(CoercePointee)] #[repr(transparent)]
struct MyPointer<'a, T, #[pointee] U: ?Sized> {
    ptr: &'a (T, U),
}

Leaking currently unstable impls:

rust/compiler/rustc_hir_typeck/src/coercion.rs

Lines 716 to 724 in 66bb586

	if has_unsized_tuple_coercion && !self.tcx.features().unsized_tuple_coercion() {
	feature_err(
	&self.tcx.sess,
	sym::unsized_tuple_coercion,
	self.cause.span,
	"unsized tuple coercion is not stable enough for use and is subject to change",
	)
	.emit();
	}

[lcnr]

#![feature(derive_coerce_pointee)]
use std::marker::PhantomData;
use std::marker::CoercePointee; 
#[derive(CoercePointee)] #[repr(transparent)]
struct MyPointer<'a, T, #[pointee] U: ?Sized> {
    ptr: &'a (T, U),
}

fn main() {
    let x = MyPointer { ptr: &(1u32, 1u32) };
    let _: MyPointer<u32, dyn Send> = x;
}

@rfcbot concern leaking unstable Unsize impls

[nikomatsakis]

@compiler-errors my preferred way to manage that is not unchecking my box but filing a concern

[compiler-errors]

Yeah, at this point @lcnr has already begun to do a good job at characterizing the types-level surface of what is being stabilized here, and funny enough that they found an interesting quirk in the process :D

[BoxyUwU]

@rfcbot concern not properly checking repr(transparent) #135206

[BoxyUwU]

DispatchFromDyn doesn't require the type parameter to only be used in one field and potentially phantomdata, instead it requires one field and arbitarily many ZSTs playground. This in theory means that dispatch from dyn impls can result in effectively arbitrary transmutes from MyZST<dyn Trait> to MyZST<Foo>.

In practice I don't think this is exploitable, regardless I don't think we should stabilize the ability for semi-user-written impls or trait bounds to be written involving this trait until the builtin checks are brought in line with CoerceUnsized.

edit: thanks to @steffahn for pointing this out

@rfcbot concern DispatchFromDyn builtin checks are weaker than CoerceUnsized

[BoxyUwU]

The CoerceUnsized builtin checks that the parameter being unsized is only used in PhantomDatas and one field does not handle aliases and just directly checks if the Ty is a PhantomData. This will have to be fixed before stabilizing lazy_type_alias so we may as well just handle it from the beginning. See the following two playground examples for things that don't compile but probably ought to before we stabilize any form of user written impls for the trait:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=35e9d30ae8a278601de5d8a45bc40417
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=850f363579a53b61b88c6b02a502c31b

@rfcbot concern CoerceUnsized builtin checks do not handle aliases

[BoxyUwU]

I've moved some of the concerns out to separate issues where people can assign themselves and Fixes #... them

[bors]

☔ The latest upstream changes (presumably #135286) made this pull request unmergeable. Please resolve the merge conflicts.