Deprecate uninitialized in favor of a new MaybeUninit type #1892

canndrew · 2017-02-09T16:14:06Z

My thoughts on what to do with uninitialized and !.

Edit: This has been updated to instead recommend deprecating uninitialized entirely. The old Inhabited trait proposal is now listed as an alternative.

Edit: FCP Proposal is #1892 (comment) (in the collapsed-by-default part).

canndrew · 2017-02-09T16:15:45Z

cc @nikomatsakis, @arielb1, @eddyb. I know y'all have some thoughts on what to do with all this.

strega-nil · 2017-02-09T16:54:02Z

Calling uninitialized is already extremely dangerous. This just seems like unnecessary complication. If you want to create a !, for some reason, whatever you're doing is probably UB anyways. At most, we might want to have a warning in the docs.

canndrew · 2017-02-09T17:24:55Z

@ubsan, what about the catch_unwind example in the RFC? That's based on a compiler bug that got created when feature(never_type) got switched on in the standard library.

Without this, any (otherwise correct) code that uses uninitialized::<T> is landmine waiting to go off when someone tries to set T = !.

nagisa · 2017-02-09T17:54:38Z

The compiler bug happened mostly because previously the T would become () due to ! not existing. I personally think this PR is going in the right direction (a-la Sized).

nagisa · 2017-02-09T17:56:35Z

text/0000-uninitialized-uninhabited.md

+
+This trait is automatically implemented for all inhabited types.
+
+Change the type of `uninitialized` to:


This conflicts somewhat with Inhabited being an auto-trait (a-la Sized), or rather T is already Inhabited unless specified otherwise (i.e. T: ?Inhabited)

Question: Does ! impl Sized? Does that even make sense?

@mark-i-m Yes, size_of::<!>() == 0, More generally though, for any given size every element of ! has that size. Similarly, any two elements of ! have the same size. These are both trivially true since ! has no elements.

So we are defining size_of::<T>() == n iff for all values v of T that v takes n bits to express, and since there are no values of T = !, this vacuously true?

Basically, yeah. There's two subtly different definitions you could give of Sized but ! satisfies both of them vacuously.

nagisa · 2017-02-09T17:58:04Z

text/0000-uninitialized-uninhabited.md

+This could be a rather large breaking change depending on how many people are
+currently calling `uninitialized::<T>` with a generic `T`. However all such
+code is already somewhat future-incompatible as it will malfunction (or panic)
+if used with `!`.


If Inhabited is auto-trait, like Sized, I do not see how that’s problematic in any way.

nagisa · 2017-02-09T17:59:10Z

text/0000-uninitialized-uninhabited.md

+The author of the crate may expect this change to be private and its effects
+contained to within the crate. But in making this change they've also stopped
+exporting the `Inhabited` impl, causing potential breakages for downstream
+users.


Again, that’s pretty much the same story with Sized. I.e. you cannot change pub struct Sized { a: [u8; 42] } to pub struct Sized { a: [u8] }.

arielb1 · 2017-02-09T17:59:13Z

We pretty much decided in the design sprint to deprecate mem::uninitialized in favor of using a MaybeUninitialized<T> union in the standard library.

canndrew · 2017-02-09T18:01:46Z

@arielb1 Ah okay. I figured that might too radical for now so I intended this RFC as a (possibly temporary) middle ground.

canndrew · 2017-02-09T18:06:15Z

@nagisa Sorry if my wording is confusing. Inhabited isn't intended to be an auto-trait like Sized in the sense that you have to explicitly opt-out with ?Inhabited. That would create unnecessary restrictions on using uninhabited types. ! and Void should be usable pretty much anywhere, it's only really uninitialized which is problematic.

nagisa · 2017-02-09T18:07:01Z

it's only really uninitialized which is problematic.

That’s not true. ptr::read (and probably a number of other functions) has the same problem as uninitialized and we ain’t deprecating that, so an auto-trait is still worthwhile IMO:

fn main() {
let x: *const ! = &0 as *const _ as *const _; // imagine this comes as an generic argument from somewhere
let z: ! = unsafe {
    ::std::mem::read(x) // boom bam blamma
};
}

canndrew · 2017-02-09T18:16:47Z

Ah true, I hadn't thought of ptr::read. transmute has the same problem but I've made a seperate RFC for that. I just went through the list of intrinsics and I couldn't find any others which return a generic T without also taking one as an argument.

I still think ?Inhabited would make uninhabited types almost unusable unless people added the ?Inhabited bound everywhere. It seems simpler and a lot less restrictive to just put T: Inhabited on ptr::read as well.

strega-nil · 2017-02-09T19:56:42Z

text/0000-uninitialized-uninhabited.md

+    match std::panic::catch_unwind(|| {
+        let val = f();
+        unsafe {
+            (*foo_ref).value = val;


This line is broken for types with Drop impls. It should be ptr::write(&mut (*foo_ref).value, val)

Hmm, I thought we settled on different rules for overwriting union fields for some reason. Thanks for the catch!

mark-i-m · 2017-02-10T05:01:32Z

text/0000-uninitialized-uninhabited.md

+```
+
+Yet calling this function does not diverge! It just breaks everything then eats
+your laundry instead.


Somehow, this seems preferable to folding my laundry at the moment...

mark-i-m · 2017-02-10T05:02:28Z

text/0000-uninitialized-uninhabited.md

+if used with `!`.
+
+Another drawback is that the `Inhabited` trait leaks private information about
+types. Consider a type with the following definition:


I am not convinced this is more serious than already-existing leaks... for example, you can already find out the size of a type. Is there any fundamental difference with this?

Not really, it's the same as Sized in this regard.

Ok, that makes sense 😄

mark-i-m · 2017-02-10T05:02:48Z

text/0000-uninitialized-uninhabited.md

+Ideally, Rust's type system should have a way of talking about initializedness
+statically. In the past there have been proposals for new pointer types which
+could safely handle uninitialized data. We should seriously consider pursuing
+one of these proposals.


👍 I totally agree! This would make static muts much more powerful while still being safe.

ranma42 · 2017-02-10T08:34:02Z

@canndrew could you add the alternative from the design sprint? (add MaybeUninit<T> and deprecate mem::uninitialized)

I find it compelling, as it removes some magic from the compiler (no special checks or automatic traits for size/inhabitedness) and lets the type system just do "its thing".

ranma42 · 2017-02-10T08:38:00Z

Sorry, I just realised the option of completely deprecating mem::uninitialized is mentioned as a future direction. While it is certainly possible to make this change in smaller steps, I think it might be very interesting to assess the potential disruption of the "direct" change (would a crater run be sufficient for this?). If deemed possible, I think that would be the best course of action.

ranma42 · 2017-02-10T08:43:33Z

Are there any plans to do the same to mem::zeroed, either through this RFC or through another one? I found no relevant results searching through the RFCs repo.

ghost · 2017-02-10T13:57:51Z

I still think ?Inhabited would make uninhabited types almost unusable unless people added the ?Inhabited bound everywhere. It seems simpler and a lot less restrictive to just put T: Inhabited on ptr::read as well.

I agree with what you're saying about ?Inhabited. I don't want that either. But then I think we'd need some kind of read_or_unreachable function without a bound anyway, otherwise Vec<T> will need an Inhabited bound (at least for remove and into_iter), which will infect a LOT of code. It's genuinely unreachable code to get to ptr::read from Vec<!> anyway because the only way to do it would be to push an uninhabited value onto it in the first place. Vec<!> is harmless, so having an Inhabited bound on it would be pointless.

I'd be (more) okay with Inhabited as a regular (compiler implemented) trait but it's completely unworkable to add it onto these functions as it would just break too much code, uninitialized, zeroed and read are all stable and there's probably a lot of code out there that calls them generically. I think the best we could do to make Inhabited as a trait work (though I don't think it will be used much, this is if you want the Inhabited trait in general rather than just for these cases) is to deprecate everything which should have the bound and then provide better alternatives: mem::MaybeInitialized<T> for mem::uninitialized and mem::zeroed. ptr::read_or_unreachable<T> and ptr::read_inhabited<T: Inhabited> for ptr::read/ptr::read_volatile/ptr::read_unaligned (those last two will become pretty verbose :(). Adding the bound (unless it's ?Inhabited) onto the functions is, unfortunately, a no-go because it will cause a lot of breakage.

Myself, I think the best route is:

Don't have an Inhabited trait, nor ?Inhabited. I don't think T: Inhabited is the best default to enforce because I think that cases where T = ! will cause issues are rare. It will lead to a lot of unnecessary restrictions on ! or will lead to ?Inhabited everywhere. Neither of those are ideal. Similarly, I don't think there are many cases where knowing that a type is inhabited is useful, but I'm not sure.
Deprecate uninitialized and zeroed and leave the signatures as they are. We could, perhaps, throw an error or lint at trans time if T = !, because it's highly likely to be a bug. I don't know if that's possible, but I think it's what is (was?) done for mem::transmute.
Add MaybeInitialized<T> to core::mem. Give it a zeroed constructor which calls memset on it. (maybe one day a const constructor by using [0u8; size_of::<T>()], but that's later). In the deprecation message for uninitialized and zeroed, direct them here.
Add a note onto ptr::read's docs that if T in uninhabited then the call must be unreachable.

djzin · 2017-02-12T17:01:54Z

Could this issue be solved with ! having no size? If the size of a type is defined to be ceil(log(n)) where n is the number of possible representations, then ! should have undefined size, or, if you like, "negative infinity" size. The main issue with this is that empty enums are already defined as being sized... and also "negative infinity" is not representable in a usize... but just a thought I had ;)

mark-i-m · 2017-02-12T18:35:40Z

I think the definition of the size of a type discussed in one of the earlier comments is nicer in that it gracefully handles ! On Feb 12, 2017 11:02 AM, "djzin" <notifications@github.com> wrote: Could this issue be solved with ! having no size? If the size of a type is defined to be ceil(log(n)) where n is the number of possible representations, then ! should have undefined size, or, if you like, "negative infinity" size. The main issue with this is that empty enums are already defined as being sized... and also "negative infinity" is not representable in a usize... but just a thought I had ;) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1892 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIazwIFjJbuaydpz5kxDgk8hWjqJSt9Xks5rbzsGgaJpZM4L8U6F> .

gnzlbg · 2018-08-11T20:12:16Z

@ubsan That makes sense. I guess one would need to drop the object instead, the storage after the drop should be uninit, so there is no need to write uninit again to it, although I guess one could do so.

RalfJung · 2018-08-17T16:53:43Z

honestly I still do not understand why creating &mut to uninitialized memory is automatically undefined behaviour and not reading from it

Well, there are other people saying the exact opposite. ;) We have to make a choice either way, but for both cases we have people arguing that this is clearly the more intuitive option.

I don't see how we could allow &T to point to uninitialized memory without making a pretty big backwards incompatible language change: all code that expects a &T and doesn't guard against T potentially being uninitialized would be at risk at best. Could it become broken or incorrect? No idea.

No, there would be no incompatible change. As @ubsan mentioned, we have two invariants at play here: Which assumptions the compiler can make for its optimizations, and which assumptions safe code can make for values it sees. There is no a priori reason to think these are the same, and in fact I think it is rather impossible to make them the same. I have a blogpost upcoming for this that should hopefully be done no later than Monday...

But just one example: A safe higher-order function which takes an argument f: fn(&i32) -> &i32 can assume that f can be called with any shared reference. So following your "violating safe code assumptions is insta-UB", it would be UB to have a function which does not have this property. On the other hand, many libraries have private functions not marked unsafe that actually are de-facto unsafe because they make extra assumptions that are guaranteed by the surrounding module. So your proposal makes all those libraries UB.

There are other problems as well. For example, the invariant that may be assumed by safe code is impossible to check for because it is frequently not computable (as in, would require solving the halting problem). I think we should have a definition of UB that can, at least in principle, be checked.

which has been proposed in rust-lang/rfcs#1892

55: internally use MaybeUninit r=japaric a=japaric which has been proposed in rust-lang/rfcs#1892 Co-authored-by: Jorge Aparicio <jorge@japaric.io>

rfcbot · 2018-08-19T09:40:34Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

Centril · 2018-08-19T10:20:22Z

Huzzah! This RFC has been merged!

Tracking issue: rust-lang/rust#53491

which has been proposed in rust-lang/rfcs#1892

@bluss

stabilize core parts of MaybeUninit and deprecate mem::uninitialized in the future (1.40.0). This is part of implementing rust-lang/rfcs#1892. Also expand the documentation a bit. This type is currently primarily useful when dealing with partially initialized arrays. In libstd, it is used e.g. in `BTreeMap` (with some unstable APIs that however can all be replaced, less ergonomically, by stable ones). What we stabilize should also be enough for `SmallVec` (Cc @bluss). Making this useful for structs requires rust-lang/rfcs#2582 or a commitment that references to uninitialized data are not insta-UB.

canndrew added 4 commits February 9, 2017 23:17

Add uninitialized/uninhabited RFC

ec78a80

Grammar fixes etc.

c5c75b6

More minor fixes

d449f85

Nah, that still doesn't read right

aae2944

nagisa reviewed Feb 9, 2017

View reviewed changes

strega-nil reviewed Feb 9, 2017

View reviewed changes

aturon added the T-lang Relevant to the language team, which will review and decide on the RFC. label Feb 9, 2017

aturon assigned nikomatsakis Feb 9, 2017

Fix MaybeUninit example

8741679

mark-i-m reviewed Feb 10, 2017

View reviewed changes

Mention deprecating uninitialized as an alternative

075b62c

XOSplicer mentioned this pull request Aug 16, 2018

Show a constant's virtual memory on validation errors rust-lang/rust#53325

Closed

japaric added a commit to rust-embedded/heapless that referenced this pull request Aug 19, 2018

internally use MaybeUninit

8baf6ec

which has been proposed in rust-lang/rfcs#1892

japaric mentioned this pull request Aug 19, 2018

internally use MaybeUninit rust-embedded/heapless#55

Merged

bors bot added a commit to rust-embedded/heapless that referenced this pull request Aug 19, 2018

Merge #55

9037d97

55: internally use MaybeUninit r=japaric a=japaric which has been proposed in rust-lang/rfcs#1892 Co-authored-by: Jorge Aparicio <jorge@japaric.io>

japaric mentioned this pull request Aug 19, 2018

Fix broken nightly: Const fn union workaround rust-embedded/heapless#52

Closed

rfcbot added finished-final-comment-period The final comment period is finished for this RFC. and removed final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. labels Aug 19, 2018

Centril mentioned this pull request Aug 19, 2018

Tracking issue for RFC 1892, "Deprecate uninitialized in favor of a new MaybeUninit type" rust-lang/rust#53491

Closed

3 tasks

RFC 1892

1b0ef45

Centril merged commit 21f887f into rust-lang:master Aug 19, 2018

XOSplicer pushed a commit to XOSplicer/heapless that referenced this pull request Aug 22, 2018

internally use MaybeUninit

b3e30ea

which has been proposed in rust-lang/rfcs#1892

ghost mentioned this pull request Sep 25, 2018

Allocate smaller nodes in unbounded channels crossbeam-rs/crossbeam-channel#81

Closed

mbrubeck mentioned this pull request Sep 25, 2018

Possible UB from use of uninitialized [&T; N] servo/rust-smallvec#126

Closed

varkor mentioned this pull request Oct 9, 2018

Less conservative uninhabitedness check rust-lang/rust#54125

Merged

Ekleog mentioned this pull request Apr 17, 2019

GenericArray should implement Index{,Mut} as well as a way to get a pointer to a field fizyk20/generic-array#75

Open

This was referenced May 1, 2019

Add as_mut_ptr to PublicKey rust-bitcoin/rust-secp256k1#105

Merged

stabilize core parts of MaybeUninit rust-lang/rust#60445

Merged

tuxsoy mentioned this pull request Oct 29, 2019

Add VK_KHR_display extension support ash-rs/ash#247

Merged

bstrie mentioned this pull request Jul 10, 2021

Defuse the bomb that is mem::uninitialized rust-lang/rust#87032

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate uninitialized in favor of a new MaybeUninit type #1892

Deprecate uninitialized in favor of a new MaybeUninit type #1892

canndrew commented Feb 9, 2017 •

edited by Centril

Loading

canndrew commented Feb 9, 2017

strega-nil commented Feb 9, 2017

canndrew commented Feb 9, 2017

nagisa commented Feb 9, 2017

nagisa Feb 9, 2017

mark-i-m Feb 10, 2017

canndrew Feb 10, 2017

mark-i-m Feb 10, 2017

canndrew Feb 10, 2017

nagisa Feb 9, 2017

nagisa Feb 9, 2017

arielb1 commented Feb 9, 2017

canndrew commented Feb 9, 2017

canndrew commented Feb 9, 2017

nagisa commented Feb 9, 2017 •

edited

Loading

canndrew commented Feb 9, 2017

strega-nil Feb 9, 2017

canndrew Feb 10, 2017

mark-i-m Feb 10, 2017

mark-i-m Feb 10, 2017

canndrew Feb 10, 2017

mark-i-m Feb 10, 2017

mark-i-m Feb 10, 2017

ranma42 commented Feb 10, 2017

ranma42 commented Feb 10, 2017

ranma42 commented Feb 10, 2017

ghost commented Feb 10, 2017

djzin commented Feb 12, 2017

mark-i-m commented Feb 12, 2017 via email

gnzlbg commented Aug 11, 2018 •

edited

Loading

RalfJung commented Aug 17, 2018 •

edited

Loading

rfcbot commented Aug 19, 2018

Centril commented Aug 19, 2018


		This trait is automatically implemented for all inhabited types.

		Change the type of `uninitialized` to:

Deprecate uninitialized in favor of a new MaybeUninit type #1892

Deprecate uninitialized in favor of a new MaybeUninit type #1892

Conversation

canndrew commented Feb 9, 2017 • edited by Centril Loading

canndrew commented Feb 9, 2017

strega-nil commented Feb 9, 2017

canndrew commented Feb 9, 2017

nagisa commented Feb 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arielb1 commented Feb 9, 2017

canndrew commented Feb 9, 2017

canndrew commented Feb 9, 2017

nagisa commented Feb 9, 2017 • edited Loading

canndrew commented Feb 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ranma42 commented Feb 10, 2017

ranma42 commented Feb 10, 2017

ranma42 commented Feb 10, 2017

ghost commented Feb 10, 2017

djzin commented Feb 12, 2017

mark-i-m commented Feb 12, 2017 via email

gnzlbg commented Aug 11, 2018 • edited Loading

RalfJung commented Aug 17, 2018 • edited Loading

rfcbot commented Aug 19, 2018

Centril commented Aug 19, 2018

canndrew commented Feb 9, 2017 •

edited by Centril

Loading

nagisa commented Feb 9, 2017 •

edited

Loading

gnzlbg commented Aug 11, 2018 •

edited

Loading

RalfJung commented Aug 17, 2018 •

edited

Loading