Possible UB from use of uninitialized [&T; N]

[mbrubeck]

Creating an array with mem::uninitialized may be UB if the array's elements contain non-null types (like &T) or uninhabited types (like !) or other types that have invalid values.

This can result in undefined behavior even if you never read the elements directly while they are uninitialized. For example, this program prints "true" if optimizations are enabled, when built with current rustc:

fn main() {
    let x: Option<[&u8; 1]> = unsafe { Some(std::mem::uninitialized()) };
    println!("{}", x.is_none());
}

SmallVec isn't eligible for null pointer optimization, so it isn't affected by this. However, it's possible that a future Rust compiler could add other optimizations that break if SmallVec uses mem::uninitialized to create arrays of such types.

The ideal solution is to switch from uninitialized to the new MaybeUninit union (rust-lang/rfcs#1892) when it becomes stable.

[mbrubeck]

See also rust-lang/rust#54542.

[bluss]

ManuallyDrop<T> changed to allow enum layout optimization using the interior value, so it regressed this crate in that way. rust-lang/rust/pull/52711

I can't make the test fail, but I think we have a problem hiding in here, as long as an optional smallvec of references is the same size as the plain type:

https://play.rust-lang.org/?version=stable&mode=release&edition=2015&gist=471955af6ec77850bf98446aba013c35

[mbrubeck]

SmallVec contains an enum, so I think the enum layout optimization is using spare bits from that enum's tag to store the Option discriminant. If this is true then we're not seeing incorrect behavior in practice yet.

[llogiq]

I recently looked into the use of MaybeUninit<T> in arrays. We'd need to extend the Array trait with an associated type of type InternalArray = [MaybeUninit<$ty>; $n] and use that internally. However that would require the heap variant to become (*mut MaybeUninit<A::Item>, usize) to ensure UB-freedom.