Lint against &T to &mut T and &T to &UnsafeCell<T> transmutes

[ChayimFriedman2]

Conversion from & to &mut are and always were immediate UB, and we already lint against them, but until now the lint did not catch the case were the reference was in a field.

Conversion from & to &UnsafeCell is more nuanced: Stacked Borrows makes it immediate UB, but in Tree Borrows it is sound.

However, even in Tree Borrows it is UB to write into that reference (if the original value was Freeze). In all cases crater found where the lint triggered, the reference was written into.

Lints (mutable_transmutes existed before):

The mutable_transmutes lint catches transmuting from &T to &mut T because it is undefined behavior.

Example

unsafe {
    let y = std::mem::transmute::<&i32, &mut i32>(&5);
}

Produces:

error: transmuting &T to &mut T is undefined behavior, even if the reference is unused, consider instead using an UnsafeCell
 --> .\example.rs:5:17
  |
5 |         let y = std::mem::transmute::<&i32, &mut i32>(&5);
  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: transmute from `&i32` to `&mut i32`
  = note: `#[deny(mutable_transmutes)]` on by default

Explanation

Certain assumptions are made about aliasing of data, and this transmute
violates those assumptions. Consider using UnsafeCell instead.

---

The unsafe_cell_transmutes lint catches transmuting or casting from &T to &UnsafeCell<T>
because it dangerous and might be undefined behavior.

Example

use std::cell::Cell;

unsafe {
    let x = 5_i32;
    let y = std::mem::transmute::<&i32, &Cell<i32>>(&x);
    y.set(6);

    let z = &*(&x as *const i32 as *const Cell<i32>);
    z.set(7);
}

Produces:

error: transmuting &T to &UnsafeCell<T> is error-prone, rarely intentional and may cause undefined behavior
 --> .\example.rs:6:17
  |
6 |         let y = std::mem::transmute::<&i32, &Cell<i32>>(&x);
  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: transmute from `*&i32` to `(*&Cell<i32>).value`
  = note: `#[deny(unsafe_cell_transmutes)]` on by default

error: transmuting &T to &UnsafeCell<T> is error-prone, rarely intentional and may cause undefined behavior
 --> .\example.rs:9:17
  |
9 |         let z = &*(&x as *const i32 as *const Cell<i32>);
  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: transmute from `i32` to `Cell<i32>.value`

Explanation

Conversion from &T to &UnsafeCell<T> might be immediate undefined behavior, depending on
unspecified details of the aliasing model.

Even if it is not, writing to it will be undefined behavior if there was no UnsafeCell in
the original T, and even if there was, it might be undefined behavior (again, depending
on unspecified details of the aliasing model).

It is also highly dangerous and error-prone, and unlikely to be useful.

Crater summary is below.

r? compiler

[rustbot]

The Miri subtree was changed

cc @rust-lang/miri

[oli-obk]

@bors try

[bors]

⌛ Testing commit 5c1d66e with merge 94971c2...

[bors]

💔 Test failed - checks-actions

[oli-obk]

@bors try

[bors]

⌛ Trying commit 59c22c3 with merge be92fb7...

[tgross35]

@bors try

[bors]

⌛ Trying commit cc72a92 with merge 0914651...

[bors]

☀️ Try build successful - checks-actions
Build commit: 0914651 (09146510cdb33d15474a35fde290187a88b1cf35)

[oli-obk]

@craterbot check

[craterbot]

👌 Experiment pr-128351 created and queued.
🤖 Automatically detected try build 0914651
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-128351 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[bors]

☔ The latest upstream changes (presumably #129665) made this pull request unmergeable. Please resolve the merge conflicts.

[nikomatsakis]

@rfcbot fcp merge

We discussed this in the lang team. No FCP is needed for the first part (a bug fix) but the second lint is new. We are in favor, however, so starting off an FCP.

[rfcbot]

Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[RalfJung]

Conversion from & to &mut are and always were immediate UB, and we already lint against them, but until now the lint did not catch the case were the reference was in a field.

Not quite -- under SB, this is only UB if the shared reference points to a Freeze type.

Under TB, it's actually only UB once you write to the &mut. However, it still seems fine to lint against the Freeze case here, this is almost never correct. If we eventually decide we want to allow this, we can relax the lint.

[RalfJung]

Can you add a test that transmuting &Cell<i32> to &mut is allowed? The reference is writeable so it seems odd to me to lint against that.

[ChayimFriedman2]

@RalfJung

Can you add a test that transmuting &Cell<i32> to &mut is allowed? The reference is writeable so it seems odd to me to lint against that.

We already lint in this case. I can change that, but given that, is that fine to not do this in this PR?

[RalfJung]

Hm, just makes me wonder if there are any other wrong assumptions encoded in the lint...

[tmandry]

@rfcbot reviewed

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.