Enable Miri to pass pointers through FFI #129684

Strophox · 2024-08-28T10:56:13Z

Following #126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers.

Simple example

extern "C" {
    fn ptr_printer(ptr: *mut i32);
}

fn main() {
    let ptr = &mut 42 as *mut i32;
    unsafe {
        ptr_printer(ptr);
    }
}

void ptr_printer(int *ptr) {
  printf("printing pointer dereference from C: %d\n", *ptr);
}

should now show printing pointer dereference from C: 42.

Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call.

r? @RalfJung

rustbot · 2024-08-28T10:56:22Z

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

Some changes occurred to the CTFE / Miri engine

cc @rust-lang/miri

The Miri subtree was changed

cc @rust-lang/miri

Strophox · 2024-08-28T11:51:25Z

Oh. Well that didn't work yet.

src/tools/miri/src/machine.rs

compiler/rustc_middle/src/mir/interpret/allocation.rs

src/tools/miri/src/alloc_addresses/mod.rs

src/tools/miri/src/machine.rs

src/tools/miri/src/shims/native_lib.rs

RalfJung · 2024-08-29T09:59:48Z

src/tools/miri/tests/native-lib/pass/call_extern_c_fn.stdout

@@ -1 +1,2 @@
 printing from C
+printing pointer dereference from C: 42


Strophox · 2024-08-29T17:11:10Z

(I might quickly try to enhance+refactor the C pointer FFI tests in Miri now.)

src/tools/miri/src/alloc_addresses/mod.rs

src/tools/miri/src/machine.rs

src/tools/miri/src/alloc_addresses/mod.rs

RalfJung

Looks good overall, thanks! Just got some nits regarding the tests.

src/tools/miri/tests/native-lib/native-lib.map

src/tools/miri/tests/native-lib/pass/ptr_read_access.rs

src/tools/miri/tests/native-lib/pass/scalar_arguments.rs

src/tools/miri/tests/native-lib/ptr_read_access.c

src/tools/miri/tests/native-lib/scalar_arguments.c

src/tools/miri/tests/ui.rs

src/tools/miri/tests/native-lib/ptr_read_access.c

RalfJung · 2024-08-30T13:38:04Z

Please squash the commits, then we can land this. :-)

Co-authored-by: Ralf Jung <post@ralfj.de>

RalfJung · 2024-08-31T07:30:21Z

Great, congrats on getting this finished!

@bors r+

bors · 2024-08-31T07:30:24Z

📌 Commit 7fde02e has been approved by RalfJung

It is now in the queue for this repository.

… r=RalfJung Enable Miri to pass pointers through FFI Following rust-lang#126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn ptr_printer(ptr: *mut i32); > } > > fn main() { > let ptr = &mut 42 as *mut i32; > unsafe { > ptr_printer(ptr); > } > } > ``` > ```c > void ptr_printer(int *ptr) { > printf("printing pointer dereference from C: %d\n", *ptr); > } > ``` > should now show `printing pointer dereference from C: 42`. > > </details> Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call. r? `@RalfJung`

…iaskrgr Rollup of 11 pull requests Successful merges: - rust-lang#128523 (Add release notes for 1.81.0) - rust-lang#129605 (Add missing `needs-llvm-components` directives for run-make tests that need target-specific codegen) - rust-lang#129650 (Clean up `library/profiler_builtins/build.rs`) - rust-lang#129651 (skip stage 0 target check if `BOOTSTRAP_SKIP_TARGET_SANITY` is set) - rust-lang#129684 (Enable Miri to pass pointers through FFI) - rust-lang#129762 (Update the `wasm-component-ld` binary dependency) - rust-lang#129782 (couple more crash tests) - rust-lang#129816 (tidy: say which feature gate has a stability issue mismatch) - rust-lang#129818 (make the const-unstable-in-stable error more clear) - rust-lang#129824 (Fix code examples buttons not appearing on click on mobile) - rust-lang#129826 (library: Fix typo in `core::mem`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#129684 - Strophox:miri-pass-pointer-to-ffi, r=RalfJung Enable Miri to pass pointers through FFI Following rust-lang#126787, the purpose of this PR is to now enable Miri to execute native calls that make use of pointers. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn ptr_printer(ptr: *mut i32); > } > > fn main() { > let ptr = &mut 42 as *mut i32; > unsafe { > ptr_printer(ptr); > } > } > ``` > ```c > void ptr_printer(int *ptr) { > printf("printing pointer dereference from C: %d\n", *ptr); > } > ``` > should now show `printing pointer dereference from C: 42`. > > </details> Note that this PR does not yet implement any logic involved in updating Miri's "analysis" state (byte initialization, provenance) upon such a native call. r? ``@RalfJung``

…-ffi, r=RalfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? `@RalfJung`

…-ffi, r=RalfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? ``@RalfJung``

Rollup merge of rust-lang#133211 - Strophox:miri-correct-state-update-ffi, r=RalfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? ``@RalfJung``

…alfJung Extend Miri to correctly pass mutable pointers through FFI Based off of rust-lang/rust#129684, this PR further extends Miri to execute native calls that make use of pointers to *mutable* memory. We adapt Miri's bookkeeping of internal state upon any FFI call that gives external code permission to mutate memory. Native code may now possibly write and therefore initialize and change the pointer provenance of bytes it has access to: Such memory is assumed to be *initialized* afterwards and bytes are given *arbitrary (wildcard) provenance*. This enables programs that correctly use mutating FFI calls to run Miri without errors, at the cost of possibly missing Undefined Behaviour caused by incorrect usage of mutating FFI. > <details> > > <summary> Simple example </summary> > > ```rust > extern "C" { > fn init_int(ptr: *mut i32); > } > > fn main() { > let mut x = std::mem::MaybeUninit::<i32>::uninit(); > let x = unsafe { > init_int(x.as_mut_ptr()); > x.assume_init() > }; > > println!("C initialized my memory to: {x}"); > } > ``` > ```c > void init_int(int *ptr) { > *ptr = 42; > } > ``` > should now show `C initialized my memory to: 42`. > > </details> r? ``@RalfJung``

rustbot assigned RalfJung Aug 28, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 28, 2024

RalfJung reviewed Aug 28, 2024

View reviewed changes

src/tools/miri/src/machine.rs Outdated Show resolved Hide resolved

RalfJung reviewed Aug 28, 2024

View reviewed changes

compiler/rustc_middle/src/mir/interpret/allocation.rs Outdated Show resolved Hide resolved

RalfJung reviewed Aug 28, 2024

View reviewed changes

src/tools/miri/src/alloc_addresses/mod.rs Outdated Show resolved Hide resolved

RalfJung reviewed Aug 29, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

RalfJung reviewed Aug 29, 2024

View reviewed changes

src/tools/miri/src/alloc_addresses/mod.rs Show resolved Hide resolved

RalfJung reviewed Aug 29, 2024

View reviewed changes

src/tools/miri/src/alloc_addresses/mod.rs Outdated Show resolved Hide resolved

src/tools/miri/src/alloc_addresses/mod.rs Show resolved Hide resolved

src/tools/miri/src/machine.rs Outdated Show resolved Hide resolved

RalfJung reviewed Aug 30, 2024

View reviewed changes

src/tools/miri/src/alloc_addresses/mod.rs Outdated Show resolved Hide resolved

RalfJung reviewed Aug 30, 2024

View reviewed changes

src/tools/miri/src/alloc_addresses/mod.rs Show resolved Hide resolved

Strophox force-pushed the miri-pass-pointer-to-ffi branch from dcdb101 to 6e23a4d Compare August 30, 2024 09:04

RalfJung reviewed Aug 30, 2024

View reviewed changes

src/tools/miri/tests/native-lib/ptr_read_access.c Outdated Show resolved Hide resolved

enable Miri to pass const pointers through FFI

7fde02e

Co-authored-by: Ralf Jung <post@ralfj.de>

Strophox force-pushed the miri-pass-pointer-to-ffi branch from 730bd97 to 7fde02e Compare August 30, 2024 14:12

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 31, 2024

This was referenced Aug 31, 2024

Rollup of 12 pull requests #129830

Closed

Rollup of 11 pull requests #129831

Merged

bors merged commit a5fb8b9 into rust-lang:master Aug 31, 2024
6 checks passed

rustbot added this to the 1.82.0 milestone Aug 31, 2024

RalfJung mentioned this pull request Sep 13, 2024

Support native FFI calls via libffi rust-lang/miri#11

Open

7 tasks

Strophox mentioned this pull request Nov 19, 2024

Extend Miri to correctly pass mutable pointers through FFI #133211

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Miri to pass pointers through FFI #129684

Enable Miri to pass pointers through FFI #129684

Strophox commented Aug 28, 2024

rustbot commented Aug 28, 2024

Strophox commented Aug 28, 2024

RalfJung Aug 29, 2024

This comment has been minimized.

Strophox commented Aug 29, 2024

RalfJung left a comment

RalfJung commented Aug 30, 2024

RalfJung commented Aug 31, 2024

bors commented Aug 31, 2024

		@@ -1 +1,2 @@
		printing from C
		printing pointer dereference from C: 42

Enable Miri to pass pointers through FFI #129684

Enable Miri to pass pointers through FFI #129684

Conversation

Strophox commented Aug 28, 2024

rustbot commented Aug 28, 2024

Strophox commented Aug 28, 2024

RalfJung Aug 29, 2024

Choose a reason for hiding this comment

This comment has been minimized.

Strophox commented Aug 29, 2024

RalfJung left a comment

Choose a reason for hiding this comment

RalfJung commented Aug 30, 2024

RalfJung commented Aug 31, 2024

bors commented Aug 31, 2024