feat: mman `NonNull` #2000

JonathanWoollett-Light · 2023-02-15T01:42:07Z

Updates functions in nix::sys::mman to use NonNull.

The case where the returned pointer is null is the error case, therefore in the Ok case we can return NonNull.

CHANGELOG.md

SteveLauC · 2023-10-02T04:22:11Z

src/sys/mman.rs

-        Ok(ret)
-    }
+
+    NonNull::new(ret).ok_or_else(Errno::last)


NonNull::new() will ONLY return a None if its argument is a null pointer, the error case of mmap, whose return value libc::MAP_FAILED ((void *) -1 = 0xffffffff) is not null:

use std::ptr::NonNull; fn main() { let ptr = NonNull::new(libc::MAP_FAILED); println!("{:?}", ptr); }

$ cargo r -q Some(0xffffffffffffffff)

which means that the current impl will return Ok(NonNull(0xffffffffff)) even when the underlying mmap() failed

I've updated the implementation considering this.

SteveLauC · 2023-10-02T04:24:35Z

Given that mmap() won't return a null pointer at all, I am actually not sure if this change is needed, thoughts on this?

cc @asomers

JonathanWoollett-Light · 2023-10-09T17:53:52Z

Given that mmap() won't return a null pointer at all, I am actually not sure if this change is needed, thoughts on this?

If it always returns a non-null pointer this seems even more reason to return NonNull.

src/sys/mman.rs

SteveLauC · 2023-11-07T04:17:39Z

Given that mmap() won't return a null pointer at all, I am actually not sure if this change is needed, thoughts on this?

If it always returns a non-null pointer this seems even more reason to return NonNull.

I think both our points are correct here:

From the interface safety perspetive, mmap() won't return a NULL pointer, so wrapping it in a NonNull is useless. However, for pointers that are not NULL, wrapping it in a NonNull will definitely make the interface more Rusty.

So I am not against this now, but still ping @asomers, I would like to hear your thoughts

Jan561 · 2023-11-10T18:34:40Z

Just adding my two cents here.

NonNull<T> differs from *mut T in two aspects:

The pointer is guaranteed to be not NULL (obviously). This allows the compiler to perform some optimizations.
NonNull<T> is covariant in T. This is usually desirable when dealing with unique pointers (i.e. we own T, e.g. Vec, Box) or when dealing with shared pointers (e.g. shared ownership Rc, Arc).

mmap allocates new virtual memory and returns a unique pointer to the allocation, which exactly fits the use case of representing ownership with NonNull. This alone makes this change worthwhile for me.

Additionally, this function is fallible and we are only one call away of turning the result into an Option by calling Result::ok, where the aforementioned optimizations apply.

The only reason why NonNull is so underused even in the standard library is that many functions have already been stabilized before NonNull was invented. The first (?) API in std that will use NonNull instead of *mut is the upcoming Allocator API, which coincidentally is also what mmap essentially is.

SteveLauC · 2023-11-12T02:06:18Z

Just adding my two cents here.

NonNull<T> differs from *mut T in two aspects:

The pointer is guaranteed to be not NULL (obviously). This allows the compiler to perform some optimizations.

NonNull<T> is covariant in T. This is usually desirable when dealing with unique pointers (i.e. we own T, e.g. Vec, Box) or when dealing with shared pointers (e.g. shared ownership Rc, Arc).

mmap allocates new virtual memory and returns a unique pointer to the allocation, which exactly fits the use case of representing ownership with NonNull. This alone makes this change worthwhile for me.

Additionally, this function is fallible and we are only one call away of turning the result into an Option by calling Result::ok, where the aforementioned optimizations apply.

The only reason why NonNull is so underused even in the standard library is that many functions have already been stabilized before NonNull was invented. The first (?) API in std that will use NonNull instead of *mut is the upcoming Allocator API, which coincidentally is also what mmap essentially is.

Makes sense to me, if we are gonna merge this change, should we also update the munmap(2) interface to something like?

pub unsafe fn munmap(addr: NonNull<c_void>, len: size_t) -> Result<()>

Jan561 · 2023-11-12T11:51:54Z

Yes, good point. For symmetry reasons, I believe all other functions in mman should implement this change, too.

JonathanWoollett-Light · 2023-11-12T14:40:14Z

Yes, good point. For symmetry reasons, I believe all other functions in mman should implement this change, too.

I've updated the PR with all the other changes.

I think a follow-up PR could update the functions to take generic NonNull<[T]> instead of taking NonNull<c_void> and size_t.

Jan561 · 2023-11-12T20:04:55Z

Yes, good point. For symmetry reasons, I believe all other functions in mman should implement this change, too.

I've updated the PR with all the other changes.

I think a follow-up PR could update the functions to take generic NonNull<[T]> instead of taking NonNull<c_void> and size_t.

I agree, this is a way nicer abstraction, but since this function allocates in pages and doesn't care about the layout of T, an idiomatic mmap should probably just return a page-aligned NonNull<[u8]> and the other functions taking the allocation as an argument should use the page-aligned starting pointer NonNull<u8>, similar to the std allocator trait.

The metadata of the slice pointer would then be the length rounded up to the next multiple of the page size.

asomers

I was on the fence about this, because NonNull is less ergonomic than a plain pointer. But symmetry with Allocator is a good enough reason to do this, I think. I just noted one minor problem in some of the tests.

asomers · 2023-11-12T21:37:44Z

test/sys/test_mman.rs

@@ -43,7 +47,7 @@ fn test_mremap_grow() {
    let slice: &mut [u8] = unsafe {
        #[cfg(target_os = "linux")]
        let mem = mremap(
-            slice.as_mut_ptr().cast(),
+            NonNull::from(&slice[..]).cast(),


Here you're mutating through an immutable pointer. That's not ok. You should do Slice::as_mut_ptr or &mut slice instead. It happens a few places below, too.

Updated it to &mut slice[..]

It seems that we can get rid of that #![allow(clippy::redundant_slicing)] if we just use the slice here, slice itself is of type &mut [u8], the constructed NonNull will be NonNull<[u8]>, then we cast() it to NonNull<c_void>

I tried this, it fails in testing.

Oops, sorry, &mut [u8] is not Copy, manually reborrowing here should do the trick

SteveLauC · 2023-11-13T00:17:06Z

src/sys/mman.rs

    }
+    else {
+        Ok(unsafe { NonNull::new_unchecked(ret) })


Given that this is unsafe, do you think that we could add a comment stating it is safe, maybe something like:

mmap() will never return a NULL pointer, it is safe to use NonNull::new_unchecked() here

Added comment.

SteveLauC · 2023-11-13T00:17:33Z

src/sys/mman.rs

@@ -468,7 +467,7 @@ pub unsafe fn mmap_anonymous(
    if ret == libc::MAP_FAILED {
        Err(Errno::last())
    } else {
-        Ok(ret)
+        Ok(unsafe { NonNull::new_unchecked(ret) })


Also this one

Added comment.

SteveLauC · 2023-11-13T00:21:43Z

src/sys/mman.rs

        new_size,
        flags.bits(),
    );

    if ret == libc::MAP_FAILED {
        Err(Errno::last())
    } else {
-        Ok(ret)
+        Ok(unsafe { NonNull::new_unchecked(ret) })


And this one: mremap() will never return a NULL pointer, it is safe to use NonNull::new_unchecked() here

Added comment.

SteveLauC · 2023-11-13T00:34:22Z

src/sys/mman.rs

+pub unsafe fn munmap(mut addr: NonNull<c_void>, len: size_t) -> Result<()> {
+    Errno::result(libc::munmap(addr.as_mut(), len)).map(drop)


Why we are using NonNull.as_mut() for munmap()? Other functions are all using NonNull.as_ptr()

Updated to NonNull::as_ptr

JonathanWoollett-Light force-pushed the mmap-nonnull branch 2 times, most recently from 9f5cfb7 to b9be0e1 Compare February 15, 2023 19:32

JonathanWoollett-Light force-pushed the mmap-nonnull branch 3 times, most recently from dfcc05d to 099f78b Compare October 1, 2023 20:46

SteveLauC requested changes Oct 2, 2023

View reviewed changes

SteveLauC self-assigned this Oct 2, 2023

JonathanWoollett-Light force-pushed the mmap-nonnull branch 5 times, most recently from 6b549e4 to f2b0029 Compare October 2, 2023 10:01

JonathanWoollett-Light force-pushed the mmap-nonnull branch from f2b0029 to 5ce9eaa Compare October 9, 2023 17:58

JonathanWoollett-Light force-pushed the mmap-nonnull branch from 5ce9eaa to 454ccfd Compare November 7, 2023 03:23

SteveLauC requested changes Nov 7, 2023

View reviewed changes

src/sys/mman.rs Outdated Show resolved Hide resolved

JonathanWoollett-Light force-pushed the mmap-nonnull branch 3 times, most recently from e15eae4 to 4a3bc01 Compare November 7, 2023 04:52

JonathanWoollett-Light force-pushed the mmap-nonnull branch from 4a3bc01 to a77786e Compare November 12, 2023 14:40

JonathanWoollett-Light force-pushed the mmap-nonnull branch 4 times, most recently from 75abb35 to a16937e Compare November 12, 2023 14:58

JonathanWoollett-Light force-pushed the mmap-nonnull branch from a16937e to a8942ce Compare November 12, 2023 16:48

JonathanWoollett-Light changed the title ~~feat: mmap NonNull~~ feat: mman NonNull Nov 12, 2023

JonathanWoollett-Light force-pushed the mmap-nonnull branch 2 times, most recently from 9b3a981 to b32fe4e Compare November 12, 2023 17:28

JonathanWoollett-Light mentioned this pull request Nov 12, 2023

Tracking Issue for raw slice getters (slice_ptr_get) rust-lang/rust#74265

Open

4 tasks

asomers requested changes Nov 12, 2023

View reviewed changes

JonathanWoollett-Light force-pushed the mmap-nonnull branch from b32fe4e to 2e5288a Compare November 12, 2023 23:01

SteveLauC reviewed Nov 13, 2023

View reviewed changes

feat: mmap NonNull

e5d99e3

JonathanWoollett-Light force-pushed the mmap-nonnull branch from 2e5288a to 22eda8d Compare November 13, 2023 18:28

JonathanWoollett-Light added 7 commits November 13, 2023 18:30

feat: munmap NonNull

879c227

feat: mremap NonNull

64b9a66

feat: madvise NonNull

fffe577

feat: msync NonNull

74c2eb0

feat: mprotect NonNull

85099da

feat: munlock NonNull

7365554

feat mlock NonNull

5756e07

JonathanWoollett-Light force-pushed the mmap-nonnull branch from 22eda8d to 5756e07 Compare November 13, 2023 18:30

SteveLauC approved these changes Nov 13, 2023

View reviewed changes

SteveLauC added this pull request to the merge queue Nov 13, 2023

Merged via the queue into nix-rust:master with commit 1ea59c7 Nov 13, 2023
35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: mman `NonNull` #2000

feat: mman `NonNull` #2000

JonathanWoollett-Light commented Feb 15, 2023 •

edited

Loading

SteveLauC Oct 2, 2023

JonathanWoollett-Light Oct 2, 2023

SteveLauC commented Oct 2, 2023

JonathanWoollett-Light commented Oct 9, 2023 •

edited

Loading

SteveLauC commented Nov 7, 2023

Jan561 commented Nov 10, 2023 •

edited

Loading

SteveLauC commented Nov 12, 2023

Jan561 commented Nov 12, 2023

JonathanWoollett-Light commented Nov 12, 2023 •

edited

Loading

Jan561 commented Nov 12, 2023

asomers left a comment

asomers Nov 12, 2023

JonathanWoollett-Light Nov 12, 2023

SteveLauC Nov 13, 2023

JonathanWoollett-Light Nov 13, 2023 •

edited

Loading

SteveLauC Nov 13, 2023

SteveLauC Nov 13, 2023

JonathanWoollett-Light Nov 13, 2023

SteveLauC Nov 13, 2023

JonathanWoollett-Light Nov 13, 2023

SteveLauC Nov 13, 2023

JonathanWoollett-Light Nov 13, 2023

SteveLauC Nov 13, 2023

JonathanWoollett-Light Nov 13, 2023 •

edited

Loading

		pub unsafe fn munmap(mut addr: NonNull<c_void>, len: size_t) -> Result<()> {
		Errno::result(libc::munmap(addr.as_mut(), len)).map(drop)

feat: mman NonNull #2000

feat: mman NonNull #2000

Conversation

JonathanWoollett-Light commented Feb 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SteveLauC commented Oct 2, 2023

JonathanWoollett-Light commented Oct 9, 2023 • edited Loading

SteveLauC commented Nov 7, 2023

Jan561 commented Nov 10, 2023 • edited Loading

SteveLauC commented Nov 12, 2023

Jan561 commented Nov 12, 2023

JonathanWoollett-Light commented Nov 12, 2023 • edited Loading

Jan561 commented Nov 12, 2023

asomers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JonathanWoollett-Light Nov 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JonathanWoollett-Light Nov 13, 2023 • edited Loading

Choose a reason for hiding this comment

feat: mman `NonNull` #2000

feat: mman `NonNull` #2000

JonathanWoollett-Light commented Feb 15, 2023 •

edited

Loading

JonathanWoollett-Light commented Oct 9, 2023 •

edited

Loading

Jan561 commented Nov 10, 2023 •

edited

Loading

JonathanWoollett-Light commented Nov 12, 2023 •

edited

Loading

JonathanWoollett-Light Nov 13, 2023 •

edited

Loading

JonathanWoollett-Light Nov 13, 2023 •

edited

Loading