Use NonNull in slice::Iter and slice::IterMut. #67588

Kixunil · 2019-12-24T15:16:19Z

ptr of slice::Iter and slice::IterMut can never be null, but this
fact wasn't exploited for layout optimizations. By changing ptr from
*<mutability> T to NonNull<T>, the compiler can now optimize layout
of Option<Iter<'a, T>>.

Closes #67228

rust-highfive · 2019-12-24T15:16:23Z

r? @withoutboats

(rust_highfive has picked a reviewer for you, use r? to override)

ranma42 · 2019-12-24T15:30:32Z

src/libcore/slice/mod.rs

@@ -3215,7 +3223,7 @@ macro_rules! iterator {
            fn next(&mut self) -> Option<$elem> {
                // could be implemented with slices, but this avoids bounds checks
                unsafe {
-                    assume(!self.ptr.is_null());
+                    assume(!self.ptr.as_ptr().is_null());


This is now implied by the NonNull type, right? Should we remove this assume?

My understanding is it's an optimization when it comes to code generation, not layout optimization. But maybe rustc is smart enough about that? I'm not that familiar with how NonNull works.

Looks like we could remove it. Is that example good enough evidence? Or should we reach out to someone who knows better?

The "this pointer is non-null" information added by rustc to LLVM IR automatically is limited to particular positions (function parameters, function return values, loads from memory) which can get eliminated by optimizations (such as inlining and SROA) before they have served their purpose. In contrast, assumes stay around basically for the entire optimization pipeline. So I am not confident that we can remove this assume without regressions. You're welcome to try to evaluate that on real world code (the example linked above is far too isolated, I would recommend history spelunking to find the benchmarks that motivated adding the assume in the first place) but I would recommend doing that in a separate PR.

Centril · 2019-12-24T15:59:06Z

r? @rkruppe

petertodd · 2019-12-24T16:48:47Z

src/libcore/slice/mod.rs

@@ -3467,7 +3470,7 @@ impl<T> AsRef<[T]> for Iter<'_, T> {
 /// [slices]: ../../std/primitive.slice.html
 #[stable(feature = "rust1", since = "1.0.0")]
 pub struct IterMut<'a, T: 'a> {
-    ptr: *mut T,
+    ptr: NonNull<T>,
    end: *mut T, // If T is a ZST, this is actually ptr+len.  This encoding is picked so that
    // ptr == end is a quick test for the Iterator being empty, that works
    // for both ZST and non-ZST.


Can they both be NonNull<T>? IIUC it's an invariant that ptr <= end, so a null end is invalid.

We don't use multiple niches yet, but a future optimization might, and it could help communicate the invariant better.

According to a comment further down end might be 0 in case of ZST.

Thanks for review, BTW! :)

Ah of course: a ZST slice can have an arbitrary starting pointer, and len is encoded as the wrapping subtraction of end - start. Thus a slice of length 1 with the max-int pointer results in end = 0.

You know, now that we have unions a simpler way to do all this could be:

union LenOrEnd<T> { end: NonNull<T>, len: usize, } pub struct IterMut<'a, T> { marker: PhantomData<&'a mut T>, start: NonNull<T>, len_or_end: LenOrEnd<T>, }

A union of SizedIterMut and ZeroSizedIterMut is another possibility. Except that currently union's don't participate in layout optimization. :/

I'll bet you this will have slightly better performance too by representing len directly rather than needing to do arithmetic to derive it.

I was thinking that the code might be improved in different ways too. I just think that we should go with smaller increments. The PR already improves something, new improvements could be in the next PR.

Anyway, I think the layout would have to be determined in type, not in union in order to enable optimizations. I'm not an expert but I suspect that making rustc smart enough to understand which part of the union is used is based on type and conditionally optimize layout would be very difficult.

Yeah, I agree that an incremental improvement is fine right now.

The above does get the NonNull<T> layout optimization as the NonNull<T> is in the struct, rather than the union: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=82eb70c78825be8dd5e48fd5d8d521cc

hanna-kruppe · 2019-12-25T10:31:52Z

I like this diff as-is, removing assumes can be a follow up. Please rebase (we avoid merge commits) and squash so I can approve it.

Kixunil · 2019-12-25T20:34:03Z

Thanks for explanation and instructions! I hope I cleaned it up correctly. I like the diff as well. I don't have enough expertise to be confident about removing assumes, so don't expect the next PR doing it from me. I'll focus on things I'm better at. :)

hanna-kruppe · 2019-12-25T20:35:57Z

Alright, thanks a lot!

@bors r+

bors · 2019-12-25T20:35:59Z

📌 Commit 047ff0b has been approved by rkruppe

Kixunil · 2019-12-25T20:38:05Z

I'm happy to have helped, thank you too!

Use NonNull in slice::Iter and slice::IterMut. `ptr` of `slice::Iter` and `slice::IterMut` can never be null, but this fact wasn't exploited for layout optimizations. By changing `ptr` from `*<mutability> T` to `NonNull<T>`, the compiler can now optimize layout of `Option<Iter<'a, T>>`. Closes rust-lang#67228

bors · 2019-12-26T07:06:53Z

⌛ Testing commit 047ff0b with merge 9b345cbc1023bfec0dfc4de2ae391bd23224582d...

rust-highfive · 2019-12-26T07:12:27Z

The job i686-apple of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-12-26T07:12:38Z

💔 Test failed - checks-azure

src/libcore/slice/mod.rs

`ptr` of `slice::Iter` and `slice::IterMut` can never be null, but this fact wasn't exploited for layout optimizations. By changing `ptr` from `*<mutability> T` to `NonNull<T>`, the compiler can now optimize layout of `Option<Iter<'a, T>>`.

hanna-kruppe · 2019-12-27T14:47:21Z

@bors r+

bors · 2019-12-27T14:47:22Z

📌 Commit 2c796ee has been approved by rkruppe

Use NonNull in slice::Iter and slice::IterMut. `ptr` of `slice::Iter` and `slice::IterMut` can never be null, but this fact wasn't exploited for layout optimizations. By changing `ptr` from `*<mutability> T` to `NonNull<T>`, the compiler can now optimize layout of `Option<Iter<'a, T>>`. Closes rust-lang#67228

@ghost

Rollup of 15 pull requests Successful merges: - #65244 (add IntoFuture trait and support for await) - #67576 (reuse `capacity` variable in slice::repeat) - #67588 (Use NonNull in slice::Iter and slice::IterMut.) - #67594 (Update libc to 0.2.66) - #67602 (Use issue = "none" instead of "0" in intrinsics) - #67604 (Add Scalar::to_(u|i)16 methods) - #67617 (Remove `compiler_builtins_lib` documentation) - #67621 (Use the correct type for static qualifs) - #67629 (Remove redundant link texts) - #67632 (Convert collapsed to shortcut reference links) - #67633 (Update .mailmap) - #67635 (Document safety of Path casting) - #67654 (Add regression test for old NLL ICE) - #67659 (Stabilize the `matches!` macro) - #67664 (Fix some mailmap entries) Failed merges: r? @ghost

rust-highfive assigned withoutboats Dec 24, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 24, 2019

ranma42 reviewed Dec 24, 2019

View reviewed changes

rust-highfive assigned hanna-kruppe and unassigned withoutboats Dec 24, 2019

petertodd reviewed Dec 24, 2019

View reviewed changes

Kixunil force-pushed the nonnull-slice-iter branch from 9a654c8 to 047ff0b Compare December 25, 2019 20:29

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 25, 2019

JohnTitor mentioned this pull request Dec 26, 2019

Rollup of 7 pull requests #67625

Closed

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 26, 2019

JohnTitor reviewed Dec 26, 2019

View reviewed changes

src/libcore/slice/mod.rs Outdated Show resolved Hide resolved

Kixunil force-pushed the nonnull-slice-iter branch from 5e84631 to 2c796ee Compare December 26, 2019 19:18

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 27, 2019

oli-obk mentioned this pull request Dec 27, 2019

Rollup of 15 pull requests #67670

Merged

bors merged commit 2c796ee into rust-lang:master Dec 28, 2019

Kixunil deleted the nonnull-slice-iter branch February 8, 2024 16:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use NonNull in slice::Iter and slice::IterMut. #67588

Use NonNull in slice::Iter and slice::IterMut. #67588

Kixunil commented Dec 24, 2019

rust-highfive commented Dec 24, 2019

ranma42 Dec 24, 2019

Kixunil Dec 24, 2019

Kixunil Dec 24, 2019

hanna-kruppe Dec 25, 2019

Centril commented Dec 24, 2019

petertodd Dec 24, 2019

Kixunil Dec 24, 2019

petertodd Dec 24, 2019

petertodd Dec 24, 2019

Kixunil Dec 24, 2019

petertodd Dec 24, 2019

hanna-kruppe commented Dec 25, 2019

Kixunil commented Dec 25, 2019

hanna-kruppe commented Dec 25, 2019

bors commented Dec 25, 2019

Kixunil commented Dec 25, 2019

bors commented Dec 26, 2019

rust-highfive commented Dec 26, 2019

bors commented Dec 26, 2019

hanna-kruppe commented Dec 27, 2019

bors commented Dec 27, 2019

Use NonNull in slice::Iter and slice::IterMut. #67588

Use NonNull in slice::Iter and slice::IterMut. #67588

Conversation

Kixunil commented Dec 24, 2019

rust-highfive commented Dec 24, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Centril commented Dec 24, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanna-kruppe commented Dec 25, 2019

Kixunil commented Dec 25, 2019

hanna-kruppe commented Dec 25, 2019

bors commented Dec 25, 2019

Kixunil commented Dec 25, 2019

bors commented Dec 26, 2019

rust-highfive commented Dec 26, 2019

bors commented Dec 26, 2019

hanna-kruppe commented Dec 27, 2019

bors commented Dec 27, 2019