Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix in-place collect not reallocating when necessary #118460

Merged
merged 1 commit into from
Dec 6, 2023

Conversation

the8472
Copy link
Member

@the8472 the8472 commented Nov 29, 2023

Regression introduced in #110353.
This was caught by miri

r? @saethlin

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 29, 2023
@@ -168,7 +168,7 @@ const fn in_place_collectible<DEST, SRC>(
step_merge: Option<NonZeroUsize>,
step_expand: Option<NonZeroUsize>,
) -> bool {
if DEST::IS_ZST || mem::align_of::<SRC>() < mem::align_of::<DEST>() {
if const { SRC::IS_ZST || DEST::IS_ZST || mem::align_of::<SRC>() < mem::align_of::<DEST>() } {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not part of the fix. the condition was already enforced before, I'm just making it more explicit for readability.

@rust-log-analyzer

This comment has been minimized.

@RalfJung
Copy link
Member

I can confirm that with this patch applied, the test suite passes in Miri.

@saethlin
Copy link
Member

saethlin commented Dec 2, 2023

The current implementation in this PR looks like it is trying to enumerate the cases where reallocation is not permitted, then if none of those cases are hit we fall through to permitting allocation reuse. I would much prefer to see this implemented with the inverse strategy; try to enumerate the cases where allocation reuse is permitted, then fall through to requiring reallocation. Alignment is trivial though, so I don't mind handling that differently.

So I'd prefer to see something where either we early-return false and default to returning true.

That being said, I slapped together this alternative implementation and it seems to pass the test suite:

const fn needs_realloc<SRC, DEST>(src_cap: usize, dst_cap: usize) -> bool {
    // Alignment must match exactly. Beyond this check we do not need to reason about it.
    if const { mem::align_of::<SRC>() != mem::align_of::<DEST>() } {
        return true;
    }

    // Exact allocation size matches can be reused
    let src_bytes = src_cap * mem::size_of::<SRC>();
    let dest_bytes = dst_cap * mem::size_of::<DEST>();
    if src_bytes == dest_bytes {
        return false;
    }

    // Everything else needs a realloc
    true
}

This implementation seems much simpler to me and I'd prefer to see it (or something similar) used if it is equivalent. Is it? If it isn't, I think we're missing a test.

@the8472
Copy link
Member Author

the8472 commented Dec 2, 2023

The difference is that I'm doing more const checks to find cases where we don't need to do runtime checks to figure out whether realloc is needed. I'd have to write codegen tests to catch that.
But sure, I can invert the condition.

bors added a commit to rust-lang/miri that referenced this pull request Dec 3, 2023
RalfJung pushed a commit to RalfJung/rust that referenced this pull request Dec 3, 2023
@the8472
Copy link
Member Author

the8472 commented Dec 3, 2023

I rewrote the needs_realloc conditions and also added a check for zero-sized allocations to catch up with #113113 which happened after I originally wrote the realloc code.

Comment on lines 194 to 195
// If src size is an integer multiple of the destination size
// the resulting capacity will also always be an integer multiple without remainder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// If src size is an integer multiple of the destination size
// the resulting capacity will also always be an integer multiple without remainder
// If src size is an integer multiple of the destination size, we know the capacity we calculate
// with src_cap * size_of::<SRC> / size_of::<DEST> does not have a remainder.

It took me a long time to understand this comment. It's referring to the division done outside this function. I've tried to make that clear.

r=me with this suggestion or equivalent clarification

@the8472
Copy link
Member Author

the8472 commented Dec 5, 2023

@bors r=saethlin

@bors
Copy link
Contributor

bors commented Dec 5, 2023

📌 Commit 13a843e has been approved by saethlin

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 5, 2023
@RalfJung
Copy link
Member

RalfJung commented Dec 6, 2023

@bors p=1
this unblocks miri-test-libstd

@bors
Copy link
Contributor

bors commented Dec 6, 2023

⌛ Testing commit 13a843e with merge 15bb3e2...

@bors
Copy link
Contributor

bors commented Dec 6, 2023

☀️ Test successful - checks-actions
Approved by: saethlin
Pushing 15bb3e2 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Dec 6, 2023
@bors bors merged commit 15bb3e2 into rust-lang:master Dec 6, 2023
12 checks passed
@rustbot rustbot added this to the 1.76.0 milestone Dec 6, 2023
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (15bb3e2): comparison URL.

Overall result: ❌ regressions - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.4% [0.4%, 0.4%] 1
Regressions ❌
(secondary)
1.0% [1.0%, 1.0%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.4% [0.4%, 0.4%] 1

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
3.2% [3.2%, 3.2%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.1% [-2.1%, -2.1%] 1
All ❌✅ (primary) - - 0

Binary size

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.0% [0.0%, 0.2%] 26
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.2% [-0.2%, -0.2%] 2
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.0% [-0.2%, 0.2%] 28

Bootstrap: 674.58s -> 675.083s (0.07%)
Artifact size: 314.17 MiB -> 314.18 MiB (0.00%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants