document `FromIterator for Vec` allocation behaviors #120355

the8472 · 2024-01-25T23:45:41Z

t-libs discussion about #120091 didn't reach a strong consensus, but it was agreed that if we keep the current behavior it should at least be documented even though it is an implementation detail.

The language is intentionally non-committal. The previous (non-existent) documentation permits a lot of implementation leeway and we want retain that. In some cases we even must retain it to be able to rip out some code paths that rely on unstable features.

rustbot · 2024-01-25T23:45:48Z

r? @cuviper

(rustbot has picked a reviewer for you, use r? to override)

library/alloc/src/vec/mod.rs

cuviper · 2024-01-26T01:33:17Z

library/alloc/src/vec/mod.rs

+/// allocations are created, only a small fraction of their items gets collected, no further use
+/// is made of the spare capacity and the resulting `Vec` is moved into a longer-lived structure
+/// this can lead to the large allocations having their lifetimes unnecessarily extended which
+/// can result in increased memory footprint.


This a long run-on sentence -- maybe:

Suggested change

/// allocations are created, only a small fraction of their items gets collected, no further use

/// is made of the spare capacity and the resulting `Vec` is moved into a longer-lived structure

/// this can lead to the large allocations having their lifetimes unnecessarily extended which

/// can result in increased memory footprint.

/// allocations are created, and only a small fraction of their items get collected, no further use

/// is made of the spare capacity. If the resulting `Vec` is moved into a longer-lived structure,

/// this can lead to the large allocations having their lifetimes unnecessarily extended which

/// can result in increased memory footprint.

But this scenario about "a large number" of allocations seems a bit too specific to me. It could just as well be one giant vector that gets reduced without any reclaim. I think the point we want to get across is that we may reuse your buffer even if the result has much more capacity that it needs.

If you only do it once then you already paid the max-RSS cost with the temporary allocation. You have to do something like this more than once to blow up max-RSS.

Focusing on RSS assumes an OS that overcommits memory, which isn't universal. ~~And even in that case, all it takes is for the code to fill the "original" vector again to pump up RSS.~~ (edit: nevermind the latter, I mixed that up with the split_off discussion.)

If you build a giant vector, and then filter-collect it to a much smaller size, you might reasonably expect that giant memory block to now be available for other allocations. This can have an impact even just once.

This a long run-on sentence -- maybe

It's supposed to be an "But when a), b) and then c) [happen] this can result X."
The suggested . broke it to "But when a), b). If c) [happens] this can result X."

Hmm, yes that's broken -- that's what I get for not quoting the whole thing when trying to revise it.

library/alloc/src/vec/mod.rs

the8472 · 2024-01-30T20:49:04Z

It'd be good if we could get ready to merge within the next week. I have put a link to the to-be-updated nightly documentation in the release notes (#120004) as context.
Or I'll need to find another place to link to.

library/alloc/src/vec/mod.rs

Co-authored-by: Josh Stone <cuviper@gmail.com>

cuviper · 2024-01-30T21:59:28Z

@bors r+ rollup

bors · 2024-01-30T21:59:30Z

📌 Commit 39dc315 has been approved by cuviper

It is now in the queue for this repository.

document `FromIterator for Vec` allocation behaviors [t-libs discussion](https://rust-lang.zulipchat.com/#narrow/stream/259402-t-libs.2Fmeetings/topic/Meeting.202024-01-24/near/417686526) about rust-lang#120091 didn't reach a strong consensus, but it was agreed that if we keep the current behavior it should at least be documented even though it is an implementation detail. The language is intentionally non-committal. The previous (non-existent) documentation permits a lot of implementation leeway and we want retain that. In some cases we even must retain it to be able to rip out some code paths that rely on unstable features.

Rollup of 12 pull requests Successful merges: - rust-lang#120207 (check `RUST_BOOTSTRAP_CONFIG` in `profile_user_dist` test) - rust-lang#120321 (pattern_analysis: cleanup the contexts) - rust-lang#120323 (On E0277 be clearer about implicit `Sized` bounds on type params and assoc types) - rust-lang#120355 (document `FromIterator for Vec` allocation behaviors) - rust-lang#120396 (Account for unbounded type param receiver in suggestions) - rust-lang#120430 (std: thread_local::register_dtor fix proposal for FreeBSD.) - rust-lang#120435 (Suggest name value cfg when only value is used for check-cfg) - rust-lang#120470 (Mark "unused binding" suggestion as maybe incorrect) - rust-lang#120472 (Make duplicate lang items fatal) - rust-lang#120490 (Don't hash lints differently to non-lints.) - rust-lang#120495 (Remove the `abi_amdgpu_kernel` feature) - rust-lang#120501 (rustdoc: Correctly handle attribute merge if this is a glob reexport) Failed merges: - rust-lang#120346 (hir: Refactor getters for owner nodes) r? `@ghost` `@rustbot` modify labels: rollup

document `FromIterator for Vec` allocation behaviors [t-libs discussion](https://rust-lang.zulipchat.com/#narrow/stream/259402-t-libs.2Fmeetings/topic/Meeting.202024-01-24/near/417686526) about rust-lang#120091 didn't reach a strong consensus, but it was agreed that if we keep the current behavior it should at least be documented even though it is an implementation detail. The language is intentionally non-committal. The previous (non-existent) documentation permits a lot of implementation leeway and we want retain that. In some cases we even must retain it to be able to rip out some code paths that rely on unstable features.

Rollup of 9 pull requests Successful merges: - rust-lang#120207 (check `RUST_BOOTSTRAP_CONFIG` in `profile_user_dist` test) - rust-lang#120321 (pattern_analysis: cleanup the contexts) - rust-lang#120355 (document `FromIterator for Vec` allocation behaviors) - rust-lang#120430 (std: thread_local::register_dtor fix proposal for FreeBSD.) - rust-lang#120469 (Provide more context on derived obligation error primary label) - rust-lang#120470 (Mark "unused binding" suggestion as maybe incorrect) - rust-lang#120472 (Make duplicate lang items fatal) - rust-lang#120495 (Remove the `abi_amdgpu_kernel` feature) - rust-lang#120501 (rustdoc: Correctly handle attribute merge if this is a glob reexport) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 9 pull requests Successful merges: - rust-lang#120207 (check `RUST_BOOTSTRAP_CONFIG` in `profile_user_dist` test) - rust-lang#120321 (pattern_analysis: cleanup the contexts) - rust-lang#120355 (document `FromIterator for Vec` allocation behaviors) - rust-lang#120430 (std: thread_local::register_dtor fix proposal for FreeBSD.) - rust-lang#120469 (Provide more context on derived obligation error primary label) - rust-lang#120472 (Make duplicate lang items fatal) - rust-lang#120490 (Don't hash lints differently to non-lints.) - rust-lang#120495 (Remove the `abi_amdgpu_kernel` feature) - rust-lang#120501 (rustdoc: Correctly handle attribute merge if this is a glob reexport) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#120355 - the8472:doc-vec-fromiter, r=cuviper document `FromIterator for Vec` allocation behaviors [t-libs discussion](https://rust-lang.zulipchat.com/#narrow/stream/259402-t-libs.2Fmeetings/topic/Meeting.202024-01-24/near/417686526) about rust-lang#120091 didn't reach a strong consensus, but it was agreed that if we keep the current behavior it should at least be documented even though it is an implementation detail. The language is intentionally non-committal. The previous (non-existent) documentation permits a lot of implementation leeway and we want retain that. In some cases we even must retain it to be able to rip out some code paths that rely on unstable features.

rustbot assigned cuviper Jan 25, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jan 25, 2024

the8472 mentioned this pull request Jan 25, 2024

Release notes for 1.76 #120004

Merged

cuviper reviewed Jan 26, 2024

View reviewed changes

Kobzol reviewed Jan 26, 2024

View reviewed changes

library/alloc/src/vec/mod.rs Outdated Show resolved Hide resolved

the8472 force-pushed the doc-vec-fromiter branch from 29d8d40 to 57009e1 Compare January 26, 2024 23:34

the8472 requested a review from cuviper January 28, 2024 12:01

kadiwa4 reviewed Jan 28, 2024

View reviewed changes

library/alloc/src/vec/mod.rs Outdated Show resolved Hide resolved

the8472 force-pushed the doc-vec-fromiter branch from 3be4a96 to 8888bfc Compare January 29, 2024 00:00

cuviper reviewed Jan 30, 2024

View reviewed changes

library/alloc/src/vec/mod.rs Outdated Show resolved Hide resolved

library/alloc/src/vec/mod.rs Outdated Show resolved Hide resolved

the8472 and others added 2 commits January 30, 2024 22:37

document FromIterator for Vec allocation behaviors

c780fe6

Apply suggestions from code review

39dc315

Co-authored-by: Josh Stone <cuviper@gmail.com>

the8472 force-pushed the doc-vec-fromiter branch from 2cddfa5 to 39dc315 Compare January 30, 2024 21:38

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 30, 2024

Nadrieril mentioned this pull request Jan 31, 2024

Rollup of 12 pull requests #120519

Closed

Nadrieril mentioned this pull request Jan 31, 2024

Rollup of 9 pull requests #120522

Closed

Nadrieril mentioned this pull request Jan 31, 2024

Rollup of 9 pull requests #120524

Merged

bors merged commit 03daaa6 into rust-lang:master Jan 31, 2024
11 checks passed

rustbot added this to the 1.77.0 milestone Jan 31, 2024

the8472 mentioned this pull request Jan 31, 2024

In-place iteration results in too big allocations #120091

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

document `FromIterator for Vec` allocation behaviors #120355

document `FromIterator for Vec` allocation behaviors #120355

the8472 commented Jan 25, 2024 •

edited

Loading

rustbot commented Jan 25, 2024

cuviper Jan 26, 2024

the8472 Jan 26, 2024

cuviper Jan 26, 2024 •

edited

Loading

cuviper Jan 26, 2024

the8472 Jan 26, 2024

cuviper Jan 26, 2024

the8472 commented Jan 30, 2024

cuviper commented Jan 30, 2024

bors commented Jan 30, 2024

document FromIterator for Vec allocation behaviors #120355

document FromIterator for Vec allocation behaviors #120355

Conversation

the8472 commented Jan 25, 2024 • edited Loading

rustbot commented Jan 25, 2024

cuviper Jan 26, 2024

Choose a reason for hiding this comment

the8472 Jan 26, 2024

Choose a reason for hiding this comment

cuviper Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

cuviper Jan 26, 2024

Choose a reason for hiding this comment

the8472 Jan 26, 2024

Choose a reason for hiding this comment

cuviper Jan 26, 2024

Choose a reason for hiding this comment

the8472 commented Jan 30, 2024

cuviper commented Jan 30, 2024

bors commented Jan 30, 2024

document `FromIterator for Vec` allocation behaviors #120355

document `FromIterator for Vec` allocation behaviors #120355

the8472 commented Jan 25, 2024 •

edited

Loading

cuviper Jan 26, 2024 •

edited

Loading