`FusedIterator` marker trait and `iter::Fuse` specialization #1581

Stebalien · 2016-04-15T16:52:09Z

This RFC adds a FusedIterator marker trait and specializes iter::Fuse to do
nothing when the underlying iterator already provides the Fuse guarantee.

See this discussion for context.

rendered

WIP PR

/cc @bluss, @gankro

apasel422 · 2016-04-15T17:09:45Z

Should this discuss the consequences of an iterator implementing this trait without meeting its requirements? I don't know if it's necessary to make this an unsafe trait, but similar discussion has occurred around ExactSizeIterator.

See also #1051.

Stebalien · 2016-04-15T17:33:08Z

@apasel422, are you worried that someone might be relying on the behavior of Fused for safety? I can't see how this would happen in practice.

apasel422 · 2016-04-15T17:41:33Z

@Stebalien I'm not sure how it would happen in practice either, just pointing out some previous discussions around this matter.

This RFC adds a `FusedIterator` marker trait and specializes `iter::Fuse` to do nothing when the underlying iterator already provides the `Fuse` guarantee.

Stebalien · 2016-04-15T17:50:52Z

@apasel422 I've added it to the unresolved questions (it's a very good question and, unfortunately, one I can't answer). Maybe if you eagerly freed or truncated something immediately after the iterator returned None and then tried to reference it later? Even if we can't come up with a good example, it might be a good idea to make this unsafe (a) to be safe and (b) to be consistent with other potential specialization traits.

nagisa · 2016-04-15T18:39:20Z

text/0000-fused-iterator.md

+noticeable overhead. Furthermore, many iterators (most if not all iterators in
+std) already act as if they were fused (this is considered to be the "polite"
+behavior). Therefore, it would be nice to be able to pay the `Fused` overhead
+iff necessary.


s/iff necessary/if possible/.

No, I meant "if and only if necessary". Did you read an extra "not" into that sentence?

Did you read an extra "not" into that sentence?

Oh, I indeed did.

That being said, iff is equivalent to logic-↔ and

“nice to be able to pay the Fused overhead” → “necessary” ∧
“necessary” → “nice to be able to pay the Fused overhead”

doesn’t really look right to me. /me shrugs.

EDIT: I propose “only when necessary” as replacement.

Sure. FYI, when writing that sentence, I didn't mean to include the "nice to be able to" in the iff. That is, I meant "pay the overhead" → "necessary to pay the overhead" ∧ "necessary to pay the overhead" -> "pay the overhead". However, I guess the second part is implied.

nagisa · 2016-04-15T18:45:22Z

I feel like we could just specialise exact cases we already know to be fused (that wouldn’t even need an RFC!). The only benefit of FusedIterator I see is a reduction in code duplication at the much larger drawback of increased API surface.

Stebalien · 2016-04-15T18:52:35Z

The RFC hides quite a bit of code. You'd have to repeat all this each time you wanted to specialize. This also prevents programmers from specializing Fuse in arbitrary ways.

bluss · 2016-04-15T19:06:57Z

I think we will have some iterator traits with much stronger "regularity" guarantees than this, maybe something like TrustedIterator. It doesn't seem needed for FusedIterator to be unsafe. Isn't it a good thing if it's a simple safe trait?

Stebalien · 2016-04-15T19:15:57Z

@nagisa a more general approach is to make it possible to specialize all iterator adapters (added to the RFC). Unfortunately, this has some drawbacks (also added to the RFC).

rkjnsn · 2016-05-03T22:26:25Z

Does specialization as it exists today provide any way to omit the flag on Fuse (or make it a zero-sized type) when the inner iterator implements FusedIterator?

Stebalien · 2016-05-04T00:24:12Z

Yes. See rust-lang/rust#32999 (comment). Unfortunately, it's significantly more complicated and requires specializing associated types.

alexcrichton · 2016-07-26T22:57:30Z

🔔 This RFC is now entering its week-long final comment period 🔔

The libs team is leaning towards merging this as it provides a clear with for already-fused iterators and also allows the same benefits externally. Seems like a win win!

…ield.

Stebalien · 2016-07-27T14:45:01Z

@alexcrichton would you like me to squash?

alexcrichton · 2016-07-27T16:42:27Z

@Stebalien nah leaving all the commits as-is is fine, we actually prefer on RFCs to not squash!

ruuda · 2016-08-10T12:13:28Z

I just came here via TWiR, sorry to be late to the party. +1 for the idea, but I think the name is a bit unfortunate. It might be confusing to people who know about stream fusion, because “fusion” is used here in a similar context but with a totally different meaning. (Paper that uses the term too.) I’ve seen the term ”subject to fusion” used in Haskell documentation, but the FusedIterator trait would have a completely different meaning in Rust. That said, I haven’t seen the term “stream fustion” used in Rust documentation, so perhaps it is not an issue.

It is probably too late to change this now, as Iterator::fuse exists already, and if you think about it for a while then stream fusion doesn’t make sense in the context of Rust iterators. But still, it would be nice if we could avoid the ambiguity.

rkjnsn · 2016-08-11T19:39:54Z

@Stebalien Is the interface the same for both approaches? That is could we start with the simpler implementation and transition to the more complex but slightly more efficient one later?

Stebalien · 2016-08-11T19:46:16Z

@Stebalien Is the interface the same for both approaches? That is could we start with the simpler implementation and transition to the more complex but slightly more efficient one later?

Yes.

alexcrichton · 2016-08-11T20:49:29Z

Discussed at libs triage the other day, the decision was to merge

Thanks again for the RFC @Stebalien!

Tracking issue

Stebalien · 2016-08-11T21:07:00Z

I''ll revive the old PR ASAP but it might that might not be till September (impending thesis deadline).

Implement 1581 (FusedIterator) * [ ] Implement on patterns. See #27721 (comment). * [ ] Handle OS Iterators. A bunch of iterators (`Args`, `Env`, etc.) in libstd wrap platform specific iterators. The current ones all appear to be well-behaved but can we assume that future ones will be? * [ ] Does someone want to audit this? On first glance, all of the iterators on which I implemented `FusedIterator` appear to be well-behaved but there are a *lot* of them so a second pair of eyes would be nice. * I haven't touched rustc internal iterators (or the internal rand) because rustc doesn't actually call `fuse()`. * `FusedIterator` can't be implemented on `std::io::{Bytes, Chars}`. Closes: #35602 (Tracking Issue) Implements: rust-lang/rfcs#1581

Implement 1581 (FusedIterator) * [ ] Implement on patterns. See rust-lang/rust#27721 (comment). * [ ] Handle OS Iterators. A bunch of iterators (`Args`, `Env`, etc.) in libstd wrap platform specific iterators. The current ones all appear to be well-behaved but can we assume that future ones will be? * [ ] Does someone want to audit this? On first glance, all of the iterators on which I implemented `FusedIterator` appear to be well-behaved but there are a *lot* of them so a second pair of eyes would be nice. * I haven't touched rustc internal iterators (or the internal rand) because rustc doesn't actually call `fuse()`. * `FusedIterator` can't be implemented on `std::io::{Bytes, Chars}`. Closes: #35602 (Tracking Issue) Implements: rust-lang/rfcs#1581

FusedIterator marker trait and iter::Fuse specialization

1fd4a15

This RFC adds a `FusedIterator` marker trait and specializes `iter::Fuse` to do nothing when the underlying iterator already provides the `Fuse` guarantee.

Stebalien force-pushed the fused branch from eb2fd88 to 1fd4a15 Compare April 15, 2016 17:42

nagisa reviewed Apr 15, 2016
View reviewed changes

note associated type alternative

4b24c5d

remove potentially confusing iffs

2f7a030

hexsel mentioned this pull request Apr 16, 2016

[WIP] Implement FusedIterator rust-lang/rust#32999

Closed

nrc added the T-libs-api Relevant to the library API team, which will review and decide on the RFC. label Apr 17, 2016

aturon self-assigned this May 2, 2016

Stebalien mentioned this pull request Jun 18, 2016

Annotate all core iterators with their fusedness rust-lang/rust#34343

Closed

aturon added the I-nominated label Jul 25, 2016

alexcrichton added final-comment-period Will be merged/postponed/closed in ~10 calendar days unless new substational objections are raised. and removed I-nominated labels Jul 26, 2016

Stebalien added 2 commits July 27, 2016 10:38

Fix type names in motivation/microbenchmark.

055c500

Add unresolved question about removal of the Fuse::done unnecessary f…

c697b6a

…ield.

alexcrichton mentioned this pull request Aug 11, 2016

FusedIterator marker trait for iter::Fuse specialization rust-lang/rust#35602

Closed

alexcrichton merged commit c697b6a into rust-lang:master Aug 11, 2016

Stebalien mentioned this pull request Aug 13, 2016

Implement 1581 (FusedIterator) rust-lang/rust#35656

Merged

3 tasks

Centril added A-traits-libstd Standard library trait related proposals & ideas A-iterators Iterator related proposals & ideas A-specialization Proposals relating to specialization. labels Nov 23, 2018

ExpHP mentioned this pull request Apr 9, 2021

Iterator::fuse is not guaranteed to fuse a generic iterator. rust-lang/rust#83969

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`FusedIterator` marker trait and `iter::Fuse` specialization #1581

`FusedIterator` marker trait and `iter::Fuse` specialization #1581

Stebalien commented Apr 15, 2016 •

edited

Loading

apasel422 commented Apr 15, 2016 •

edited

Loading

Stebalien commented Apr 15, 2016

apasel422 commented Apr 15, 2016

Stebalien commented Apr 15, 2016

nagisa Apr 15, 2016

Stebalien Apr 15, 2016

nagisa Apr 15, 2016 •

edited

Loading

Stebalien Apr 15, 2016

nagisa commented Apr 15, 2016

Stebalien commented Apr 15, 2016

bluss commented Apr 15, 2016

Stebalien commented Apr 15, 2016

rkjnsn commented May 3, 2016

Stebalien commented May 4, 2016

alexcrichton commented Jul 26, 2016

Stebalien commented Jul 27, 2016

alexcrichton commented Jul 27, 2016

ruuda commented Aug 10, 2016

rkjnsn commented Aug 11, 2016

Stebalien commented Aug 11, 2016

alexcrichton commented Aug 11, 2016

Stebalien commented Aug 11, 2016

FusedIterator marker trait and iter::Fuse specialization #1581

FusedIterator marker trait and iter::Fuse specialization #1581

Conversation

Stebalien commented Apr 15, 2016 • edited Loading

apasel422 commented Apr 15, 2016 • edited Loading

Stebalien commented Apr 15, 2016

apasel422 commented Apr 15, 2016

Stebalien commented Apr 15, 2016

nagisa Apr 15, 2016

Choose a reason for hiding this comment

Stebalien Apr 15, 2016

Choose a reason for hiding this comment

nagisa Apr 15, 2016 • edited Loading

Choose a reason for hiding this comment

Stebalien Apr 15, 2016

Choose a reason for hiding this comment

nagisa commented Apr 15, 2016

Stebalien commented Apr 15, 2016

bluss commented Apr 15, 2016

Stebalien commented Apr 15, 2016

rkjnsn commented May 3, 2016

Stebalien commented May 4, 2016

alexcrichton commented Jul 26, 2016

Stebalien commented Jul 27, 2016

alexcrichton commented Jul 27, 2016

ruuda commented Aug 10, 2016

rkjnsn commented Aug 11, 2016

Stebalien commented Aug 11, 2016

alexcrichton commented Aug 11, 2016

Stebalien commented Aug 11, 2016

`FusedIterator` marker trait and `iter::Fuse` specialization #1581

`FusedIterator` marker trait and `iter::Fuse` specialization #1581

Stebalien commented Apr 15, 2016 •

edited

Loading

apasel422 commented Apr 15, 2016 •

edited

Loading

nagisa Apr 15, 2016 •

edited

Loading