Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Basic StreamExt::{flatten_unordered, flat_map_unordered} impls #2083

Merged
merged 23 commits into from
Feb 6, 2022
Merged

Basic StreamExt::{flatten_unordered, flat_map_unordered} impls #2083

merged 23 commits into from
Feb 6, 2022

Conversation

olegnn
Copy link
Contributor

@olegnn olegnn commented Feb 17, 2020 •
edited
Loading

Closes #1781
Closes #2378

Basic implementations of flatten_unordered and flat_map_unordered using FuturesUnordered in combination with custom poll state and wakers under the hood. I didn't measure performance but I think it's fast enough. It differs from flatten and flat_map because it polls stream and inner streams concurrently.

Edited: I added benchmarks, which can be compared with futures_unordered. flatten_unordered with a stream of 10000 oneshots each of which produces an inner stream with one item taking approx about x1.5 time of the original futures_unordered benchmark with 10000 oneshots.

New update: after changing the logic to make a stream being polled in a loop instead of once per poll_next, flatten_unordered takes about 1 - 1.1x time of the original futures_unordered benchmark.

@olegnn olegnn force-pushed the stream_flat_map_unordered branch from c91d08b to 090d95d Compare February 17, 2020 14:45
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from 4b94925 to dae2bb9 Compare February 17, 2020 16:29
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from 0b40d25 to bd9a86c Compare February 18, 2020 17:32
@olegnn olegnn changed the title Basic StreamExt::flat_map_unordered impl Basic StreamExt::{flatten_unordered, flat_map_unordered} impls May 17, 2020
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 5ba808d to 6adcfee Compare May 17, 2020 13:10
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 2ca3e96 to f2ddcfd Compare May 28, 2020 16:33
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 8 times, most recently from e402c5f to 1324ea9 Compare June 12, 2020 14:16
@olegnn olegnn requested a review from taiki-e January 9, 2022 14:56
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 3 times, most recently from fe55019 to 952ccf7 Compare January 9, 2022 22:13
taiki-e
taiki-e reviewed Jan 11, 2022
View reviewed changes
Copy link
Member

@taiki-e taiki-e left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

futures-util/src/stream/stream/flatten_unordered.rs Outdated Show resolved Hide resolved
futures-util/src/stream/stream/mod.rs Outdated Show resolved Hide resolved
futures/tests/stream.rs Outdated Show resolved Hide resolved
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from bf4a826 to 12bee98 Compare January 18, 2022 19:59
taiki-e
taiki-e reviewed Jan 22, 2022
View reviewed changes
futures/tests/stream.rs Outdated Show resolved Hide resolved
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from d632a0f to 4091d66 Compare January 23, 2022 12:29
@olegnn olegnn requested a review from taiki-e January 26, 2022 13:24
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 2 times, most recently from 6096ba6 to 330d93d Compare January 26, 2022 13:55
@olegnn olegnn force-pushed the stream_flat_map_unordered branch 3 times, most recently from f55cdcd to c6a9357 Compare January 26, 2022 22:47
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from c6a9357 to 69f60cc Compare January 26, 2022 22:52
@olegnn olegnn force-pushed the stream_flat_map_unordered branch from d600a6a to a50c9e4 Compare January 27, 2022 12:40
taiki-e
taiki-e approved these changes Feb 6, 2022
View reviewed changes
Copy link
Member

@taiki-e taiki-e left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@taiki-e taiki-e merged commit 9cb6723 into rust-lang:master Feb 6, 2022
@taiki-e taiki-e added the 0.3-backport: pending The maintainer accepted to backport this to the 0.3 branch, but backport has not been done yet. label Feb 6, 2022
taiki-e pushed a commit that referenced this pull request Feb 6, 2022
@olegnn @taiki-e
Basic StreamExt::{flatten_unordered, flat_map_unordered} impls (#2083)
e780fa0
@taiki-e taiki-e mentioned this pull request Feb 6, 2022
Merged
@taiki-e taiki-e added 0.3-backport: completed and removed 0.3-backport: pending The maintainer accepted to backport this to the 0.3 branch, but backport has not been done yet. labels Feb 6, 2022
taiki-e pushed a commit that referenced this pull request Feb 6, 2022
@olegnn @taiki-e
Basic StreamExt::{flatten_unordered, flat_map_unordered} impls (#2083)
94b508b
@taiki-e
Copy link
Member

taiki-e commented Feb 6, 2022

Published in 0.3.20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@taiki-e taiki-e taiki-e approved these changes

Assignees
No one assigned
Labels
0.3-backport: completed A-stream Area: futures::stream
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

StreamExt::flat_map_unordered Stream::flatten_unordered would be nice
2 participants
@olegnn @taiki-e