-
Notifications
You must be signed in to change notification settings - Fork 34
Allow interleaved mapping in async iterators #262
Comments
I'd also love if we could even change the semantics of async generators so that |
@bergus See some discussion of that here, though of course such a change would not be in scope for this proposal in particular. I note that, with the change I'm proposing in this issue, you could get the same effect by doing |
It's technically allowed by the protocol, but is it a problem that this design allows for a Note that this would be a difference from the synchronous version: const results = [
{ done: false, value: "A" },
{ done: true },
{ done: false, value: "B" },
];
class CustomSyncIterator extends Iterator {
[Symbol.iterator]() { return this; }
#index = 0;
next() {
return results[this.#index++] ?? { done: true };
}
}
class CustomAsyncIterator extends Iterator {
[Symbol.asyncIterator]() { return this; }
#index = 0;
async next() {
return results[this.#index++] ?? { done: true };
}
}
const syncIterator = new CustomSyncIterator().map(value => value.repeat(5));
const syncResults = [
syncIterator.next(),
syncIterator.next(),
syncIterator.next(),
];
const asyncIterator = new CustomAsyncIterator().map(value => value.repeat(5));
const asyncResults = await Promise.all([
asyncIterator.next(),
asyncIterator.next(),
asyncIterator.next(),
]);
console.log(syncResults); // [{ done: false, value: "AAAAA" }, { done: true }, { done: true }]
console.log(asyncResults); // [{ done: false, value: "AAAAA" }, { done: true }, { done: false, value: "BBBBB" }] |
To be clear I'm not saying that my sample code would be literally the implementation, just demonstrating how it's possible to get parallelism. (Note that the sample code creates a new |
Yeah, this is all down the the level of the async iterator protocol. My initial impression is also that you won't be able to do this, because according to the async iterator protocol the only way to find out if an iterable is done is to await its next value. |
My implementation there isn't meant to be complete, but it does support concurrency if you call |
Yeah you're right. I withdraw that criticism. |
Closing this issue as "we expect to do this and need to work out the details". For discussion of the details, follow along and contribute at https://github.com/tc39/proposal-async-iterator-helpers. |
This is reviving #128, basically.
It would be nice if code like
could perform the fetches in parallel. Right now, because async iterator helpers are essentially "implemented" as async generators, it can't - the second call to
.next
will be queued until the first one finishes, rather than immediately being forwarded to the underlying iterator.If the implementation of
map
were differentlike this
It is less clear how, and whether, to allow parallelism in other helpers. I think they all have pretty natural semantics, but I have not yet worked through all of them in detail.
More speculatively, at a later date this would allow us to add a helper (say
.bufferAhead(N)
) to eagerly pump an async iterator and buffer the results. That would let you make any async iterator parallel with bounded concurrency, assuming the iterator was capable of supporting parallelism (so e.g..map
applied to the result of an async generator, but not an async generator itself), without changing the ordering semantics of the result.See slides here and some discussion here.
The text was updated successfully, but these errors were encountered: