-
Notifications
You must be signed in to change notification settings - Fork 34
Concurrency in async iterators? #203
Comments
That seems like something wildly out of scope of this proposal. |
While the idea of iterators which eagerly start work early is interesting, and is something we've talked about some before (e.g. x and the following day), it's definitely not in scope for this particular proposal - there's a lot of design space here, and it would serve a pretty different function than the helpers here. |
What is the process for follow up proposals? Do I have to wait for this to land or reach a certain stage before submitting a stand alone proposal? |
The process for contributing is documented here. No need to wait for this to advance to put something together, though do note you'll need to get someone who serves on the committee to champion it. |
After thinking about this more, I think that leaving the door open for a follow-on proposal here would actually require a tweak to the semantics in the helpers currently in this proposal. I've opened #262 to track. I like the idea of a helper to eagerly pull from an iterator multiple times and buffer results, like your |
The async iterator helpers described in this spec are serial -
await it.next()
, thenawait mapper(value)
, then loop.This example from the readme would only send one fetch request at a time.
This example, using aws-sdk paginators, sends one query api call then one fetch at a time for each item in the page before starting the next page query.
Operations like this can improve throughput with concurrent execution.
Could async iterator helpers support eagerly getting the next items and buffering them for the next call?
Could the helpers call their callback functions concurrently, before waiting for their results?
This could be implemented as arguments to each helper:
Or their own helper method
Parallel could support both ordered and unordered execution.
I think this request could be extended to
for await
loops as wellThe text was updated successfully, but these errors were encountered: