Improve API of Streams

[calvinlfer]

The current API of streams is quite sparse/low-level requiring the user of the API to drop down to the underlying representation of Emit in order to deal with complexity (if you take a look at Virgil Kyo to see me struggling 😂). The underlying representation feels akin to ZIO’s channel or FS2’s Pull but it would be much nicer to offer more higher level combinators that prevent/delay the user of the API to reach for lower levels immediately and allows one to stay at a very high level of abstraction until one cannot and requires fine grained control and performance. The idea at least naively would be to eventually replicate the API surface (or provide similar or better expressivity) of the other 2 streaming libraries.

[algora-pbc]

💎 $500 bounty • Kaizen Solutions

Steps to solve:

1. Start working: Comment /attempt #679 with your implementation plan
2. Submit work: Create a pull request including /claim #679 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to getkyo/kyo!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @maujim	Oct 31, 2024, 3:40:04 PM	WIP
🟢 @hearnadam	Nov 18, 2024, 5:35:27 AM	WIP

[maujim]

Can you please clarify what the acceptance criteria are? I would like to /attempt #679

Options

• Cancel my attempt

[hearnadam]

/attempt #679

Algora profile	Completed bounties	Tech	Active attempts	Options
@hearnadam	4 getkyo bounties	Scala, Shell, Rust		Cancel attempt

[calvinlfer]

Given my experience of having used FS2 Streams & ZIO Streams which are pull-based solutions. I would love to see the following, @hearnadam & @fwbrasil please feel free to add or subtract since you folks have way more expertise:

• The ability to evaluate multiple effects in parallel (mapZIOPar and mapAsync/parEvalMap) preserving order
• The ability to evaluate multiple effects in parallel (mapZIOParUnordered and mapAsync/parEvalMapUnordered) without preserving order
• The ability to perform rechunking on the stream elements
• The ability to group multiple elements together using time and space (groupedWithin(x.seconds, y.sizeOfChunk))
• The ability to merge multiple Streams together (not sure if this makes sense)
• An API like Pull/ZChannel to model Streams - we may already have this
• The ability to have different ways to evaluate streams (runDrain/runCollect/etc.)
• More high level operators to create Streams - like paginate/ZStreams.async
• Integration with the rest of the Kyo ecosystem (Queue/Channel/etc.)

[DamianReeves]

I'd suggest adding a merge operation as well.