Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid storing more transformed elements than necessary #3376

Merged
merged 6 commits into from
Feb 18, 2024

Conversation

serras
Copy link
Member

@serras serras commented Feb 16, 2024

This is a counter-proposal for #3374 @kyay10

This introduces a new (internal) version of forEachAccumulating which works differently before and after the error. That way we can simply stop adding things after the first error.

@serras serras self-assigned this Feb 16, 2024
Copy link
Contributor

github-actions bot commented Feb 16, 2024

Kover Report

File Coverage [65.41%]
arrow-libs/core/arrow-core/src/commonMain/kotlin/arrow/core/raise/RaiseAccumulate.kt 65.41%
Total Project Coverage 52.81%

Copy link
Member

@nomisRev nomisRev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach @serras.

I think this would make the PR #3374 unnecessary, since the optimisation is applied in the base implementation. 👍

Comment on lines 571 to 572
@BuilderInference blockUntilError: RaiseAccumulate<Error>.(A) -> Unit,
@BuilderInference blockAfterError: RaiseAccumulate<Error>.(A) -> Unit
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Do we really need two lambdas here? 🤔

Or was that for JvmName? Otherwise I am also fine using forEachAccumulatingImpl or something. I've started doing that in my own projects, similar how to KotlinX Coroutines does this with produceImpl. flattenMergeImpl, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do, because we still need to call transform on each value, just not add the result to the list.

Copy link
Collaborator

@kyay10 kyay10 Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have:

  @BuilderInference block: RaiseAccumulate<Error>.(A) -> B,
  add: (A, B) -> Unit

and simply we only call add if we don't have any errors.
We could also just have a Boolean parameter instead, with similar effects.

I really also dislike the binary-size blowup that can occur because mapOrAccumulate inlines its block in 2 different places.

) {
val error: MutableList<Error> = mutableListOf()
var errorArose = false
Copy link
Collaborator

@kyay10 kyay10 Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: same as above, but with error.isEmpty()

Copy link
Collaborator

@kyay10 kyay10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about the unnecessary duplication of the lambdas of forEachAccumulating and mapOrAccumulate.

@serras
Copy link
Member Author

serras commented Feb 18, 2024

Indeed, @kyay10's comment about size explosion was on point. I've reworked the implementation to use a version with a single function, with a Boolean with errors. (Note however that this function is only exposed for inline purposes, not for other users of the library)

Copy link
Collaborator

@kyay10 kyay10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this!!

@@ -521,10 +521,19 @@ public inline fun <Error, A> Raise<Error>.forEachAccumulating(
iterator: Iterator<A>,
combine: (Error, Error) -> Error,
@BuilderInference block: RaiseAccumulate<Error>.(A) -> Unit
): Unit = forEachAccumulatingImpl(iterator, combine) { item, _ -> block(item) }

@PublishedApi @JvmSynthetic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synthetic is a nice touch here! Note that this technically doesn't 100% absolve us of ABI compatibility issues according to the discussion on Kotlin/binary-compatibility-validator#165. I wish we had access to @InlineOnly, then maybe BCV would consider those methods to not be part of our ABI, but alas.

@serras serras merged commit e2c53e8 into main Feb 18, 2024
11 checks passed
@serras serras deleted the serras/for-each-accum branch February 18, 2024 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants