-
Notifications
You must be signed in to change notification settings - Fork 10.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spliterators for sorted collections cause inefficient stream operations when natural ordering is used #6187
Comments
Spliterators returned by CollectSpliterators.indexed and ImmutableSortedSet.spliterator violated the Spliterator API by not returning null in getComparator when the source items are naturally sorted. The effect was that sort operations on Streams backed by these Spliterators were not optimized away, resulting in additional unnecessary sorting. Fixes google#6187.
Nice find, and thanks for the deep links. It's of course generally impossible to know whether a We should search for a JDK issue; given that it introduced that |
Oh funny, I didn't bother mentioning it could special-case |
Spliterators returned by CollectSpliterators.indexed and ImmutableSortedSet.spliterator violated the Spliterator API by not returning null in getComparator when the source items are naturally sorted. The effect was that sort operations on Streams backed by these Spliterators were not optimized away, resulting in additional unnecessary sorting. Fixes google#6187.
I agree that the JDK ought to handle this check itself, although it wouldn't help Guava at the moment due to |
This has always struck me as a weird feature, but people use it, and it "works" in the sense of "does not cause NPE" (though it may cause CCE...). I see both calls to `sort(null)` and calls like `sort(priorityQueue.comparator())` (which _might_ be `null`; [example](https://github.com/google/guava/blob/e82e2a2c07c68108f318958ee0355cc835c97743/guava-testlib/src/com/google/common/collect/testing/testers/SortedSetNavigationTester.java#L57)). And while I prefer a world in which methods like `comparator()` never return `null`, as we arranged for [in `SortedMultiset`](https://guava.dev/SortedMultiset#comparator()), there are apparently [downsides](google/guava#6187) to using a `Comparator` that implements natural order rather than using `null`. Is any of that convincing? :) This is definitely a case in which I can see how eisop would want to stay on its own path. My main motivation for doing this now is that I hear that the current signature is causing us trouble during Java->Kotlin transpilation. I can get more details.
* Annotate the parameter of `List.sort` as `@Nullable`. This has always struck me as a weird feature, but people use it, and it "works" in the sense of "does not cause NPE" (though it may cause CCE...). I see both calls to `sort(null)` and calls like `sort(priorityQueue.comparator())` (which _might_ be `null`; [example](https://github.com/google/guava/blob/e82e2a2c07c68108f318958ee0355cc835c97743/guava-testlib/src/com/google/common/collect/testing/testers/SortedSetNavigationTester.java#L57)). And while I prefer a world in which methods like `comparator()` never return `null`, as we arranged for [in `SortedMultiset`](https://guava.dev/SortedMultiset#comparator()), there are apparently [downsides](google/guava#6187) to using a `Comparator` that implements natural order rather than using `null`. Also, Werner points us to `Arrays.sort`. (#13)
The Spliterators returned by sorted collections such as
ImmutableSortedSet
andRegularImmutableAsList
result in unnecessary sorting occurring when a sort operation is performed on their associated Stream. The root cause is that theSpliterators'
getComparator()
implementations always return aComparator
instance, even when natural ordering is used.According to the Javadoc,
null
should be returned bygetComparator()
to indicate the source items are sorted in natural order.You can see here that the
SORTED
flag gets unset when a spliterator has theSORTED
characteristic but does not return anull
comparator. And here is where the missed optimization happens due to that.The issue is trivially reproducible by doing the following, and stepping through the code in SortedOps where the expected no-op / optimization would ideally happen:
The text was updated successfully, but these errors were encountered: