Improve performance of other non GroupsAdapter aggregates: implement convert_to_state
#11819
Labels
enhancement
New feature or request
convert_to_state
#11819
Is your feature request related to a problem or challenge?
@korowa added "skip partial aggregation mode" in #11627 which helps with high cardinality aggregates by doing minimal work for the first phase of the aggregation. This mode is triggered dynamically based on how effective the first aggregation phase is working.
In order to use this new mode, the corresponding
GroupsAccumulator
needs to implement theconvert_to_state
methoddatafusion/datafusion/expr/src/groups_accumulator.rs
Lines 166 to 213 in c340b6a
Some aggregates implement the GroupsAccumulator interface directly, but by default they will use the
GroupsAccumulatorAdapter
along with theAccumulator
traitDescribe the solution you'd like
Implement
covert_to_state
forhttps://github.com/apache/datafusion/blob/b685e2d4f1f245dd1dbe468b32b115ae99316689/datafusion/physical-expr/src/aggregate/groups_accumulator/adapter.rs#L247-L246
Add tests in
datafusion/datafusion/sqllogictest/test_files/aggregate_skip_partial.slt
Lines 18 to 19 in c340b6a
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: