-
Notifications
You must be signed in to change notification settings - Fork 753
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: update sort algorithm using loser tree for multi sort merge #15869
Conversation
Signed-off-by: coldWater <forsaken628@gmail.com>
Signed-off-by: coldWater <forsaken628@gmail.com>
Signed-off-by: coldWater <forsaken628@gmail.com>
Is there any performance comparison data of this pr? |
Not yet, do I need to add an algorithm level benchmark, or just ci-benchmark? |
Signed-off-by: coldWater <forsaken628@gmail.com>
you can pref it in local, eg:
|
No noticeable changes. There seems to be a slight improvement from flamegraph, not obvious. It could also be due to the distribution of the data. data setread rows: 25000000 sqlsortselect * from t order by a,b ignore_result; windowSELECT MAX(d) OVER (PARTITION BY a) FROM t ignore_result; |
The codes LGTM. Some comments about this pr: Better add a new setting named Then we can create the |
Signed-off-by: coldWater <forsaken628@gmail.com>
src/query/pipeline/transforms/src/processors/transforms/sort/rows/simple.rs
Show resolved
Hide resolved
Signed-off-by: coldWater <forsaken628@gmail.com>
Signed-off-by: coldWater <forsaken628@gmail.com>
Here's another part about Merger's changes. forsaken628/databend@loser-tree...forsaken628:databend:peek-mut The cursor contains Arc, and iterative cloning and deletion incurs a lot of overhead, so I'm replacing cloning with in-place updating. Should these two parts of the changes be separately PR or combined? |
You can add into this pr directly. |
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
Fixes #11604
Tests
Type of change
This change is