[GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions #8105

taiyang-li · 2024-11-29T09:57:01Z

What changes were proposed in this pull request?

Changes:

Refactor and simplify implementation for sparkDecimal* functions
Use clang built-in i256 to replace CH Int256 in sparkDecimal* functions.
Support JIT for sparkDivide and sparkCastFloatToInt* functions(default disabled)
Support JIT for sparkDecimal* functions (default disabled)
Support JIT for cast from/to decimal functions (default disabled)

(Fixes: #7534)

Benefits:

JIT: no benefits, note that JIT is disabled by default because of performance regression caused by llvm-library upgrade([GLUTEN-7534][CH] Support JIT for more expressions and decimal types(except decimal256 and int256) Kyligence/ClickHouse#509 (comment)).
Refactor and optimize sparkDecimal* functions: speed up by 6% in our production query.

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

github-actions · 2024-11-29T09:57:21Z

#7534

github-actions · 2024-11-29T09:57:32Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T04:01:48Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T06:52:16Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T06:55:38Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T09:20:55Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T10:34:28Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-02T10:44:49Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-03T02:07:24Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-03T03:07:36Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-03T07:01:38Z

Run Gluten Clickhouse CI on x86

taiyang-li · 2024-12-03T09:56:38Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-03T09:57:53Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-04T02:45:51Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-04T03:43:40Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-05T10:48:38Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-05T11:21:53Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-06T02:46:41Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-06T03:59:46Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-06T04:59:20Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-06T07:56:02Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-06T10:11:45Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-10T03:57:37Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-10T10:43:37Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-25T07:52:24Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-25T07:55:04Z

Run Gluten Clickhouse CI on x86

github-actions · 2024-12-26T04:18:04Z

Run Gluten Clickhouse CI on x86

taiyang-li · 2024-12-26T09:26:23Z

We found serious performance regression after ClickHouse bumping llvm from 15.x to 18.x. Turn it into draft status.

github-actions · 2024-12-26T10:08:32Z

Run Gluten Clickhouse CI on x86

baibaichen · 2024-12-27T00:58:44Z

We need performace test for this PR.

baibaichen · 2024-12-27T03:25:02Z

let's add benchmark codes @taiyang-li

github-actions · 2024-12-27T06:17:19Z

Run Gluten Clickhouse CI on x86

baibaichen

LGTM

github-actions bot added the CLICKHOUSE label Nov 29, 2024

taiyang-li force-pushed the gluten_more_jits branch from 132bf39 to dcb761d Compare December 2, 2024 04:01

taiyang-li mentioned this pull request Dec 2, 2024

[GLUTEN-7534][CH] Refactor logics about jit settings #7536

Closed

taiyang-li changed the title ~~[GLUTEN-7534][CH][TEST] Run uts when JIT enabled~~ [GLUTEN-7534][CH][TEST] Run uts with JIT enabled Dec 2, 2024

taiyang-li force-pushed the gluten_more_jits branch from 7774004 to 5179b98 Compare December 4, 2024 02:45

taiyang-li force-pushed the gluten_more_jits branch from 68c7267 to 27fde6f Compare December 6, 2024 07:55

taiyang-li added 5 commits December 25, 2024 15:45

apply builtin i256

a33c049

support function transfrom_keys/transform_values

ba196a4

fix building error for transform_keys/values parser

3bd27bd

fix style

22ce267

fix failed uts

3769fc3

taiyang-li force-pushed the gluten_more_jits branch from df0c575 to 3769fc3 Compare December 25, 2024 07:51

github-actions bot removed the CORE works for Gluten Core label Dec 25, 2024

fix code style

ed48070

fix building

a9808fc

taiyang-li marked this pull request as draft December 26, 2024 09:25

taiyang-li added 2 commits December 26, 2024 17:46

revert JITS

6484bc1

fix building

5dd98e7

taiyang-li marked this pull request as ready for review December 26, 2024 10:13

taiyang-li changed the title ~~[GLUTEN-7534][CH] Complete JIT for sparkXX functions and optimize performance of sparkDecimalXX functions~~ [GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions Dec 26, 2024

taiyang-li added 5 commits December 27, 2024 14:15

poc to improve if when result type is floating point or small decimal

c074a5d

benchmark about inplace filtering

3562f5e

fix building

4dac60f

poc about clang bigint

e7f278d

fix building

4568ad6

baibaichen approved these changes Dec 27, 2024

View reviewed changes

taiyang-li merged commit 74cc9d7 into apache:main Dec 27, 2024
8 checks passed

baibaichen mentioned this pull request Jan 6, 2025

[CH]current code base compile failed for ARM #8408

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions #8105

[GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions #8105

taiyang-li commented Nov 29, 2024 •

edited

Loading

github-actions bot commented Nov 29, 2024

github-actions bot commented Nov 29, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

taiyang-li commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 4, 2024

github-actions bot commented Dec 4, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 25, 2024

github-actions bot commented Dec 25, 2024

github-actions bot commented Dec 26, 2024

taiyang-li commented Dec 26, 2024

github-actions bot commented Dec 26, 2024

baibaichen commented Dec 27, 2024

baibaichen commented Dec 27, 2024

github-actions bot commented Dec 27, 2024

baibaichen left a comment

[GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions #8105

[GLUTEN-7534][CH] Refactor and optimize sparkDecimalXXX functions #8105

Conversation

taiyang-li commented Nov 29, 2024 • edited Loading

What changes were proposed in this pull request?

How was this patch tested?

github-actions bot commented Nov 29, 2024

github-actions bot commented Nov 29, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 2, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

taiyang-li commented Dec 3, 2024

github-actions bot commented Dec 3, 2024

github-actions bot commented Dec 4, 2024

github-actions bot commented Dec 4, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 5, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 10, 2024

github-actions bot commented Dec 25, 2024

github-actions bot commented Dec 25, 2024

github-actions bot commented Dec 26, 2024

taiyang-li commented Dec 26, 2024

github-actions bot commented Dec 26, 2024

baibaichen commented Dec 27, 2024

baibaichen commented Dec 27, 2024

github-actions bot commented Dec 27, 2024

baibaichen left a comment

Choose a reason for hiding this comment

taiyang-li commented Nov 29, 2024 •

edited

Loading