Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improve](function) opt aes_encrypt/decrypt function to handle const column #37194

Merged
merged 1 commit into from
Jul 5, 2024
Merged

[improve](function) opt aes_encrypt/decrypt function to handle const column #37194

merged 1 commit into from
Jul 5, 2024

Conversation

zhangstar333
Copy link
Contributor

Proposed changes

select to_base64(aes_encrypt(column,'F3229A0B371ED2D9441B830D21A390C3', '0123456789'));
In most case of user, the second and third argument are const literal.
so handle the special case.

mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (5.20 sec)



mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (7.05 sec)

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39725 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1511edfbebc3428ed4f166393d28e872814d7276, data reload: false

------ Round 1 ----------------------------------
q1	17619	4295	4311	4295
q2	2010	199	185	185
q3	10447	1211	1167	1167
q4	10195	752	793	752
q5	7501	2579	2594	2579
q6	219	142	139	139
q7	959	598	602	598
q8	9227	2042	2043	2042
q9	8983	6445	6445	6445
q10	9005	3721	3724	3721
q11	463	239	237	237
q12	527	240	230	230
q13	17774	2974	2975	2974
q14	278	219	228	219
q15	537	475	479	475
q16	522	381	378	378
q17	952	725	703	703
q18	7801	7363	7486	7363
q19	2579	1454	1377	1377
q20	651	324	321	321
q21	4991	3186	3981	3186
q22	400	339	348	339
Total cold run time: 113640 ms
Total hot run time: 39725 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4367	4218	4210	4210
q2	378	273	265	265
q3	2979	2684	2725	2684
q4	1983	1699	1708	1699
q5	5497	5690	5497	5497
q6	225	130	137	130
q7	2205	1806	1797	1797
q8	3227	3422	3396	3396
q9	8703	8678	8871	8678
q10	4164	3927	3961	3927
q11	590	505	483	483
q12	797	641	643	641
q13	16386	3218	3285	3218
q14	325	281	285	281
q15	533	500	483	483
q16	510	447	439	439
q17	1818	1531	1506	1506
q18	8173	7917	7839	7839
q19	4409	1581	1508	1508
q20	2122	1846	1886	1846
q21	9614	4920	5133	4920
q22	643	557	599	557
Total cold run time: 79648 ms
Total hot run time: 56004 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 170200 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1511edfbebc3428ed4f166393d28e872814d7276, data reload: false

query1	2775	374	378	374
query2	10040	2546	2210	2210
query3	8319	215	227	215
query4	22058	17548	17204	17204
query5	3635	464	459	459
query6	257	160	162	160
query7	4580	295	299	295
query8	325	299	296	296
query9	8448	2365	2353	2353
query10	567	311	289	289
query11	10550	10092	9965	9965
query12	117	93	86	86
query13	1644	379	369	369
query14	9764	6511	7146	6511
query15	263	187	188	187
query16	8063	273	285	273
query17	1878	550	536	536
query18	2139	286	277	277
query19	327	155	161	155
query20	89	82	87	82
query21	219	152	137	137
query22	4203	4108	3899	3899
query23	34130	33519	33355	33355
query24	8185	2890	2832	2832
query25	594	405	402	402
query26	709	162	159	159
query27	2284	321	325	321
query28	6094	2133	2130	2130
query29	914	698	659	659
query30	235	154	167	154
query31	958	751	766	751
query32	96	55	54	54
query33	669	319	377	319
query34	868	467	494	467
query35	749	654	626	626
query36	1084	987	966	966
query37	142	76	79	76
query38	2917	2849	2843	2843
query39	881	836	856	836
query40	202	120	126	120
query41	52	54	49	49
query42	117	96	96	96
query43	582	528	532	528
query44	1079	748	737	737
query45	202	165	164	164
query46	1078	731	729	729
query47	1851	1757	1750	1750
query48	370	311	296	296
query49	829	407	408	407
query50	762	397	389	389
query51	6814	6749	6786	6749
query52	113	94	94	94
query53	360	287	298	287
query54	726	468	450	450
query55	75	71	73	71
query56	282	265	271	265
query57	1095	1033	1027	1027
query58	244	256	251	251
query59	3416	3265	3381	3265
query60	298	296	287	287
query61	92	90	89	89
query62	590	445	434	434
query63	316	291	291	291
query64	8467	2240	1763	1763
query65	3125	3088	3107	3088
query66	737	326	329	326
query67	15494	14907	14789	14789
query68	4727	544	541	541
query69	727	419	340	340
query70	1207	1131	1152	1131
query71	419	280	290	280
query72	8738	5212	2777	2777
query73	756	317	318	317
query74	5868	5548	5487	5487
query75	4048	2640	2719	2640
query76	3411	969	940	940
query77	694	306	301	301
query78	10500	9874	9723	9723
query79	3492	505	517	505
query80	1631	469	471	469
query81	560	216	225	216
query82	1435	111	107	107
query83	410	173	173	173
query84	273	93	89	89
query85	1315	282	272	272
query86	455	294	291	291
query87	3322	3043	3115	3043
query88	3942	2451	2437	2437
query89	496	392	383	383
query90	1861	188	191	188
query91	129	101	101	101
query92	59	48	49	48
query93	4735	528	514	514
query94	1249	188	188	188
query95	406	318	327	318
query96	601	269	268	268
query97	3195	3028	3037	3028
query98	216	202	188	188
query99	1151	853	824	824
Total cold run time: 283165 ms
Total hot run time: 170200 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1511edfbebc3428ed4f166393d28e872814d7276, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.66	0.08	0.07
query5	0.51	0.49	0.49
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.49
query10	0.55	0.54	0.55
query11	0.15	0.11	0.11
query12	0.15	0.13	0.13
query13	0.58	0.59	0.59
query14	0.77	0.81	0.79
query15	0.86	0.83	0.80
query16	0.37	0.37	0.35
query17	0.96	0.97	1.00
query18	0.24	0.24	0.25
query19	1.75	1.83	1.71
query20	0.01	0.02	0.02
query21	15.45	0.84	0.65
query22	5.00	6.66	1.68
query23	18.21	1.38	1.27
query24	2.19	0.23	0.22
query25	0.16	0.09	0.09
query26	0.27	0.18	0.17
query27	0.08	0.08	0.08
query28	13.22	1.04	1.00
query29	12.57	3.38	3.36
query30	0.25	0.07	0.05
query31	2.85	0.38	0.40
query32	3.27	0.49	0.47
query33	2.92	2.92	2.95
query34	17.25	4.41	4.42
query35	4.50	4.53	4.51
query36	0.65	0.46	0.50
query37	0.19	0.15	0.16
query38	0.15	0.15	0.16
query39	0.05	0.03	0.04
query40	0.18	0.14	0.13
query41	0.10	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 110.27 s
Total hot run time: 30.5 s

HappenLee
HappenLee approved these changes Jul 5, 2024
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 5, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 5, 2024

PR approved by anyone and no changes requested.

wangbo
wangbo approved these changes Jul 5, 2024
View reviewed changes
Copy link
Contributor

@wangbo wangbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wangbo wangbo merged commit 39b358c into apache:master Jul 5, 2024
26 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
@zhangstar333 @dataroaring
[improve](function) opt aes_encrypt/decrypt function to handle const …
3f362c9 
…column (#37194)

## Proposed changes
```
select to_base64(aes_encrypt(column,'F3229A0B371ED2D9441B830D21A390C3', '0123456789'));
In most case of user, the second and third argument are const literal.
so handle the special case.

mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (5.20 sec)



mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (7.05 sec)

```
@gavinchou gavinchou mentioned this pull request Aug 19, 2024
Open
cambyzju pushed a commit to cambyzju/incubator-doris that referenced this pull request Aug 27, 2024
@zhangstar333 @cambyzju
[improve](function) opt aes_encrypt/decrypt function to handle const …
07abe62 
…column (apache#37194)

## Proposed changes
```
select to_base64(aes_encrypt(column,'F3229A0B371ED2D9441B830D21A390C3', '0123456789'));
In most case of user, the second and third argument are const literal.
so handle the special case.

mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (5.20 sec)



mysql [test]>select count(aes_encrypt(a, 'seed1seed2seed3','0123456789')) from json_table_2;
+-----------------------------------------------------------+
| count(aes_encrypt(a, '***', '0123456789', 'AES_256_CBC')) |
+-----------------------------------------------------------+
|                                                  10000001 |
+-----------------------------------------------------------+
1 row in set (7.05 sec)

```
@cambyzju cambyzju mentioned this pull request Aug 27, 2024
Merged
yiguolei pushed a commit that referenced this pull request Aug 27, 2024
@cambyzju @zhangstar333
[improve](function) opt aes_encrypt/decrypt function to handle const …
b7e0bfa 
…column (#37194) (#39954)

pick #37194 to branch-2.1

---------

Co-authored-by: zhangstar333 <87313068+zhangstar333@users.noreply.github.com>
@yiguolei yiguolei mentioned this pull request Sep 5, 2024
Open
3 tasks
@cambyzju cambyzju mentioned this pull request Sep 12, 2024
Merged
lide-reed pushed a commit that referenced this pull request Sep 14, 2024
@cambyzju
[fix](encrypt) wrong mode arg of encrypt and decrypt function make BE…
6c995e9 
… crash (#40726)

Not supported mode arg in sm4_encrypt and sm4_decrypt make BE crash.

Intro by #37194
cambyzju added a commit to cambyzju/incubator-doris that referenced this pull request Sep 14, 2024
@cambyzju
[fix](encrypt) wrong mode arg of encrypt and decrypt function make BE…
2db6b11 
… crash (apache#40726)

Not supported mode arg in sm4_encrypt and sm4_decrypt make BE crash.

Intro by apache#37194
dataroaring pushed a commit that referenced this pull request Sep 26, 2024
@cambyzju @dataroaring
[fix](encrypt) wrong mode arg of encrypt and decrypt function make BE…
4e9d0fb 
… crash (#40726)

Not supported mode arg in sm4_encrypt and sm4_decrypt make BE crash.

Intro by #37194
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangbo wangbo wangbo approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.6-merged dev/3.0.1-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@zhangstar333 @doris-robot @HappenLee @wangbo @yiguolei @dataroaring