Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](nereids)keep equal predicate as join conjunct even if it can be fold to null literal #35811

Merged
merged 1 commit into from
Jun 4, 2024
Merged

[fix](nereids)keep equal predicate as join conjunct even if it can be fold to null literal #35811

merged 1 commit into from
Jun 4, 2024

Conversation

starocean999
Copy link
Contributor

consider sql select null in ( select k1 from t);
the join conjunct null = k1 would be fold to null. But be doesn't support null literal as join conjunct. This pr keeps null = k1 to make be happy

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39869 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c068a3fce55f8bd53e58650fd44b0aa52e4f3082, data reload: false

------ Round 1 ----------------------------------
q1	18298	4509	4420	4420
q2	2669	203	187	187
q3	12396	1132	1163	1132
q4	10210	749	762	749
q5	7534	2707	2672	2672
q6	219	134	134	134
q7	938	606	586	586
q8	9227	2050	2064	2050
q9	8820	6472	6468	6468
q10	8882	3740	3702	3702
q11	467	246	239	239
q12	401	232	230	230
q13	17759	2985	2982	2982
q14	259	223	223	223
q15	520	484	481	481
q16	483	375	377	375
q17	952	629	732	629
q18	8024	7463	7482	7463
q19	6174	1411	1496	1411
q20	682	304	306	304
q21	5060	3167	3102	3102
q22	402	330	337	330
Total cold run time: 120376 ms
Total hot run time: 39869 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4279	4223	4223
q2	370	264	262	262
q3	2960	2725	2683	2683
q4	1888	1580	1600	1580
q5	5255	5276	5272	5272
q6	223	131	127	127
q7	2124	1687	1669	1669
q8	3177	3301	3278	3278
q9	8260	8319	8355	8319
q10	3848	3721	3693	3693
q11	606	492	501	492
q12	747	572	583	572
q13	16319	2990	3061	2990
q14	289	266	251	251
q15	519	486	473	473
q16	484	413	420	413
q17	1773	1472	1480	1472
q18	7640	7377	7405	7377
q19	2697	1667	1650	1650
q20	2019	1816	1773	1773
q21	4725	4655	4679	4655
q22	598	536	548	536
Total cold run time: 70913 ms
Total hot run time: 53760 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172606 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c068a3fce55f8bd53e58650fd44b0aa52e4f3082, data reload: false

query1	922	399	377	377
query2	6443	2528	2431	2431
query3	6648	207	213	207
query4	18823	17436	17400	17400
query5	4162	447	450	447
query6	257	159	160	159
query7	4594	294	291	291
query8	323	286	282	282
query9	8459	2408	2386	2386
query10	460	294	271	271
query11	10634	10265	10159	10159
query12	136	91	86	86
query13	1643	359	365	359
query14	10148	6380	7557	6380
query15	227	194	184	184
query16	7835	268	260	260
query17	1767	527	537	527
query18	1953	266	266	266
query19	218	155	153	153
query20	94	86	86	86
query21	204	138	125	125
query22	4442	4216	4226	4216
query23	33576	32963	33187	32963
query24	11914	2842	2923	2842
query25	656	348	356	348
query26	1829	155	158	155
query27	3003	349	324	324
query28	7628	2056	2052	2052
query29	1118	607	604	604
query30	272	147	153	147
query31	927	739	726	726
query32	99	57	59	57
query33	749	289	278	278
query34	961	488	482	482
query35	745	606	621	606
query36	1072	943	929	929
query37	156	67	69	67
query38	2864	2796	2714	2714
query39	841	800	789	789
query40	283	123	126	123
query41	53	52	53	52
query42	123	97	97	97
query43	580	531	536	531
query44	1254	722	743	722
query45	202	176	169	169
query46	1089	719	744	719
query47	1863	1757	1811	1757
query48	371	320	297	297
query49	1146	421	403	403
query50	794	379	395	379
query51	7011	6883	6756	6756
query52	105	91	88	88
query53	391	298	294	294
query54	960	448	439	439
query55	73	71	70	70
query56	277	255	255	255
query57	1145	1101	1020	1020
query58	267	246	249	246
query59	3530	3329	3214	3214
query60	297	268	268	268
query61	92	92	90	90
query62	659	443	451	443
query63	357	292	295	292
query64	9802	2196	1693	1693
query65	3208	3085	3122	3085
query66	1378	347	332	332
query67	15576	14973	15098	14973
query68	4628	547	551	547
query69	436	302	307	302
query70	1072	1115	1128	1115
query71	402	284	284	284
query72	7119	5575	5309	5309
query73	795	325	325	325
query74	6000	5604	5585	5585
query75	3493	2672	2692	2672
query76	2770	930	967	930
query77	443	295	293	293
query78	10412	9905	9755	9755
query79	2731	502	518	502
query80	1105	475	455	455
query81	515	221	223	221
query82	946	106	99	99
query83	230	164	168	164
query84	239	81	90	81
query85	1461	270	266	266
query86	447	317	308	308
query87	3328	3114	3067	3067
query88	4319	2362	2345	2345
query89	481	402	381	381
query90	1786	195	193	193
query91	124	99	96	96
query92	63	50	48	48
query93	1895	504	497	497
query94	1210	200	196	196
query95	418	315	319	315
query96	584	266	269	266
query97	3212	2998	3054	2998
query98	309	216	211	211
query99	1241	848	838	838
Total cold run time: 275902 ms
Total hot run time: 172606 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c068a3fce55f8bd53e58650fd44b0aa52e4f3082, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.69	0.07	0.07
query5	0.48	0.49	0.48
query6	1.12	0.72	0.72
query7	0.01	0.01	0.01
query8	0.05	0.05	0.04
query9	0.53	0.50	0.49
query10	0.55	0.55	0.53
query11	0.16	0.11	0.11
query12	0.14	0.12	0.11
query13	0.59	0.59	0.59
query14	0.79	0.78	0.77
query15	0.83	0.81	0.82
query16	0.37	0.37	0.36
query17	0.98	0.99	0.96
query18	0.22	0.27	0.22
query19	1.88	1.69	1.76
query20	0.01	0.01	0.01
query21	15.45	0.71	0.68
query22	3.62	7.22	2.18
query23	18.36	1.32	1.18
query24	1.70	0.27	0.20
query25	0.14	0.08	0.07
query26	0.27	0.17	0.18
query27	0.08	0.08	0.08
query28	13.36	1.01	1.00
query29	12.65	3.30	3.31
query30	0.24	0.05	0.05
query31	2.87	0.38	0.39
query32	3.29	0.48	0.48
query33	2.92	2.83	2.90
query34	17.18	4.46	4.44
query35	4.43	4.48	4.48
query36	0.64	0.46	0.47
query37	0.18	0.15	0.15
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.14
query41	0.09	0.04	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 108.7 s
Total hot run time: 30.66 s

@starocean999 starocean999 marked this pull request as ready for review June 4, 2024 01:50
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 4, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 4, 2024

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit 25f0182 into apache:master Jun 4, 2024
28 of 30 checks passed
dataroaring pushed a commit that referenced this pull request Jun 4, 2024
@starocean999 @dataroaring
[opt](nereids) keep equal predicate as join conjunct even if it can b…
f9461a1 
…e fold to null literal (#35811)

consider sql: select null in ( select k1 from t);
the join conjunct null = k1 would be fold to null literal. But be doesn't
support null literal as join conjunct. This pr keeps null = k1 to make
be happy
@starocean999 starocean999 mentioned this pull request Jun 4, 2024
Merged
yiguolei pushed a commit that referenced this pull request Jun 4, 2024
@starocean999
[fix](nereids)keep equal predicate as join conjunct even if it can be…
c23ab25 
… fold to null literal (#35842)

pick from master #35811

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
seawinde pushed a commit to seawinde/doris that referenced this pull request Jun 5, 2024
@starocean999 @seawinde
[opt](nereids) keep equal predicate as join conjunct even if it can b…
db9add4 
…e fold to null literal (apache#35811)

consider sql: select null in ( select k1 from t);
the join conjunct null = k1 would be fold to null literal. But be doesn't
support null literal as join conjunct. This pr keeps null = k1 to make
be happy
@morningman morningman mentioned this pull request Jun 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@924060929 924060929 924060929 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.4-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@starocean999 @doris-robot @924060929 @morrySnow @morningman @dataroaring