Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](inverted index) Error parsing NULL_LITERAL #37842

Merged
merged 1 commit into from
Jul 16, 2024

Conversation

zzzxl1993
Copy link
Contributor

@zzzxl1993 zzzxl1993 commented Jul 15, 2024

Proposed changes

  1. NULL_LITERAL and other LITERAL will enter the else process incorrectly
  2. SELECT * FROM table_name WHERE column_name NOT IN (1, 2, NULL); The NOT IN operator returns an empty result set when the collection contains NULL.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@@ -2811,7 +2812,8 @@ void SegmentIterator::_calculate_pred_in_remaining_conjunct_root(
} else if (_is_literal_node(node_type)) {
auto v_literal_expr = static_cast<const doris::vectorized::VLiteral*>(expr.get());
_column_predicate_info->query_values.insert(v_literal_expr->value());
} else {
} else if (node_type == TExprNodeType::BINARY_PRED || node_type == TExprNodeType::MATCH_PRED ||
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to refactor in order to unify all the code to the same behavior.

Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 15, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 408784f2f633078a1fd39cd0ba7f8c843596e39f, data reload: false

------ Round 1 ----------------------------------
q1	17621	5335	4266	4266
q2	2031	202	194	194
q3	10461	1153	1123	1123
q4	10200	729	765	729
q5	7622	2690	2611	2611
q6	221	133	134	133
q7	955	595	587	587
q8	9218	2053	2088	2053
q9	8660	6563	6618	6563
q10	8696	3811	3760	3760
q11	441	238	237	237
q12	408	231	220	220
q13	17782	2949	2954	2949
q14	274	225	240	225
q15	523	466	505	466
q16	476	386	382	382
q17	965	658	709	658
q18	8197	7585	7329	7329
q19	5920	1252	1378	1252
q20	692	327	326	326
q21	4887	3167	3297	3167
q22	348	278	278	278
Total cold run time: 116598 ms
Total hot run time: 39508 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4274	4321	4274
q2	363	269	261	261
q3	3015	2790	2773	2773
q4	1884	1604	1616	1604
q5	5299	5297	5330	5297
q6	219	132	129	129
q7	2064	1739	1729	1729
q8	3220	3349	3301	3301
q9	8447	8350	8431	8350
q10	3916	3652	3692	3652
q11	601	483	493	483
q12	790	618	574	574
q13	16962	2997	2973	2973
q14	301	266	271	266
q15	507	473	474	473
q16	473	415	418	415
q17	1782	1464	1451	1451
q18	7640	7696	7476	7476
q19	2425	1528	1637	1528
q20	2007	1841	1793	1793
q21	9849	4668	4608	4608
q22	546	492	512	492
Total cold run time: 76695 ms
Total hot run time: 53902 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172727 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 408784f2f633078a1fd39cd0ba7f8c843596e39f, data reload: false

query1	902	365	370	365
query2	6450	1819	1856	1819
query3	6658	207	226	207
query4	28221	17344	17268	17268
query5	4177	485	488	485
query6	287	182	184	182
query7	4586	290	293	290
query8	246	199	199	199
query9	8599	2407	2413	2407
query10	420	275	269	269
query11	10587	10049	10255	10049
query12	126	82	81	81
query13	1635	365	369	365
query14	10279	7534	7723	7534
query15	211	162	164	162
query16	7771	312	306	306
query17	1815	539	523	523
query18	1950	272	267	267
query19	196	147	159	147
query20	89	79	77	77
query21	202	124	126	124
query22	4305	4081	4108	4081
query23	33618	32989	33119	32989
query24	12055	2902	2864	2864
query25	664	372	369	369
query26	1783	151	148	148
query27	2922	266	271	266
query28	7662	2024	2010	2010
query29	1163	613	607	607
query30	285	146	147	146
query31	966	705	732	705
query32	97	51	55	51
query33	765	291	285	285
query34	979	485	490	485
query35	677	556	572	556
query36	1100	936	945	936
query37	281	76	78	76
query38	2910	2751	2734	2734
query39	870	812	809	809
query40	278	118	118	118
query41	47	44	44	44
query42	119	104	103	103
query43	505	462	467	462
query44	1205	716	728	716
query45	193	161	158	158
query46	1092	721	724	721
query47	1894	1779	1808	1779
query48	379	288	295	288
query49	1234	402	405	402
query50	777	400	399	399
query51	6945	6816	6684	6684
query52	96	92	89	89
query53	354	286	279	279
query54	985	452	445	445
query55	75	72	74	72
query56	285	281	274	274
query57	1173	1079	1088	1079
query58	252	251	248	248
query59	2859	2664	2904	2664
query60	299	279	275	275
query61	97	97	101	97
query62	857	656	667	656
query63	367	285	283	283
query64	10495	2189	1663	1663
query65	3178	3067	3100	3067
query66	1355	339	331	331
query67	15348	14938	15150	14938
query68	4643	547	542	542
query69	552	399	356	356
query70	1141	1120	1052	1052
query71	400	279	268	268
query72	7743	5571	5726	5571
query73	756	323	326	323
query74	6150	5671	5703	5671
query75	3416	2683	2696	2683
query76	2725	971	876	876
query77	445	319	310	310
query78	9701	9202	9143	9143
query79	2695	526	521	521
query80	2332	495	484	484
query81	597	223	223	223
query82	962	139	140	139
query83	320	175	178	175
query84	274	94	97	94
query85	1414	360	359	359
query86	476	318	322	318
query87	3286	3127	3111	3111
query88	4265	2379	2393	2379
query89	478	386	387	386
query90	1812	198	198	198
query91	148	116	113	113
query92	68	53	50	50
query93	2331	513	517	513
query94	1280	224	223	223
query95	414	321	445	321
query96	651	268	265	265
query97	3167	3027	3029	3027
query98	212	201	194	194
query99	1488	1274	1262	1262
Total cold run time: 287594 ms
Total hot run time: 172727 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 408784f2f633078a1fd39cd0ba7f8c843596e39f, data reload: false

query1	0.04	0.04	0.02
query2	0.08	0.04	0.03
query3	0.22	0.04	0.04
query4	1.67	0.08	0.08
query5	0.50	0.49	0.50
query6	1.14	0.72	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.49
query10	0.53	0.55	0.55
query11	0.15	0.11	0.11
query12	0.15	0.13	0.12
query13	0.60	0.59	0.58
query14	0.76	0.77	0.77
query15	0.86	0.80	0.83
query16	0.36	0.37	0.35
query17	0.94	0.99	1.04
query18	0.22	0.22	0.21
query19	1.78	1.66	1.74
query20	0.01	0.01	0.01
query21	15.42	0.76	0.67
query22	4.03	7.20	2.66
query23	18.24	1.45	1.23
query24	2.12	0.22	0.23
query25	0.15	0.08	0.08
query26	0.30	0.21	0.21
query27	0.45	0.22	0.22
query28	13.24	1.03	1.01
query29	12.66	3.27	3.25
query30	0.25	0.06	0.06
query31	2.87	0.39	0.39
query32	3.30	0.46	0.48
query33	2.87	2.90	2.89
query34	17.01	4.36	4.34
query35	4.46	4.41	4.38
query36	0.64	0.46	0.46
query37	0.19	0.16	0.16
query38	0.15	0.14	0.14
query39	0.04	0.04	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.36 s
Total hot run time: 31.19 s

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@airborne12 airborne12 merged commit 21e5374 into apache:master Jul 16, 2024
28 of 33 checks passed
@airborne12 airborne12 removed the doing label Jul 16, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
## Proposed changes

1. NULL_LITERAL and other LITERAL will enter the else process
incorrectly
2. SELECT * FROM table_name WHERE column_name NOT IN (1, 2, NULL); The
NOT IN operator returns an empty result set when the collection contains
NULL.
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Jul 21, 2024
1. NULL_LITERAL and other LITERAL will enter the else process
incorrectly
2. SELECT * FROM table_name WHERE column_name NOT IN (1, 2, NULL); The
NOT IN operator returns an empty result set when the collection contains
NULL.
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Jul 21, 2024
## Proposed changes

1. NULL_LITERAL and other LITERAL will enter the else process
incorrectly
2. SELECT * FROM table_name WHERE column_name NOT IN (1, 2, NULL); The
NOT IN operator returns an empty result set when the collection contains
NULL.
yiguolei pushed a commit that referenced this pull request Jul 25, 2024
zzzxl1993 added a commit to zzzxl1993/doris that referenced this pull request Aug 1, 2024
1. NULL_LITERAL and other LITERAL will enter the else process
incorrectly
2. SELECT * FROM table_name WHERE column_name NOT IN (1, 2, NULL); The
NOT IN operator returns an empty result set when the collection contains
NULL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants