Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAST which only changes the length of the string type column will cause full scan instead of index seek #45199

Closed
pcqz opened this issue Jul 6, 2023 · 3 comments · Fixed by #46303
Assignees
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@pcqz
Copy link

pcqz commented Jul 6, 2023

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

create table t1(a varchar(100) primary key);
create table t2(a varchar(10) primary key);
create view v as select a from t1 union select a from t2;
explain select * from v where a='test';

2. What did you expect to see? (Required)

+----------------------+---------+------+--------------------------------------+-------------------------------------------------------+
| id                   | estRows | task | access object                        | operator info                                         |
+----------------------+---------+------+--------------------------------------+-------------------------------------------------------+
| HashAgg_13           | 16.00   | root |                                      | group by:Column#3, funcs:firstrow(Column#3)->Column#3 |
| └─Union_14           | 20.00   | root |                                      |                                                       |
|   ├─Point_Get_16     | 1.00    | root | table:t1, clustered index:PRIMARY(a) |                                                       |
|   └─Point_Get_18     | 1.00    | root | table:t2, clustered index:PRIMARY(a) |                                                       |
+----------------------+---------+------+--------------------------------------+-------------------------------------------------------+

3. What did you see instead (Required)

+--------------------------------+----------+-----------+--------------------------------------+-------------------------------------------------------------------------------------+
| id                             | estRows  | task      | access object                        | operator info                                                                       |
+--------------------------------+----------+-----------+--------------------------------------+-------------------------------------------------------------------------------------+
| HashAgg_13                     | 6408.00  | root      |                                      | group by:Column#3, funcs:firstrow(Column#3)->Column#3                               |
| └─Union_14                     | 8010.00  | root      |                                      |                                                                                     |
|   ├─Point_Get_16               | 1.00     | root      | table:t1, clustered index:PRIMARY(a) |                                                                                     |
|   └─Projection_17              | 8000.00  | root      |                                      | cast(test.t2.a, varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin)->Column#3   |
|     └─TableReader_20           | 8000.00  | root      |                                      | data:Selection_19                                                                   |
|       └─Selection_19           | 8000.00  | cop[tikv] |                                      | eq(cast(test.t2.a, varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin), "test") |
|         └─TableFullScan_18     | 10000.00 | cop[tikv] | table:t2                             | keep order:false, stats:pseudo                                                      |
+--------------------------------+----------+-----------+--------------------------------------+-------------------------------------------------------------------------------------+

4. What is your TiDB version? (Required)

master

@pcqz pcqz added the type/bug The issue is confirmed as a bug. label Jul 6, 2023
@ti-chi-bot ti-chi-bot added affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. labels Aug 16, 2023
@winoros
Copy link
Member

winoros commented Aug 17, 2023

The initial plan before predicate-push-down will be like Filter(a'='test')->Union{antoherChild, Projetion(cast(a, 100)->a')->Scan} and the projection's input column is a, output expression is cast(1, 100), as the new a'.
When we push down the filter a='test', we meet the projection and do column substitution. we see the a'='test' and then transform the a' to cast(a, 100) then push the converted filter down to the projection.

@winoros winoros changed the title Unexpected type conversion on filter confitions with different varchar length in view CAST which only changes the length of the string type column will cause full scan instead of index seek Aug 17, 2023
@winoros
Copy link
Member

winoros commented Aug 17, 2023

The cast(a, 100)->a' is needed.
Suppose that the length of the table column a is 4. The behavior of a = 'testtesttest' and cast(a as char(100)) = 'testtesttest' is different. The first one will truncate the const column while the second one enlarges the length of the column so no truncate will happen.

If we don't write UNION, just write where cast(a as char(10)) = 'testtesttest', we also cannot use index seek.

@winoros
Copy link
Member

winoros commented Aug 17, 2023

A possible solution is that we directly modify the ranger, detect the CASTs which only change the length, and construct an index range for them

ti-chi-bot bot pushed a commit that referenced this issue Aug 31, 2023
…n doesn't contain any precision loss in some cases (#46303)

close #45199
ti-chi-bot bot pushed a commit that referenced this issue Oct 12, 2023
…n doesn't contain any precision loss in some cases | tidb-test=pr/2224 (#46303) (#46548)

close #45199
ti-chi-bot bot pushed a commit that referenced this issue Oct 31, 2023
…n doesn't contain any precision loss in some cases (#46303) (#46546)

close #45199
ti-chi-bot bot pushed a commit that referenced this issue Oct 31, 2023
…n doesn't contain any precision loss in some cases (#46547)

close #45199
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. affects-7.1 This bug affects the 7.1.x(LTS) versions. severity/moderate sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
5 participants