Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support elt pushdown #5496

Merged
merged 20 commits into from
Aug 8, 2022
Merged

feat: Support elt pushdown #5496

merged 20 commits into from
Aug 8, 2022

Conversation

Willendless
Copy link
Contributor

@Willendless Willendless commented Jul 28, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: close #5104

related pr: pingcap/tidb#36735

Problem Summary:

What is changed and how it works?

  • implement FunctionElt class to handle elt function push down

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Support to pushdown elt to TiFlash

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 28, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Lloyd-Pottiger
  • hongyunyan

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 28, 2022
@Willendless Willendless mentioned this pull request Jul 28, 2022
Open
@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jul 28, 2022
@Willendless
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Jul 28, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               141    59.37%          45                11    75.56%         638               271    57.52%         412               190    53.88%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 4    75.00%
Functions/FunctionsString.cpp                1480               374    74.73%         279                71    74.55%        3208               761    76.28%         914               298    67.40%
Functions/tests/gtest_strings_elt.cpp         521               126    75.82%           3                 0   100.00%         123                 0   100.00%         152                78    48.68%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2385               653    72.62%         344                90    73.84%        4028              1054    73.83%        1494               570    61.85%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18843      9437             49.92%    214938  95439        55.60%

full coverage report (for internal network access only)

@Willendless
Copy link
Contributor Author

/run-all-test

@Willendless
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Jul 28, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               141    59.37%          45                11    75.56%         638               271    57.52%         412               190    53.88%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 4    75.00%
Functions/FunctionsString.cpp                1480               374    74.73%         279                71    74.55%        3208               761    76.28%         914               298    67.40%
Functions/tests/gtest_strings_elt.cpp         521               126    75.82%           3                 0   100.00%         123                 0   100.00%         152                78    48.68%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2385               653    72.62%         344                90    73.84%        4028              1054    73.83%        1494               570    61.85%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18843      9440             49.90%    214938  95415        55.61%

full coverage report (for internal network access only)

@Willendless Willendless changed the title [WIP] Support elt pushdown feat: Support elt pushdown Jul 29, 2022
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2022
@Willendless Willendless changed the title feat: Support elt pushdown [WIP] feat: Support elt pushdown Jul 29, 2022
@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2022
@Willendless Willendless changed the title [WIP] feat: Support elt pushdown [DNM] feat: Support elt pushdown Jul 29, 2022
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2022
@Willendless
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Jul 29, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               141    59.37%          45                11    75.56%         638               271    57.52%         412               190    53.88%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 5    68.75%
Functions/FunctionsString.cpp                1480               374    74.73%         279                71    74.55%        3208               761    76.28%         914               298    67.40%
Functions/tests/gtest_strings_elt.cpp         521               126    75.82%           3                 0   100.00%         123                 0   100.00%         152                78    48.68%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2385               653    72.62%         344                90    73.84%        4028              1054    73.83%        1494               571    61.78%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18843      9437             49.92%    214938  95380        55.62%

full coverage report (for internal network access only)

@Willendless Willendless changed the title [DNM] feat: Support elt pushdown feat: Support elt pushdown Jul 29, 2022
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 4, 2022
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 4, 2022
@Willendless Willendless changed the title feat: Support elt pushdown [DNM] feat: Support elt pushdown Aug 4, 2022
@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 5, 2022
@Willendless
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Aug 6, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 5    68.75%
Functions/FunctionsString.cpp                1491               373    74.98%         281                71    74.73%        3222               758    76.47%         920               297    67.72%
Functions/tests/gtest_strings_elt.cpp         133                21    84.21%           3                 0   100.00%         176                 0   100.00%          14                12    14.29%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2008               541    73.06%         346                86    75.14%        4095              1037    74.68%        1362               502    63.14%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19037      9232             51.50%    215816  94045        56.42%

full coverage report (for internal network access only)

@Willendless Willendless changed the title [DNM] feat: Support elt pushdown feat: Support elt pushdown Aug 6, 2022
@Willendless
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Aug 6, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 4    75.00%
Functions/FunctionsString.cpp                1491               373    74.98%         281                71    74.73%        3222               758    76.47%         920               297    67.72%
Functions/tests/gtest_strings_elt.cpp         133                21    84.21%           3                 0   100.00%         176                 0   100.00%          14                12    14.29%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2008               541    73.06%         346                86    75.14%        4095              1037    74.68%        1362               501    63.22%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19037      9232             51.50%    215816  94034        56.43%

full coverage report (for internal network access only)

SeaRise
SeaRise approved these changes Aug 8, 2022
View reviewed changes
Copy link
Contributor

@SeaRise SeaRise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SeaRise
Copy link
Contributor

SeaRise commented Aug 8, 2022

/merge

@ti-chi-bot
Copy link
Member

@SeaRise: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 81ebb92

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 8, 2022
@ti-chi-bot
Copy link
Member

@Willendless: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@sre-bot
Copy link
Collaborator

sre-bot commented Aug 8, 2022

Coverage for changed files

Filename                                  Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp                347               135    61.10%          45                 7    84.44%         638               257    59.72%         412               188    54.37%
Functions/FunctionHelpers.h                    37                12    67.57%          17                 8    52.94%          59                22    62.71%          16                 4    75.00%
Functions/FunctionsString.cpp                1491               373    74.98%         281                71    74.73%        3222               758    76.47%         920               297    67.72%
Functions/tests/gtest_strings_elt.cpp         133                21    84.21%           3                 0   100.00%         176                 0   100.00%          14                12    14.29%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                        2008               541    73.06%         346                86    75.14%        4095              1037    74.68%        1362               501    63.22%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
19037      9230             51.52%    215816  93959        56.46%

full coverage report (for internal network access only)

@ti-chi-bot ti-chi-bot merged commit bcf72e3 into pingcap:master Aug 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xzhangxian1008 xzhangxian1008 xzhangxian1008 left review comments

@Lloyd-Pottiger Lloyd-Pottiger Lloyd-Pottiger approved these changes

@hongyunyan hongyunyan hongyunyan approved these changes

@SeaRise SeaRise SeaRise approved these changes

Assignees
No one assigned
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement Elt function push down
7 participants
@Willendless @ti-chi-bot @sre-bot @xzhangxian1008 @SeaRise @hongyunyan @Lloyd-Pottiger