Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: revert mostly changes in #11678 #12481

Merged
merged 4 commits into from
Oct 16, 2019
Merged

executor: revert mostly changes in #11678 #12481

merged 4 commits into from
Oct 16, 2019

Conversation

SunRunAway
Copy link
Contributor

@SunRunAway SunRunAway commented Sep 29, 2019 •
edited
Loading

What problem does this PR solve?

(a following PR for #12480)
After #12480, we do not need to copy row data in window functions.

What is changed and how it works?

Revert mostly changes in #11678, and get the performance loss back.

name                                                           old time/op    new time/op    delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12               92.9µs ±13%    86.0µs ± 7%     ~     (p=0.310 n=5+5)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              202µs ± 5%     230µs ± 8%  +13.65%  (p=0.008 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12             8.24ms ± 1%    8.37ms ± 3%     ~     (p=0.310 n=5+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12           8.53ms ± 1%    8.60ms ± 0%   +0.92%  (p=0.032 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12      8.58ms ± 1%    8.59ms ± 2%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12            11.8ms ± 2%     9.2ms ± 1%  -22.03%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12      11.8ms ± 1%     9.2ms ± 1%  -22.36%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12       12.2ms ± 1%     9.4ms ± 1%  -23.20%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12    11.9ms ± 1%     9.4ms ± 1%  -21.40%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12           8.70ms ± 0%    8.72ms ± 1%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12            13.3ms ± 1%    10.5ms ± 1%  -20.79%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12             13.0ms ± 2%    10.4ms ± 1%  -19.48%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12     8.97ms ± 1%    9.00ms ± 5%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12      8.90ms ± 1%    9.04ms ± 2%   +1.59%  (p=0.008 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12       8.96ms ± 1%    9.14ms ± 2%     ~     (p=0.056 n=5+5)

name                                                           old alloc/op   new alloc/op   delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12               82.5kB ± 0%    82.5kB ± 0%     ~     (all equal)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              105kB ± 1%     106kB ± 0%     ~     (p=0.151 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12             12.1MB ± 0%    12.1MB ± 0%     ~     (p=0.254 n=4+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12           7.96MB ± 0%    7.97MB ± 0%     ~     (p=0.230 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12      7.97MB ± 0%    7.97MB ± 0%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12            14.0MB ± 0%     8.0MB ± 0%  -43.08%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12      14.0MB ± 0%     8.0MB ± 0%  -43.07%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12       14.0MB ± 0%     8.0MB ± 0%  -43.07%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12    14.0MB ± 0%     8.0MB ± 0%  -43.10%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12           7.97MB ± 0%    7.97MB ± 0%     ~     (p=0.079 n=4+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12            9.57MB ± 0%    7.97MB ± 0%  -16.71%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12             9.60MB ± 0%    7.97MB ± 0%  -16.90%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12     7.96MB ± 0%    7.97MB ± 0%     ~     (p=0.262 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12      7.97MB ± 0%    7.96MB ± 0%     ~     (p=0.294 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12       7.97MB ± 0%    7.97MB ± 0%     ~     (p=0.452 n=5+5)

name                                                           old allocs/op  new allocs/op  delta
WindowRows/(func:row_number,_ndv:10,_rows:1000)-12                  162 ± 0%       162 ± 0%     ~     (all equal)
WindowRows/(func:row_number,_ndv:1000,_rows:1000)-12              1.68k ± 1%     1.68k ± 0%     ~     (p=1.000 n=5+5)
WindowRows/(func:row_number,_ndv:10,_rows:100000)-12              5.17k ± 0%     5.17k ± 0%     ~     (p=0.238 n=4+5)
WindowRows/(func:row_number,_ndv:1000,_rows:100000)-12            14.1k ± 0%     14.1k ± 0%     ~     (p=0.175 n=5+5)
WindowFunctions/(func:row_number,_ndv:1000,_rows:100000)-12       14.1k ± 0%     14.1k ± 0%     ~     (p=1.000 n=5+5)
WindowFunctions/(func:rank,_ndv:1000,_rows:100000)-12             33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:dense_rank,_ndv:1000,_rows:100000)-12       33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:cume_dist,_ndv:1000,_rows:100000)-12        33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:percent_rank,_ndv:1000,_rows:100000)-12     33.1k ± 0%     14.1k ± 0%  -57.38%  (p=0.008 n=5+5)
WindowFunctions/(func:ntile,_ndv:1000,_rows:100000)-12            14.1k ± 0%     14.1k ± 0%     ~     (p=0.079 n=4+5)
WindowFunctions/(func:lead,_ndv:1000,_rows:100000)-12              114k ± 0%       14k ± 0%  -87.63%  (p=0.008 n=5+5)
WindowFunctions/(func:lag,_ndv:1000,_rows:100000)-12               115k ± 0%       14k ± 0%  -87.74%  (p=0.008 n=5+5)
WindowFunctions/(func:first_value,_ndv:1000,_rows:100000)-12      14.1k ± 0%     14.1k ± 0%     ~     (p=0.262 n=5+5)
WindowFunctions/(func:last_value,_ndv:1000,_rows:100000)-12       14.1k ± 0%     14.1k ± 0%     ~     (p=0.294 n=5+5)
WindowFunctions/(func:nth_value,_ndv:1000,_rows:100000)-12        14.1k ± 0%     14.1k ± 0%     ~     (p=0.452 n=5+5)

Check List

Tests

  • None

Code changes

  • None

Side effects

  • None

Related changes

  • None

Release note

  • None

@SunRunAway SunRunAway mentioned this pull request Sep 29, 2019
Merged
@codecov
Copy link

codecov bot commented Sep 29, 2019 •
edited
Loading

Codecov Report

Merging #12481 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12481   +/-   ##
===========================================
  Coverage   80.0316%   80.0316%           
===========================================
  Files           462        462           
  Lines        105592     105592           
===========================================
  Hits          84507      84507           
  Misses        14884      14884           
  Partials       6201       6201

@SunRunAway
Copy link
Contributor Author

/run-all-tests

@SunRunAway
Copy link
Contributor Author

/run-unit-test

@SunRunAway SunRunAway changed the title [WIP] executor: revert mostly changes in #11678 executor: revert mostly changes in #11678 Oct 12, 2019
alivxxx
alivxxx reviewed Oct 14, 2019
View reviewed changes
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This was referenced Oct 14, 2019
Merged
Merged
lzmhhh123
lzmhhh123 reviewed Oct 14, 2019
View reviewed changes
Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, shall we cherry-pick this PR in 3.0 and 3.1?

executor/aggfuncs/func_cume_dist.go Outdated Show resolved Hide resolved
@SunRunAway
Copy link
Contributor Author

/run-all-tests

@SunRunAway SunRunAway requested a review from lzmhhh123 October 15, 2019 12:12
lzmhhh123
lzmhhh123 approved these changes Oct 16, 2019
View reviewed changes
Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lzmhhh123 lzmhhh123 added status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. labels Oct 16, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

Your auto merge job has been accepted, waiting for 12724

@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

@SunRunAway merge failed.

@SunRunAway
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

Your auto merge job has been accepted, waiting for 12700

@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

/run-all-tests

@sre-bot sre-bot merged commit 609b529 into pingcap:master Oct 16, 2019
@sre-bot sre-bot mentioned this pull request Oct 16, 2019
Merged
@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

cherry pick to release-3.0 in PR #12770

@sre-bot sre-bot mentioned this pull request Oct 16, 2019
Merged
@sre-bot
Copy link
Contributor

sre-bot commented Oct 16, 2019

cherry pick to release-3.1 in PR #12771

@SunRunAway SunRunAway deleted the window-func2 branch October 16, 2019 18:00
sre-bot added a commit that referenced this pull request Oct 17, 2019
@sre-bot
executor: revert mostly changes in #11678 (#12481) (#12771)
1983b54
sre-bot added a commit that referenced this pull request Oct 17, 2019
@sre-bot
executor: revert mostly changes in #11678 (#12481) (#12770)
150dd34
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
@SunRunAway @XiaTianliang
executor: revert mostly changes in pingcap#11678 (pingcap#12481)
bcc4858
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alivxxx alivxxx alivxxx left review comments

@lzmhhh123 lzmhhh123 lzmhhh123 approved these changes

Assignees
No one assigned
Labels
sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@SunRunAway @sre-bot @alivxxx @lzmhhh123