Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: control Chunk size for StreamAgg&HashAgg #9512

Merged
merged 6 commits into from
Mar 6, 2019
Merged

executor: control Chunk size for StreamAgg&HashAgg #9512

merged 6 commits into from
Mar 6, 2019

Conversation

qw4990
Copy link
Contributor

@qw4990 qw4990 commented Feb 28, 2019

What problem does this PR solve?

Control the number of rows in chunks returned by TableReader & IndexReader & IndexLookup.
Following up #9452, this PR is a subtask of #9166.

What is changed and how it works?

  1. make StreamAgg and HashAgg support chunk size control.
  2. add some unit tests.

Check List

Tests

  • Unit test

@qw4990 qw4990 added the sig/execution SIG execution label Feb 28, 2019
@codecov-io
Copy link

codecov-io commented Feb 28, 2019 •
edited
Loading

Codecov Report

Merging #9512 into master will increase coverage by 0.01%.
The diff coverage is 81.25%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #9512      +/-   ##
==========================================
+ Coverage   67.38%   67.39%   +0.01%     
==========================================
  Files         375      375              
  Lines       78783    78780       -3     
==========================================
+ Hits        53087    53096       +9     
+ Misses      20927    20918       -9     
+ Partials     4769     4766       -3
Impacted Files Coverage Δ
executor/aggregate.go 79.11% <81.25%> (+3.22%) ⬆️
ddl/delete_range.go 78.3% <0%> (-1.06%) ⬇️
store/tikv/lock_resolver.go 41.7% <0%> (-0.95%) ⬇️
expression/schema.go 93.75% <0%> (-0.79%) ⬇️
executor/join.go 79.06% <0%> (-0.52%) ⬇️
util/systimemon/systime_mon.go 100% <0%> (+20%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4ccc4c1...f86e579. Read the comment docs.

winoros
winoros reviewed Mar 1, 2019
View reviewed changes
executor/aggregate.go Outdated Show resolved Hide resolved
@qw4990 qw4990 force-pushed the chunkSizeControl_StreamAgg/HashAgg branch from e96b277 to 866025f Compare March 4, 2019 05:25
zz-jason
zz-jason reviewed Mar 4, 2019
View reviewed changes
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz-jason zz-jason added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 4, 2019
@lysu lysu self-assigned this Mar 4, 2019
@qw4990 qw4990 requested review from lysu and removed request for lysu March 6, 2019 02:35
alivxxx
alivxxx previously approved these changes Mar 6, 2019
View reviewed changes
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alivxxx alivxxx added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 6, 2019
@alivxxx
Copy link
Contributor

alivxxx commented Mar 6, 2019

/run-all-tests

@qw4990 qw4990 mentioned this pull request Mar 6, 2019
Merged
@qw4990 qw4990 force-pushed the chunkSizeControl_StreamAgg/HashAgg branch from d45a319 to f86e579 Compare March 6, 2019 08:00
@qw4990
Copy link
Contributor Author

qw4990 commented Mar 6, 2019

/run-all-tests

@qw4990 qw4990 merged commit 2f69a28 into pingcap:master Mar 6, 2019
qw4990 added a commit to qw4990/tidb that referenced this pull request Apr 12, 2019
@qw4990
executor: control Chunk size for StreamAgg&HashAgg (pingcap#9512)
dee51e9
@qw4990 qw4990 mentioned this pull request Apr 12, 2019
Merged
zz-jason pushed a commit that referenced this pull request Apr 15, 2019
@qw4990 @zz-jason
executor: control Chunk size for StreamAgg&HashAgg (#9512) (#10133)
356b0e5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@winoros winoros winoros left review comments

@zz-jason zz-jason zz-jason left review comments

@alivxxx alivxxx alivxxx approved these changes

@XuHuaiyu XuHuaiyu Awaiting requested review from XuHuaiyu

@lysu lysu Awaiting requested review from lysu

@eurekaka eurekaka Awaiting requested review from eurekaka

Assignees

@lysu lysu

Labels
sig/execution SIG execution status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@qw4990 @codecov-io @alivxxx @zz-jason @winoros @lysu