Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Performance Enhancement of Table Scanning #6834

Open
JinheLin opened this issue Feb 16, 2023 · 1 comment
Open

Performance Enhancement of Table Scanning #6834

JinheLin opened this issue Feb 16, 2023 · 1 comment
Assignees
@JinheLin
Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.

Comments

@JinheLin
Copy link
Contributor

JinheLin commented Feb 16, 2023 •
edited
Loading

No description provided.

@JinheLin JinheLin added type/enhancement The issue or PR belongs to an enhancement. component/storage labels Feb 16, 2023
@JinheLin JinheLin self-assigned this Feb 16, 2023
@flowbehappy
Copy link
Contributor

flowbehappy commented Feb 16, 2023 •
edited
Loading

Background of this task: Currently, small queries (e.g. scan 2 million rows and do aggregation) can not fully use CPU resources on 8 logical core machines (e.g. r5.2xlarge).

Also, consider making the default thread pool size to be 2x of CPU number. And check whether it is more efficient for small queries.

@JinheLin JinheLin mentioned this issue Feb 23, 2023
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Feb 24, 2023
@JinheLin
Refine storage read thread count. (#6885)
5872f16 
ref #6834
@JinheLin JinheLin mentioned this issue Mar 14, 2023
Closed
5 tasks
@JinheLin JinheLin mentioned this issue Oct 19, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 23, 2023
@JinheLin
Refine SegmentReadTask for disaggregated-mode and op-mode. (#8216)
da347e2 
ref #6834
@JinheLin JinheLin mentioned this issue Oct 25, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 28, 2023
@JinheLin
Storages: Refine SegmentReadTaskPool and SegmentReadTaskScheduler (#8237
3cc5dca 
)

ref #6834
This was referenced Nov 1, 2023
Closed
Closed
Closed
Merged
ti-chi-bot bot pushed a commit that referenced this issue Nov 4, 2023
@JinheLin
Storages: disaggregated mode supports storage read thread. (#8272)
cfe64ce 
ref #6834
@JinheLin JinheLin mentioned this issue Nov 7, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 8, 2023
@JinheLin
Support Clang Thread Safety Analysis (#8330)
4ee89b1 
ref #6350, ref #6834
@JinheLin JinheLin mentioned this issue Nov 21, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Nov 21, 2023
@JinheLin
Rename some names of DB::DM::Remote::Serializer. (#8401)
3d3a5f1 
ref #6834
@JinheLin JinheLin mentioned this issue Dec 13, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Dec 14, 2023
@JinheLin
Storages: small refine of MergedTask. (#8512)
cfbf378 
ref #6834
yibin87 pushed a commit to yibin87/tiflash that referenced this issue Dec 15, 2023
@JinheLin @yibin87
Storages: small refine of MergedTask. (pingcap#8512)
929b061 
ref pingcap#6834
@JinheLin JinheLin mentioned this issue Dec 20, 2023
Merged
12 tasks
@ti-chi-bot ti-chi-bot mentioned this issue Dec 22, 2023
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jan 3, 2024
@JinheLin
Storages: refine SegmentReadTaskScheduler (#8557)
64c1ae5 
ref #6834
@JinheLin JinheLin mentioned this issue Jan 18, 2024
Merged
12 tasks
This was referenced Jul 24, 2024
Closed
Merged
ti-chi-bot bot added a commit that referenced this issue Jul 29, 2024
@JinheLin @ti-chi-bot @JaySon-Huang
Storages: Add Block pop-up latency metrics (#9260)
954f147 
ref #6834

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Co-authored-by: JaySon <tshent@qq.com>
@JinheLin JinheLin mentioned this issue Sep 5, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Sep 6, 2024
@JinheLin @Lloyd-Pottiger
Storages: add metrics of read thread (#9412)
0a9c4bd 
ref #6834

Co-authored-by: Lloyd-Pottiger <60744015+Lloyd-Pottiger@users.noreply.github.com>
This was referenced Sep 9, 2024
Merged
Merged
Merged
Merged
@JinheLin JinheLin changed the title Read Thread Pool Enhancement Performance Enhancement of Table Scanning Oct 25, 2024
@JinheLin JinheLin mentioned this issue Oct 25, 2024
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Oct 28, 2024
@JinheLin @Lloyd-Pottiger
Storages: Use std::vector<UInt8> in BitmapFilter (#9552)
b36199e 
ref #6834

Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com>

Co-authored-by: Lloyd-Pottiger <yan1579196623@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JinheLin JinheLin

Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@flowbehappy @JinheLin