-
Notifications
You must be signed in to change notification settings - Fork 753
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: change max threads to max io requests for fuse read data #8270
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
8ba4bba
to
cc274fe
Compare
logic test wait for: #8283 |
@mergify update |
✅ Branch has been successfully updated |
sqllogic test failure due to the docker max open file not being set as purpose. |
Because of some pipeline resize issue, I convert this PR to draft. |
donot know why this stateless test failed. suite_name: base/02_function/02_0005_function_compare, https://github.com/datafuselabs/databend/actions/runs/3274136199/jobs/5387717926#step:4:1879 |
Some sqllogic base tests still failure: |
@mergify update |
✅ Branch has been successfully updated |
Move to #8321 |
I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/
Summary
For object storage, the more read requests, the faster until the network bandwidth is reached. We have applied this mechanism to read snapshot&segment files(see #8153).
This PR tries to fast-read block files with more
FuseEngineSource
source pipes to the pipeline:change the pipe numbers from
max_threads
tomax_storage_io_requests
Performance test:
Table
select sum(c8) from t7861
Fixes #8263