Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Roadmap] Implement IO manager #7581

Closed
morningman opened this issue Jan 2, 2022 · 1 comment · Fixed by #7994
Closed

[Roadmap] Implement IO manager #7581

morningman opened this issue Jan 2, 2022 · 1 comment · Fixed by #7994

Comments

@morningman
Copy link
Contributor

root issue

@yinzhijian
Copy link
Contributor

性能测试

思路

构造一个场景使得当某个盘热点时,其它盘的查询不受影响(表现为比原始版本的性能更好)。

环境

单BE,6块disk。

额外配置:

disable_storage_page_cache = true #禁用pagecache
doris_scanner_thread_pool_thread_num=24

单FE。

方法

数据以及环境准备

  1. 总共有两个ssb数据库,ssb1跟ssb2,两者的表结构、表数据完全一致。不同之处在于,ssb1的所有分片落在disk1上,ssb2的分片落在disk2~disk6上。【disk1热点不影响其它disk的查询处理】
  2. 对disk1模拟io热点。
    1. 修改代码,在env_posix.cpp的do_readv_at函数中,添加读取disk1文件时sleep 1ms的逻辑(模拟io wait)。

性能测试

  1. 同时运行两个jmeter,分别对ssb1和ssb2发起单次40并发的压测。
    1. 环境设置:set parallel_fragment_exec_instance_num=8
    2. sql为select count(*) from lineorder;【其它sql结果类似】
  2. 测试结果:

ssb2的整体耗时如下(其它盘性能有明显提升):

\总时间\ 向量化版本 非向量化版本
disk绑定版本 51s 2min40s
原始版本 1min08s 2min54s

ssb1的整体耗时如下(热点盘无影响):

\总时间\ 向量版本 非向量版本
disk绑定版本 1min34s 3min15s
原始版本 1min35s 3min14s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants