Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

新增对doris的支持 #2536

Merged
merged 7 commits into from
Mar 17, 2024
Merged

新增对doris的支持 #2536

merged 7 commits into from
Mar 17, 2024

Conversation

Grain-Yu
Copy link
Contributor

支持doris的查询、上线审核
相关讨论:#2175

支持doris的查询、上线审核
相关讨论:#2175
Copy link

codecov bot commented Mar 15, 2024

Codecov Report

Attention: Patch coverage is 59.09091% with 54 lines in your changes are missing coverage. Please review.

Project coverage is 76.82%. Comparing base (033964c) to head (62db9a2).

Files Patch % Lines
sql/engines/doris.py 43.75% 54 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2536      +/-   ##
==========================================
- Coverage   76.97%   76.82%   -0.15%     
==========================================
  Files         115      117       +2     
  Lines       15943    16073     +130     
==========================================
+ Hits        12272    12348      +76     
- Misses       3671     3725      +54     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sql/engines/doris.py Outdated Show resolved Hide resolved
改为从MysqlEngine类继承
@Grain-Yu Grain-Yu requested a review from LeoQuote March 16, 2024 10:26
sql/engines/doris.py Dismissed Show dismissed Hide dismissed
sql/engines/doris.py Dismissed Show dismissed Hide dismissed
Copy link
Collaborator

@LeoQuote LeoQuote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

麻烦对照再检查下, 能不覆盖的方法就不要覆盖, 并不是说你直接转换了基类就行了

sql/engines/doris.py Outdated Show resolved Hide resolved
sql/engines/doris.py Outdated Show resolved Hide resolved
sql/engines/doris.py Outdated Show resolved Hide resolved
sql/engines/doris.py Outdated Show resolved Hide resolved
@Grain-Yu Grain-Yu requested a review from LeoQuote March 16, 2024 14:57
Copy link
Collaborator

@LeoQuote LeoQuote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不好意思因为doris 和 mysql 的协议是完全兼容的,能否过一遍你所有的方法,能复用的就删除,无法复用的请在方法注释,或代码评论中说明需要覆盖的原因,这样能最大程度的简化代码,减轻审核的难度。

另外CI的这个检查,lint 和覆盖率是必须要过的,可以参考下 black 的使用方法,单元测试这个如果你能力有限,我有空帮你补一下也是可以的

@Grain-Yu
Copy link
Contributor Author

Grain-Yu commented Mar 16, 2024

不好意思因为doris 和 mysql 的协议是完全兼容的,能否过一遍你所有的方法,能复用的就删除,无法复用的请在方法注释,或代码评论中说明需要覆盖的原因,这样能最大程度的简化代码,减轻审核的难度。

另外CI的这个检查,lint 和覆盖率是必须要过的,可以参考下 black 的使用方法,单元测试这个如果你能力有限,我有空帮你补一下也是可以的

class DorisEngine中的改动说明

auto_backup:
回滚功能暂未做全面测试,因此改成false

server_version:
MysqlEngine通过conn.get_server_info()获取,返回的值是"5.7.20-16log"这种形式,
对于doris不适用,通过执行show frontends获取版本信息

query:
MysqlEngine中的autocommit设置对于doris无实际作用。
MysqlEngine通过set session max_execution_time={max_execution_time}设置最大执行时间,对于doris不适用
mysql的column_type列类型来自column_types_map中预定义的数据类型,对于doris不适用

get_all_databases:
MysqlEngine中屏蔽的库为("information_schema", "performance_schema", "mysql", "test", "sys"),
对于doris是("__internal_schema", "INFORMATION_SCHEMA", "information_schema")

query_check:
MysqlEngine中的"不应该查看mysql.user表"规则,对doris不适用

execute_check:
MysqlEngine中的“判断Inception检测结果”,对doris不适用。这一部分参考了PgSQLRngine的execute_check

execute_workflow:
MysqlEngine中的判断实例是否只读对doris不适用,改为原生执行

execute:
参考CassandraEngine的实现,将执行和获取结果的逻辑放到一起。

@Grain-Yu
Copy link
Contributor Author

单元测试实在是不会,请帮忙添加

@Grain-Yu Grain-Yu requested a review from LeoQuote March 16, 2024 23:40
@LeoQuote
Copy link
Collaborator

https://github.com/Grain-Yu/Archery/pull/1 把这个合了看一下, 下次提 pr 记得新开一个分支, 主分支有保护我这边不能直接推

@LeoQuote
Copy link
Collaborator

@hhyo 我帮忙写了点测试覆盖率还是低了点, 不过没关系这是 engine 级别的, 不会危害到主程序的监控, 等有空我再归并一些重复的优化一下

@LeoQuote
Copy link
Collaborator

@Grain-Yu 另外我这边没 doris 的环境, 还请帮忙测试下能否正常运行

@hhyo hhyo merged commit 2d272f2 into hhyo:master Mar 17, 2024
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants