Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add mysql registry plugin #10406

Merged
merged 12 commits into from
Jun 13, 2022
Merged

Add mysql registry plugin #10406

merged 12 commits into from
Jun 13, 2022

Conversation

ruanwenjun
Copy link
Member

@ruanwenjun ruanwenjun commented Jun 10, 2022 •
edited
Loading

Purpose of the pull request

close #10408
Add new registry plugin - mysql. If user don't have a zookeeper cluster, they can still use DolphinScheduler.

todo: add doc

Brief change log

  • Add a new module dolphinscheduler-registry-mysql.
  • Add dependency dolphinscheduler-registry-mysql to api/master/worker

Verify this pull request

Change the registry.type to mysql and start a ds cluster.

@SbloodyS SbloodyS added feature new feature backend labels Jun 11, 2022
@SbloodyS SbloodyS added this to the 3.1.0-alpha milestone Jun 11, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jun 11, 2022 •
edited
Loading

Codecov Report

Merging #10406 (ef35887) into dev (b0d9d3f) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                dev   #10406      +/-   ##
============================================
- Coverage     40.56%   40.53%   -0.04%     
- Complexity     4767     4772       +5     
============================================
  Files           877      878       +1     
  Lines         35623    35741     +118     
  Branches       3945     3969      +24     
============================================
+ Hits          14452    14489      +37     
- Misses        19738    19807      +69     
- Partials       1433     1445      +12
Impacted Files Coverage Δ
...r/plugin/registry/zookeeper/ZookeeperRegistry.java 48.21% <100.00%> (ø)
...egistry/zookeeper/ZookeeperRegistryProperties.java 53.48% <100.00%> (ø)
...pache/dolphinscheduler/service/corn/CronUtils.java 64.77% <0.00%> (-5.60%) ⬇️
...e/dolphinscheduler/remote/NettyRemotingClient.java 50.00% <0.00%> (-2.78%) ⬇️
...cheduler/api/service/impl/ExecutorServiceImpl.java 40.10% <0.00%> (-2.59%) ⬇️
.../server/master/runner/WorkflowExecuteRunnable.java 7.75% <0.00%> (-0.07%) ⬇️
.../org/apache/dolphinscheduler/api/enums/Status.java 100.00% <0.00%> (ø)
.../org/apache/dolphinscheduler/common/Constants.java 80.95% <0.00%> (ø)
...r/plugin/task/sqoop/parameter/SqoopParameters.java 53.33% <0.00%> (ø)
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b0d9d3f...ef35887. Read the comment docs.

kezhenxu94
kezhenxu94 requested changes Jun 11, 2022
View reviewed changes
...heduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-mysql/README.md Outdated Show resolved Hide resolved
...scheduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-mysql/pom.xml Outdated Show resolved Hide resolved
dolphinscheduler-api/pom.xml Outdated Show resolved Hide resolved
...heduler-registry/dolphinscheduler-registry-plugins/dolphinscheduler-registry-mysql/README.md Show resolved Hide resolved
...try-mysql/src/main/java/org/apache/dolphinscheduler/plugin/registry/mysql/MysqlOperator.java Outdated Show resolved Hide resolved
pom.xml Outdated Show resolved Hide resolved
.../java/org/apache/dolphinscheduler/plugin/registry/zookeeper/ZookeeperRegistryProperties.java Show resolved Hide resolved
...rc/main/java/org/apache/dolphinscheduler/plugin/registry/mysql/task/RegistryLockManager.java Outdated Show resolved Hide resolved
...-registry-plugins/dolphinscheduler-registry-mysql/src/main/resources/mysql_registry_init.sql Outdated Show resolved Hide resolved
...c/main/java/org/apache/dolphinscheduler/plugin/registry/mysql/task/SubscribeDataManager.java Outdated Show resolved Hide resolved
kezhenxu94
kezhenxu94 previously approved these changes Jun 11, 2022
View reviewed changes
Copy link
Member

@kezhenxu94 kezhenxu94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits, generally good to me

...try-mysql/src/main/java/org/apache/dolphinscheduler/plugin/registry/mysql/MysqlOperator.java Outdated Show resolved Hide resolved
...try-mysql/src/main/java/org/apache/dolphinscheduler/plugin/registry/mysql/MysqlOperator.java Show resolved Hide resolved
@caishunfeng
Copy link
Contributor

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs Vulnerability A 0 Vulnerabilities Security Hotspot E 4 Security Hotspots Code Smell A 5 Code Smells

5.0% 5.0% Coverage 0.0% 0.0% Duplication

@ruanwenjun please take a look of the Security Hotspots.

@sonarcloud
Copy link

sonarcloud bot commented Jun 12, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 5 Code Smells

4.9% 4.9% Coverage
0.0% 0.0% Duplication

@ruanwenjun
Copy link
Member Author

SonarCloud Quality Gate failed.    Quality Gate failed
Bug A 0 Bugs Vulnerability A 0 Vulnerabilities Security Hotspot E 4 Security Hotspots Code Smell A 5 Code Smells
5.0% 5.0% Coverage 0.0% 0.0% Duplication

@ruanwenjun please take a look of the Security Hotspots.

Done

caishunfeng
caishunfeng approved these changes Jun 13, 2022
View reviewed changes
Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @ruanwenjun to add this feature.

@SbloodyS
Copy link
Member

Should we add this feature to the docs too? @ruanwenjun

@ruanwenjun
Copy link
Member Author

Should we add this feature to the docs too? @ruanwenjun

Yes, right now, I have added a README.md under the dolphinscheduler-registry-mysql,
We may need to add a new label about registry under the docs. This will need to do some changes to about the website.

SbloodyS
SbloodyS approved these changes Jun 13, 2022
View reviewed changes
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@ruanwenjun ruanwenjun merged commit 78c5fcc into apache:dev Jun 13, 2022
@ruanwenjun ruanwenjun deleted the dev_wenjun_mysqlRegistry branch June 21, 2022 09:49
hstdream pushed a commit to hstdream/dolphinscheduler that referenced this pull request Jun 23, 2022
@ruanwenjun
Add mysql registry plugin (apache#10406)
57ffeab 
* Add mysql registry plugin
ITBOX-ITBOY pushed a commit to ITBOX-ITBOY/dolphinscheduler that referenced this pull request Jul 8, 2022
@ruanwenjun
Add mysql registry plugin (apache#10406)
7f01005 
* Add mysql registry plugin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SbloodyS SbloodyS SbloodyS approved these changes

@caishunfeng caishunfeng caishunfeng approved these changes

@kezhenxu94 kezhenxu94 kezhenxu94 left review comments

Assignees

@ruanwenjun ruanwenjun

Labels
backend feature new feature
Projects
DolphinScheduler Roadmap
Status: Done
Milestone
3.1.0
Development

Successfully merging this pull request may close these issues.

[Feature][Registry] Add MySQL as registry plugin
5 participants
@ruanwenjun @codecov-commenter @caishunfeng @SbloodyS @kezhenxu94