Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize MasterServer, add MasterRPCService #10371

Merged
merged 10 commits into from
Jun 9, 2022
Merged

Optimize MasterServer, add MasterRPCService #10371

merged 10 commits into from
Jun 9, 2022

Conversation

ruanwenjun
Copy link
Member

Purpose of the pull request

  • Add MasterRPCServer, clean the code in MasterServer.
  • Avoid NPR in MasterSchedulerService

caishunfeng
caishunfeng reviewed Jun 8, 2022
View reviewed changes
Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good optimization, and some review comments here.

...main/java/org/apache/dolphinscheduler/server/master/processor/queue/TaskExecuteRunnable.java Outdated Show resolved Hide resolved
...in/java/org/apache/dolphinscheduler/server/master/processor/queue/TaskExecuteThreadPool.java Outdated Show resolved Hide resolved
...inscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/MasterServer.java Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

Merging #10371 (0880aea) into dev (ffe809b) will decrease coverage by 0.06%.
The diff coverage is 22.15%.

❗ Current head 0880aea differs from pull request most recent head 4ecbf46. Consider uploading reports for the commit 4ecbf46 to get more accurate results

@@             Coverage Diff              @@
##                dev   #10371      +/-   ##
============================================
- Coverage     40.56%   40.50%   -0.07%     
+ Complexity     4739     4736       -3     
============================================
  Files           869      871       +2     
  Lines         35273    35320      +47     
  Branches       3935     3936       +1     
============================================
- Hits          14309    14305       -4     
- Misses        19534    19588      +54     
+ Partials       1430     1427       -3
Impacted Files Coverage Δ
.../org/apache/dolphinscheduler/common/Constants.java 80.95% <0.00%> (ø)
...r/plugin/datasource/hive/HiveDataSourceClient.java 1.33% <0.00%> (-0.02%) ⬇️
...e/dolphinscheduler/server/master/MasterServer.java 0.00% <0.00%> (ø)
...ter/processor/queue/StateEventResponseService.java 0.00% <0.00%> (ø)
...er/master/processor/queue/TaskExecuteRunnable.java 0.00% <0.00%> (ø)
.../master/processor/queue/TaskExecuteThreadPool.java 2.27% <0.00%> (ø)
...master/registry/MasterConnectionStateListener.java 0.00% <0.00%> (ø)
...inscheduler/server/master/rpc/MasterRPCServer.java 0.00% <0.00%> (ø)
...uler/server/master/runner/EventExecuteService.java 0.00% <0.00%> (ø)
...er/server/master/runner/FailoverExecuteThread.java 0.00% <ø> (ø)
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffe809b...4ecbf46. Read the comment docs.

@ruanwenjun ruanwenjun requested a review from caishunfeng June 8, 2022 04:25
ruanwenjun and others added 10 commits June 8, 2022 19:54
@ruanwenjun
Optimize MasterServer, avoid NPE
ac57818
@ruanwenjun
Add optional
405d566
@ruanwenjun
Add some comments
88b4959
@ruanwenjun
Add license header
c8aba4f
@ruanwenjun
Remove unsed test
3b79a17
@ruanwenjun
Add master schedule check failover task
c7bd12c
@ruanwenjun
remove try
cfcf45e
@ruanwenjun @caishunfeng
Update dolphinscheduler-master/src/main/java/org/apache/dolphinschedu…
0e11829 
…ler/server/master/processor/queue/TaskExecuteRunnable.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
@ruanwenjun @caishunfeng
Update dolphinscheduler-master/src/main/java/org/apache/dolphinschedu…
3086fe2 
…ler/server/master/processor/queue/TaskExecuteThreadPool.java

Co-authored-by: caishunfeng <caishunfeng2021@gmail.com>
@ruanwenjun
fix typo
d4a000f
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_optimizeMaster branch from 2119a1b to d4a000f Compare June 8, 2022 11:54
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jun 8, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 44 Code Smells

7.4% 7.4% Coverage
0.5% 0.5% Duplication

caishunfeng
caishunfeng approved these changes Jun 9, 2022
View reviewed changes
Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ruanwenjun ruanwenjun merged commit 3ecbee3 into apache:dev Jun 9, 2022
@ruanwenjun ruanwenjun deleted the dev_wenjun_optimizeMaster branch June 9, 2022 02:00
Tianqi-Dotes pushed a commit to Tianqi-Dotes/dolphinscheduler that referenced this pull request Jun 16, 2022
@ruanwenjun @caishunfeng
Optimize MasterServer, add MasterRPCService (apache#10371)
8bcc609
Tianqi-Dotes pushed a commit to Tianqi-Dotes/dolphinscheduler that referenced this pull request Jun 16, 2022
@ruanwenjun @caishunfeng
Optimize MasterServer, add MasterRPCService (apache#10371)
95efb50
ITBOX-ITBOY pushed a commit to ITBOX-ITBOY/dolphinscheduler that referenced this pull request Jul 8, 2022
@ruanwenjun
Optimize MasterServer, add MasterRPCService (apache#10371)
67799e2 
* Optimize MasterServer, avoid NPE
@ruanwenjun ruanwenjun added this to the 3.0.0-release milestone Jul 14, 2022
ruanwenjun added a commit that referenced this pull request Jul 19, 2022
@ruanwenjun
Optimize MasterServer, add MasterRPCService (#10371)
81cadd1 
* Optimize MasterServer, avoid NPE

(cherry picked from commit 3ecbee3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@caishunfeng caishunfeng caishunfeng approved these changes

@SbloodyS SbloodyS Awaiting requested review from SbloodyS SbloodyS is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.0.0
Development

Successfully merging this pull request may close these issues.
3 participants
@ruanwenjun @codecov-commenter @caishunfeng