Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schedule, schedulers: make schedulers aware of placement rules #1999

Merged
merged 6 commits into from
Dec 18, 2019

Conversation

disksing
Copy link
Contributor

@disksing disksing commented Dec 9, 2019

Signed-off-by: disksing i@disksing.com

What problem does this PR solve?

Make schedulers aware of placement rules.

What is changed and how it works?

When placement rule is enabled, use RuleFilter instead of DistinctScoreFilter.

Signed-off-by: disksing <i@disksing.com>
@disksing disksing added the component/schedule Scheduling logic. label Dec 9, 2019
@disksing disksing added this to the v4.0.0-beta milestone Dec 9, 2019
@disksing disksing requested review from lhy1024 and rleungx December 9, 2019 12:13
@codecov-io
Copy link

codecov-io commented Dec 9, 2019

Codecov Report

Merging #1999 into master will increase coverage by 0.04%.
The diff coverage is 74.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1999      +/-   ##
==========================================
+ Coverage   76.83%   76.88%   +0.04%     
==========================================
  Files         183      183              
  Lines       18301    18323      +22     
==========================================
+ Hits        14062    14088      +26     
+ Misses       3175     3172       -3     
+ Partials     1064     1063       -1
Impacted Files Coverage Δ
server/core/region_option.go 84.82% <100%> (ø) ⬆️
server/schedulers/shuffle_hot_region.go 62.62% <100%> (-3.34%) ⬇️
server/schedule/filter/filters.go 77.15% <100%> (+1.72%) ⬆️
server/schedule/region_scatterer.go 80.85% <40%> (-2.49%) ⬇️
server/schedulers/adjacent_region.go 74.34% <40%> (-1.06%) ⬇️
server/schedulers/hot_region.go 85.3% <70%> (-0.45%) ⬇️
server/region_syncer/client.go 77.63% <0%> (-5.27%) ⬇️
client/client.go 70.9% <0%> (-1.23%) ⬇️
server/member/leader.go 79.08% <0%> (-1.03%) ⬇️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3583afd...84ee9d9. Read the comment docs.

Copy link
Contributor

@lhy1024 lhy1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add tests for this PR?

Comment on lines 397 to 400
srcPeer := srcRegion.GetStorePeer(srcStoreID)
if srcPeer == nil {
return nil, nil, nil, Influence{}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why need to judge it in hot region rather than other schedulers?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use continue to just skip this peer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shafreeck good idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lhy1024 because hot region scheduler selects region from stats, which may be out of date. Others do not have this problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lhy1024 For shcedulers that include label related tests (hotWrite and shuffleHot), I have changed them to test twice, the second time run with placement rules enabled.

@disksing disksing mentioned this pull request Dec 13, 2019
9 tasks
@nolouch
Copy link
Contributor

nolouch commented Dec 18, 2019

PTAL @rleungx @shafreeck

@disksing disksing requested a review from lhy1024 December 18, 2019 06:43
Copy link
Member

@rleungx rleungx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@disksing
Copy link
Contributor Author

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 18, 2019
@sre-bot
Copy link
Contributor

sre-bot commented Dec 18, 2019

/run-all-tests

@sre-bot sre-bot merged commit 28d086e into tikv:master Dec 18, 2019
@disksing disksing deleted the placement12 branch December 18, 2019 08:31
Huster-ljw pushed a commit to Huster-ljw/pd that referenced this pull request Dec 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/schedule Scheduling logic. status/can-merge Indicates a PR has been approved by a committer.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants