Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

koord-scheduler: only allow the leader to initialize controllers #764

Merged

Conversation

xulinfei1996
Copy link
Contributor

@xulinfei1996 xulinfei1996 commented Oct 31, 2022

Signed-off-by: xulinfei.xlf xulinfei.xlf@alibaba-inc.com

Ⅰ. Describe what this PR does

Ⅱ. Does this pull request fix one issue?

fixes #762

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in make test

@codecov
Copy link

codecov bot commented Oct 31, 2022

Codecov Report

Base: 68.40% // Head: 68.20% // Decreases project coverage by -0.19% ⚠️

Coverage data is based on head (c83db49) compared to base (a8e1fbc).
Patch coverage: 32.50% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #764      +/-   ##
==========================================
- Coverage   68.40%   68.20%   -0.20%     
==========================================
  Files         209      210       +1     
  Lines       24053    24120      +67     
==========================================
- Hits        16453    16452       -1     
- Misses       6455     6527      +72     
+ Partials     1145     1141       -4     
Flag Coverage Δ
unittests 68.20% <32.50%> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...duler/plugins/elasticquota/quota_overuse_revoke.go 70.58% <0.00%> (-9.66%) ⬇️
pkg/scheduler/frameworkext/controllers.go 14.28% <14.28%> (ø)
pkg/scheduler/frameworkext/framework_extender.go 60.16% <16.66%> (-2.34%) ⬇️
pkg/scheduler/plugins/coscheduling/coscheduling.go 62.20% <23.80%> (-9.73%) ⬇️
pkg/scheduler/plugins/elasticquota/controller.go 51.11% <60.00%> (-0.62%) ⬇️
pkg/scheduler/plugins/elasticquota/plugin.go 69.86% <66.66%> (-3.79%) ⬇️
...eduler/plugins/coscheduling/controller/podgroup.go 74.14% <77.77%> (-0.23%) ⬇️
...r/plugins/elasticquota/core/group_quota_manager.go 72.55% <100.00%> (+0.05%) ⬆️
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@eahydra
Copy link
Member

eahydra commented Oct 31, 2022

The kubernetes scheduling framework supports the configuration of the same plugin into multiple profiles. For example, schedulerName: a and schedulerName: b can configure the same plugin, and the same plugin will be constructed twice, generating two instances. Similarly, there should be only one ElasticQuota Controller globally, so not only should the current scheduler be the Leader, but also consider avoiding building multiple ElasticQuota Controllers. One of the core issues here is to consider how to deal with the coupling relationship between ElasticQuota Plugin, ElasticQuota Controller and Quota Manager.

@eahydra
Copy link
Member

eahydra commented Oct 31, 2022

There is another solution, instead of passing the status of the leader around and just to control whether the controller is started, it is better to define a new interface, and the FrameworkExtender perceives whether the Plugin provides a Controller, and saves these Controller instances, and then in the It is enough to start these Controllers when scheduler.Scheduler is started. In this way, the implementation is clearer, has good scalability, and is minimally intrusive to existing code, and can maintain only one Controller globally.

@eahydra eahydra changed the title koord-scheduler: only allow the leader to update elasticQuota and Cos… koord-scheduler: only allow the leader to initialize Controllers Oct 31, 2022
@eahydra eahydra changed the title koord-scheduler: only allow the leader to initialize Controllers koord-scheduler: only allow the leader to initialize controllers Oct 31, 2022
@xulinfei1996 xulinfei1996 force-pushed the leaderlock-quota-gang branch 3 times, most recently from f462f83 to a7c3d2a Compare November 1, 2022 09:54
@xulinfei1996 xulinfei1996 force-pushed the leaderlock-quota-gang branch 2 times, most recently from 753d82a to c27a314 Compare November 1, 2022 12:59
cmd/koord-scheduler/app/config/config.go Outdated Show resolved Hide resolved
pkg/scheduler/plugins/elasticquota/plugin.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/controllers/types.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/controllers/controllers.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/framework_extender.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/controllers.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/controllers.go Outdated Show resolved Hide resolved
pkg/scheduler/frameworkext/controllers.go Outdated Show resolved Hide resolved
Signed-off-by: xulinfei.xlf <xulinfei.xlf@alibaba-inc.com>
Copy link
Member

@eahydra eahydra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Contributor

@buptcozy buptcozy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: eahydra

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@koordinator-bot koordinator-bot bot merged commit 160c79d into koordinator-sh:main Nov 2, 2022
Copy link

@yihuifeng yihuifeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants