Preemption based on resource flavor order

[ahg-g]

What would you like to be added:

The order of ResourceFlavors within a ClusterQueue represents preference of consumption.

Consider the case where a CQ has two RFs, standard then spot; and consider a group of low priority jobs coming in first consuming all standard quota, then a higher priority job is submitted; although the high priority job could start using the spot quota, I would like it to preempt the lower priority jobs so it can consume resources from the preferred RF (standard).

A similar scenario happens with borrowing: if CQ2 is borrowing resources from CQ1 from higher order RF, I want new jobs submitted to CQ1 to reclaim borrowed resources to fit in the preferred RF.

This behavior can be added as a knob on the CQ.

Why is this needed:

Allows setting up CQs in way that ensures stronger guarantees on their nominal quota.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[alculquicondor]

Related to #312

[KunWuLuan]

/assign

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[KunWuLuan]

/remove-lifecycle stale

[alculquicondor]

/reopen
Implementation is still under review.

[k8s-ci-robot]

@alculquicondor: Reopened this issue.

In response to this:

/reopen
Implementation is still under review.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[alculquicondor]

@KunWuLuan can you work on the documentation for this feature?

[KunWuLuan]

sure