feat(sheduler): take resource quota into consideration during pod group enqueue procedure

[YuxiJin-tobeyjin]

Try to solve: #1014

1. Take resourceQuota in the namespace into consideration during podGroup enqueue procedure

For example you can set your resource quota as follows, then your podGroup will stay in pending state if the resource request plus usage is more than quota

spec:
  hard:
    requests.cpu: "32"
    requests.memory: 32Gi
    requests.nvidia.com/gpu: "10"

2. Add a flag to control this feature, default is “false“, which means disable

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: YuxiJin-tobeyjin
To complete the pull request process, please assign kevin-wangzefeng
You can assign the PR to them by writing /assign @kevin-wangzefeng in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[YuxiJin-tobeyjin]

/assign @kevin-wangzefeng

[Thor-wl]

@william-wang @k82cn

[shinytang6]

l think its better to put idle.Sub(minReq); inqueue = true here so that the following else clause can be removed :)

[YuxiJin-tobeyjin]

Actually idle.Sub(minReq)；inqueue = true doesn't belong to this if branch;

If we put it here, idle.Sub will not be calculated under circumstances when there is no resourceQuota in the namespace but ssn.JobEnqueueable(job) && minReq.LessEqual(idle) is satisfied.

[shinytang6]

And l think even though job.PodGroup.Spec.MinResources == nil, resourceQuota should also be checked?

[YuxiJin-tobeyjin]

Considering minResources is nil, the pods of this podgroup won't consume any resoureQuota, so we don't need it to be checked? WDYT

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[Thor-wl]

Please resolve the conflictions

[Thor-wl]

Please fix the conflictions and bugs

[YuxiJin-tobeyjin]

Please fix the conflictions and bugs

Okay， I will try to handle the conflicts. Would you please give me some clues about bugs，tks.

[shinytang6]

Please fix the conflictions and bugs

Okay， I will try to handle the conflicts. Would you please give me some clues about bugs，tks.

Can y please resolve the conflicts, l think the original code is almost ready

[YuxiJin-tobeyjin]

Since enqueue structure has been changed in master and there are some logic defects in this design, we will give a new proposal with a plugin realization in the near future.

@merryzhou

This PR can be closed.