Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In the same queue, a low-priority job preempts resources of a high-priority job #2612

Closed
wangyang0616 opened this issue Dec 22, 2022 · 0 comments · Fixed by #2613
Closed
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@wangyang0616
Copy link
Member

What happened:
Create two jobs in the same queue. Set the priority of job-a to 100 and that of job-b to 10.

Create job-a, set the number of copies to 12, and set minAvailable to 2 to use up idle CPUs in the cluster. After all pods of job-a are running, create job-b with replicas set to 6 and minAvailable set to 2. Job-b preempts job-a resources.

What you expected to happen:
The pod of job-b is in the pending state, and the pod of job-a is not preempted.

How to reproduce it (as minimally and precisely as possible):

  • job-a
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: share-1-priority
value: 100
globalDefault: false
description: "This priority class should be used for volcano job only."
---
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: job-a
spec:
  priorityClassName: share-1-priority
  schedulerName: volcano
  minAvailable: 2
  tasks:
    - replicas: 12
      name: "test"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 1000"]
              imagePullPolicy: IfNotPresent
              name: running
              resources:
                requests:
                  cpu: "1"
          restartPolicy: OnFailure
  • job-b
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: share-2-priority
value: 10
globalDefault: false
description: "This priority class should be used for volcano job only."
---
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: job-b
spec:
  priorityClassName: share-2-priority
  schedulerName: volcano
  minAvailable: 2
  tasks:
    - replicas: 6
      name: "test"
      template:
        spec:
          containers:
            - image: alpine
              command: ["/bin/sh", "-c", "sleep 1000"]
              imagePullPolicy: IfNotPresent
              name: running
              resources:
                requests:
                  cpu: "1"
          restartPolicy: OnFailure

Anything else we need to know?:

apiVersion: v1
data:
  volcano-scheduler-ci.conf: |
    actions: "enqueue, allocate, backfill, reclaim, preempt"
    tiers:
    - plugins:
      - name: priority
      - name: gang
      - name: conformance
      - name: sla
    - plugins:
      - name: overcommit
      - name: drf
      - name: predicates
      - name: proportion
      - name: nodeorder
      - name: binpack

Environment:

  • Volcano Version:
  • Kubernetes version (use kubectl version):
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant