-
Notifications
You must be signed in to change notification settings - Fork 14.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document labels, annotations and taints for JobSet #47383
base: main
Are you sure you want to change the base?
Changes from 5 commits
099e386
0d815b6
1be0ff2
eabf9c1
1fb8971
d21c85e
fc4c0c7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -2332,6 +2332,108 @@ Starting in v1.16, this annotation was removed in favor of | |||||||
[Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/). | ||||||||
{{< /note >}} | ||||||||
|
||||||||
### jobset.sigs.k8s.io/jobset-name | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/jobset-name: "my-jobset"` | ||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This label/annotation is used to store the name of the JobSet to which a Job or Pod belongs. | ||||||||
[JobSet](https://jobset.sigs.k8s.io) is an extension API that you can deploy into your Kubernetes cluster. | ||||||||
|
||||||||
### jobset.sigs.k8s.io/replicatedjob-replicas | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/replicatedjob-replicas: "5"` | ||||||||
|
||||||||
Used on: ReplicatedJobs | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in? |
||||||||
|
||||||||
This label/annotation specifies the number of replicas for a ReplicatedJob. | ||||||||
|
||||||||
### jobset.sigs.k8s.io/replicatedjob-name | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/replicatedjob-name: "my-replicatedjob"` | ||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This label/annotation is used to index into a Job's labels and retrieve the name of the parent ReplicatedJob. | ||||||||
sftim marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
### jobset.sigs.k8s.io/job-index | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/job-index: "0"` | ||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This label/annotation is set by the JobSet controller on child Jobs and Pods. It contains the index of the Job replica within its parent ReplicatedJob. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There's no ReplicatedJob API, it is just part of the JobSet API: https://github.com/kubernetes-sigs/jobset/blob/b92cbdcbe77f168a82cf12c9f349bdd053d2ae5c/api/jobset/v1alpha2/jobset_types.go#L85 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, this needs fixing. |
||||||||
|
||||||||
### jobset.sigs.k8s.io/job-key | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/job-key=0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
This doesn't look right. Is There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is, we set the value as the SHA256 hash of the namespaced job name here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, but a SHA256 hash is 64 UTF-8 (or ASCII) characters, and conformant label values are documented as being up to 63 characters long. Something doesn't add up. Maybe there's an off-by-one in the API server (I hope not). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry, it's SHA1 not SHA256, my mistake. So it's only 40 characters. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK. Minor docs bug then; we can merge this with or without a fix (still would need a post-merge fixup). |
||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This label/annotation is set by the JobSet controller on child Jobs and Pods of a JobSet. The value will be the SHA256 hash of the namespaced Job name. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
||||||||
### alpha.jobset.sigs.k8s.io/exclusive-topology | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `alpha.jobset.sigs.k8s.io/exclusive-topology: "zone"` | ||||||||
|
||||||||
Used on: JobSets, ReplicatedJobs | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is no API kind named ReplicatedJob, but the annotation can be used on the JobTemplate defined inside the ReplicatedJob struct There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. OK, so |
||||||||
|
||||||||
You can set this label/annotation on a [JobSet](https://jobset.sigs.k8s.io) or a ReplicatedJob template to ensure exclusive job placement per topology group. | ||||||||
sftim marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
### alpha.jobset.sigs.k8s.io/node-selector | ||||||||
|
||||||||
Type: Label, Annotation | ||||||||
|
||||||||
Example: `alpha.jobset.sigs.k8s.io/node-selector-strategy=true` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This label/annotation can be applied to a JobSet. When it's set, the JobSet controller modifies the Jobs and their corresponding Pods by adding node selectors. This ensures exclusive job placement per topology domain, restricting the scheduling of these Pods to specific nodes based on the strategy. | ||||||||
|
||||||||
### alpha.jobset.sigs.k8s.io/namespaced-job | ||||||||
sftim marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
Type: Annotation, Label | ||||||||
|
||||||||
Example: `alpha.jobset.sigs.k8s.io/default_myjobset-replicatedjob-0` | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There should be a value for the label. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As an aside, this will run into name length issues. The JobSet and Namespace can both have names that are up to 63 characters long, so the concatenation won't always fit into a label value. (but not really relevant to this PR). |
||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
The JobSet controller adds this label/annotation to Jobs and Pods that are part of a JobSet. It stores the namespaced Job name, which is used for logging and event management within the JobSet context. | ||||||||
|
||||||||
### alpha.jobset.sigs.k8s.io/no-schedule | ||||||||
|
||||||||
Type: Taint | ||||||||
|
||||||||
Example: `alpha.jobset.sigs.k8s.io/no-schedule: NoSchedule` | ||||||||
|
||||||||
Used on: Nodes | ||||||||
|
||||||||
The [JobSet](https://jobset.sigs.k8s.io) controller uses this taint to support its node labeling exclusive placement strategy. | ||||||||
|
||||||||
### jobset.sigs.k8s.io/coordinator | ||||||||
|
||||||||
Type: Annotation, Label | ||||||||
Adarsh-verma-14 marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
Example: `jobset.sigs.k8s.io/coordinator: "myjobset-workers-0-0.headless-svc"` | ||||||||
|
||||||||
Used on: Jobs, Pods | ||||||||
|
||||||||
This annotation/label is used on Jobs and Pods to store a stable network endpoint where the coordinator | ||||||||
pod can be reached if the [JobSet](https://jobset.sigs.k8s.io) spec defines the `.spec.coordinator` field. | ||||||||
|
||||||||
## Annotations used for audit | ||||||||
|
||||||||
<!-- sorted by annotation --> | ||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.