Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document labels, annotations and taints for JobSet #47383

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Document labels, annotations and taints for JobSet #47383

Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
099e386
adding labels and annotation for JobSet
Adarsh-verma-14 Aug 7, 2024
0d815b6
update description for labels/annotaionand taint also
Adarsh-verma-14 Aug 9, 2024
1be0ff2
update description for labels/annotaionan
Adarsh-verma-14 Sep 11, 2024
eabf9c1
update types and description for labels/annotation
Adarsh-verma-14 Sep 20, 2024
1fb8971
update some changes regarding ReplicatedJob
Adarsh-verma-14 Sep 22, 2024
d21c85e
Modified some sentence
Adarsh-verma-14 Sep 29, 2024
fc4c0c7
update description for exclusive-topology
Adarsh-verma-14 Oct 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions content/en/docs/reference/labels-annotations-taints/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2332,6 +2332,108 @@ Starting in v1.16, this annotation was removed in favor of
[Pod Priority](/docs/concepts/scheduling-eviction/pod-priority-preemption/).
{{< /note >}}

### jobset.sigs.k8s.io/jobset-name

Type: Label, Annotation

Example: `jobset.sigs.k8s.io/jobset-name: "my-jobset"`

Used on: Jobs, Pods

This label/annotation is used to store the name of the JobSet that a Job or Pod belongs to.
[JobSet](https://jobset.sigs.k8s.io) is an extension API that you can deploy into your Kubernetes cluster.
sftim marked this conversation as resolved.
Show resolved Hide resolved

### jobset.sigs.k8s.io/replicatedjob-replicas

Type: Label, Annotation

Example: `jobset.sigs.k8s.io/replicatedjob-replicas: "5"`

Used on: ReplicatedJobs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in?


This label/annotation specifies the number of replicas for a ReplicatedJob.

### jobset.sigs.k8s.io/replicatedjob-name

Type: Label, Annotation

Example: `jobset.sigs.k8s.io/replicatedjob-name: "my-replicatedjob"`

Used on: Jobs, Pods

This label/annotation stores the name of the ReplicatedJob that this Job or Pod is part of.
sftim marked this conversation as resolved.
Show resolved Hide resolved

### jobset.sigs.k8s.io/job-index

Type: Label, Annotation

Example: `jobset.sigs.k8s.io/job-index: "0"`

Used on: Jobs, Pods

This label/annotation is set by the JobSet controller on child Jobs and Pods. It contains the index of the Job replica within its parent ReplicatedJob.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no ReplicatedJob API, it is just part of the JobSet API: https://github.com/kubernetes-sigs/jobset/blob/b92cbdcbe77f168a82cf12c9f349bdd053d2ae5c/api/jobset/v1alpha2/jobset_types.go#L85

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this needs fixing.


### jobset.sigs.k8s.io/job-key

Type: Label, Annotation

Example: `jobset.sigs.k8s.io/job-key=0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example: `jobset.sigs.k8s.io/job-key=0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb`
Example: `jobset.sigs.k8s.io/job-key: 0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb`

This doesn't look right. Is 0f1e93893c4cb372080804ddb9153093cb0d20cefdd37f653e739c232d363feb definitely a valid label value? It looks longer than is allowed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, we set the value as the SHA256 hash of the namespaced job name here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but a SHA256 hash is 64 UTF-8 (or ASCII) characters, and conformant label values are documented as being up to 63 characters long.

Something doesn't add up. Maybe there's an off-by-one in the API server (I hope not).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it's SHA1 not SHA256, my mistake. So it's only 40 characters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Minor docs bug then; we can merge this with or without a fix (still would need a post-merge fixup).


Used on: Jobs, Pods

This label/annotation is set by the JobSet controller on child Jobs and Pods of a JobSet. The value will be the SHA256 hash of the namespaced Job name.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This label/annotation is set by the JobSet controller on child Jobs and Pods of a JobSet. The value will be the SHA256 hash of the namespaced Job name.
The JobSet controller sets this label (and also an annotation with the same key) on child Jobs and
Pods of a JobSet. The value is the SHA256 hash of the namespaced Job name.


### alpha.jobset.sigs.k8s.io/exclusive-topology

Type: Label, Annotation

Example: `alpha.jobset.sigs.k8s.io/exclusive-topology: "zone"`

Used on: JobSets, ReplicatedJobs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there definitely an API kind named ReplicatedJob? If so, what API group would I find this in?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no API kind named ReplicatedJob, but the annotation can be used on the JobTemplate defined inside the ReplicatedJob struct

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so Used on: would be JobSet, Job I think.
JobTemplate is also not an API kind (unlike say the unusual case of PodTemplate)


You can set this label/annotation on a [JobSet](https://jobset.sigs.k8s.io) or a ReplicatedJob template to ensure exclusive job placement per topology group.
sftim marked this conversation as resolved.
Show resolved Hide resolved

### alpha.jobset.sigs.k8s.io/node-selector

Type: Label, Annotation

Example: `alpha.jobset.sigs.k8s.io/node-selector-strategy=true`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example: `alpha.jobset.sigs.k8s.io/node-selector-strategy=true`
Example: `alpha.jobset.sigs.k8s.io/node-selector-strategy: "true"`


Used on: Jobs, Pods

This label/annotation can be applied to a JobSet. When it's set, the JobSet controller modifies the Jobs and their corresponding Pods by adding node selectors. This ensures exclusive job placement per topology domain, restricting the scheduling of these Pods to specific nodes based on the strategy.

### alpha.jobset.sigs.k8s.io/namespaced-job
sftim marked this conversation as resolved.
Show resolved Hide resolved

Type: Annotation, Label

Example: `alpha.jobset.sigs.k8s.io/default_myjobset-replicatedjob-0`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a value for the label.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example: `alpha.jobset.sigs.k8s.io/default_myjobset-replicatedjob-0`
Example: `alpha.jobset.sigs.k8s.io/namespaced-job: default_myjobset-replicatedjob-0`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an aside, this will run into name length issues. The JobSet and Namespace can both have names that are up to 63 characters long, so the concatenation won't always fit into a label value.

(but not really relevant to this PR).


Used on: Jobs, Pods

The JobSet controller adds this label/annotation to Jobs and Pods that are part of a JobSet. It stores the namespaced Job name, which is used for logging and event management within the JobSet context.

### alpha.jobset.sigs.k8s.io/no-schedule

Type: Taint

Example: `alpha.jobset.sigs.k8s.io/no-schedule: NoSchedule`

Used on: Nodes

The [JobSet](https://jobset.sigs.k8s.io) controller uses this taint to support its node labeling exclusive placement strategy.

### jobset.sigs.k8s.io/coordinator

Type: Annotation, Label
Adarsh-verma-14 marked this conversation as resolved.
Show resolved Hide resolved

Example: `jobset.sigs.k8s.io/coordinator: "myjobset-workers-0-0.headless-svc"`

Used on: Jobs, Pods

This annotation/label is used on Jobs and Pods to store a stable network endpoint where the coordinator
pod can be reached if the [JobSet](https://jobset.sigs.k8s.io) spec defines the `.spec.coordinator` field.

## Annotations used for audit

<!-- sorted by annotation -->
Expand Down