Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have prow inform child jobs about their parent #3966

Closed
wants to merge 1 commit into from
Closed

Have prow inform child jobs about their parent #3966

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions prow/kube/prowjob.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,13 @@ const (
)

type ProwJob struct {
APIVersion string `json:"apiVersion,omitempty"`
Kind string `json:"kind,omitempty"`
Metadata ObjectMeta `json:"metadata,omitempty"`
Spec ProwJobSpec `json:"spec,omitempty"`
Status ProwJobStatus `json:"status,omitempty"`
APIVersion string `json:"apiVersion,omitempty"`
Kind string `json:"kind,omitempty"`
Metadata ObjectMeta `json:"metadata,omitempty"`
Spec ProwJobSpec `json:"spec,omitempty"`
Status ProwJobStatus `json:"status,omitempty"`
ParentStatus ProwJobStatus `json:"parent_status,omitempty"`
ParentJob string `json:"parent_job,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API-wise it would be more correct to use OwnerReferences in the object metadata.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/kubernetes/community/blob/548eb7292d45a4af78e0141d33094130dfe6080a/contributors/design-proposals/garbage-collection.md
https://github.com/kubernetes/community/blob/31d62c70214d66ca2c21716c02b83d9087fdf3ff/contributors/design-proposals/controller-ref.md

It doesn't make sense to fully support owner references (adoption, orphaning) but we could use them for denoting relationships between prowjobs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't want to support any form of ownership, and I specifically intend to inject those environment variables mimicking the variables supplied to the current job, but for the parent. I think we may also need to support this for Jenkins jobs

I don't currently have other use cases besides a build job being followed by test jobs, but I think you could pass anything you wanted through the output buckets.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the flow is the build job pushes the artifacts in a bucket and plank injects the location of the bucket in the test jobs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an option, but I wanted to avoid restricting it to build jobs so the flow is:
Job writes to logs bucket alongside started.json, finished.json, etc with data for the child job to consume (say child_data.json or something), prow injects the parent job name / buildID into the child when it starts and the child goes to logs/$PARENT_JOB_NAME/$PARENT_BUILD_ID/ and uses child_data.json to do whatever (in this get the build location).

}

type ProwJobSpec struct {
Expand Down
22 changes: 20 additions & 2 deletions prow/plank/controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,10 @@ func (c *Controller) syncJenkinsJob(pj kube.ProwJob, reports chan<- kube.ProwJob
pj.Status.State = kube.SuccessState
pj.Status.Description = "Jenkins job succeeded."
for _, nj := range pj.Spec.RunAfterSuccess {
if _, err := c.kc.CreateProwJob(NewProwJob(nj)); err != nil {
nj := NewProwJob(nj)
nj.ParentJob = pj.Spec.Job
nj.ParentStatus = pj.Status
if _, err := c.kc.CreateProwJob(nj); err != nil {
return fmt.Errorf("error starting next prowjob: %v", err)
}
}
Expand Down Expand Up @@ -393,7 +396,10 @@ func (c *Controller) syncKubernetesJob(pj kube.ProwJob, pm map[string]kube.Pod,
pj.Status.Description = "Job succeeded."
reports <- pj
for _, nj := range pj.Spec.RunAfterSuccess {
if _, err := c.kc.CreateProwJob(NewProwJob(nj)); err != nil {
nj := NewProwJob(nj)
nj.ParentJob = pj.Spec.Job
nj.ParentStatus = pj.Status
if _, err := c.kc.CreateProwJob(nj); err != nil {
return fmt.Errorf("error starting next prowjob: %v", err)
}
}
Expand Down Expand Up @@ -443,6 +449,18 @@ func (c *Controller) startPod(pj kube.ProwJob) (string, string, error) {
Value: buildID,
},
)
if pj.ParentJob != "" {
spec.Containers[i].Env = append(spec.Containers[i].Env,
kube.EnvVar{
Name: "PARENT_JOB_NAME",
Value: pj.ParentJob,
},
kube.EnvVar{
Name: "PARENT_JOB_BUILD_NUMBER",
Value: pj.ParentStatus.BuildID,
},
)
}
if pj.Spec.Type == kube.PeriodicJob {
continue
}
Expand Down