studyJob cannot recover once Completed or Failed

[hougangliu]

Now studyJob cannot recover once Completed or Failed.
When a studyJob CRD created, I can update it by "kubectl apply" or else, but if the studyJob condition is Completed or Failed, we never start next suggestion schedule.
For example, a user creates a studyJob with an invalid workerSpec, which would lead to spawnWorker return error and studyJob goes to Failed. Then the user can correct workerSpec by "kube apply", expecting studyJob re-triggered. However, nothing will happen.

@here we should discuss the behavior of updating a studyJob CRD (when it is in RUNNING, Completed or FAILED)

	if instance.Status.Condition == katibv1alpha1.ConditionCompleted || instance.Status.Condition == katibv1alpha1.ConditionFailed {
		nextSuggestionSchedule = false
	}

[hougangliu]

/help

[hougangliu]

/remove-help

[hougangliu]

@YujiOshima can you add label community/discussion for it

[hougangliu]

1. At least, for Failed studyJob, we should try to rehandle it when updated.
2. For completed studyJob, we should reject updating it by webhook(we need upgrade controller-runtime, and webhook can validate studyJob to fix studyJob controller is blocked by bad CR manifests #314).

[jlewi]

This seems like its working as intended to me.

Once a job reaches a terminal state (failed or succeseded) updates to the job should not be allowed.
This is consistent with how native K8s jobs work.

If a user wants to update the spec they could create a new job.

/cc @johnugeorge @richardsliu

[johnugeorge]

Agree. K8s job works in the same way.

[gaocegege]

/close

We deprecated v1alpha1.

[k8s-ci-robot]

@gaocegege: Closing this issue.

In response to this:

/close

We deprecated v1alpha1.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.