ignore tfjob/pytorch job if corresponding CRD not created #335
Conversation
|
/assign @richardsliu @YujiOshima |
|
/assign @johnugeorge |
|
@johnugeorge Do we still need this fix based on what we discussed today? |
pkg/controller/studyjob/utils.go
Outdated
| func validateWorkerResource(wkind string) error { | ||
| for _, crd := range invalidCRDResources { | ||
| if crd == wkind { | ||
| return fmt.Errorf("Cannot support %s; If CRD for it installed, please restart studyjob-controller to take effect", wkind) |
There was a problem hiding this comment.
Maybe specifically say "Please install the CRD and restart studyjob-controller"?
| @@ -117,7 +121,8 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error { | |||
| OwnerType: &katibv1alpha1.StudyJob{}, | |||
| }) | |||
| if err != nil { | |||
There was a problem hiding this comment.
invalidCRDResources should be updated only when CRD is not installed. We should explicitly check whether CRD exists or not. If watch fails for any other reason(even when CRD is installed), controller should fail by raising error.
You can use crd clientset to check.
https://github.com/kubeflow/tf-operator/blob/master/cmd/tf-operator.v1beta1/app/server.go#L192
pkg/controller/studyjob/utils.go
Outdated
| return nil | ||
| } | ||
|
|
||
| func ignoreWatchError(err error, job string) bool { |
There was a problem hiding this comment.
The name of the function is a little confusing. Maybe change it to "isFatalWatchError" and return the opposite boolean values?
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: richardsliu The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
As a short-term solution
Fixes: #317
This change is