-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
keep bootstrapper pod alive when error occurs #921
Conversation
/assign @ankushagarwal |
/retest |
bootstrap/cmd/bootstrap/main.go
Outdated
log.Fatalf("%v\n", err) | ||
if s.InCluster && s.KeepAlive { | ||
log.Infof("Bootstrapper failed with error: %v\n", err) | ||
log.Infof("Keeping pod alive...") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please log a description of why we're doing this so that it's easier for the user to understand what's happening
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ankushagarwal The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/hold |
bootstrap/cmd/bootstrap/main.go
Outdated
@@ -43,6 +44,14 @@ func main() { | |||
} | |||
|
|||
if err := app.Run(s); err != nil { | |||
log.Fatalf("%v\n", err) | |||
if s.InCluster && s.KeepAlive { | |||
log.Infof("Bootstrapper failed with error: %v\n", err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use log.Errorf here since you are logging an error.
One minor comment but otherwise looks good. |
/lgtm |
/retest |
/hold cancel |
Looks like a test flake; filed /test all |
/retest |
3 similar comments
/retest |
/retest |
/retest |
/test all Looks like the most recent failure was a minikube error waiting for the VM to be deleted. |
* keep bootstrapper pod alive when running inside k8s * Edit message when error happens * handle review feedback
…rces (kubeflow#921) * Applications should not try to take ownership of cluster scoped resources. * See kubeflow#4767 - If applications try to take ownership of cluster scoped resources or resources in other namespaces this can violate requirements of the K8s GC and lead to unpredictable behavior. * It looks like this might be causing the profile controller deployment to get GC'd after 24 hours. * Bump application version.
fix #901
This change is