-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: always use -ignore-system on node drain with CSI #8606
Conversation
Postrun hooks for allocation runners don't currently block the registration of terminal health with the servers, which is what allows system jobs to be drained. So draining nodes with jobs that claim CSI volumes requires the `-ignore-system` job to ensure that the postrun hook for service jobs gets a chance to execute.
@@ -65,8 +67,10 @@ operation is desired. | |||
- `-no-deadline`: No deadline allows the allocations to drain off the node | |||
without being force stopped after a certain deadline. | |||
|
|||
- `-ignore-system`: Ignore system allows the drain to complete without stopping | |||
system job allocations. By default system jobs are stopped last. | |||
- `-ignore-system`: Ignore system allows the drain to complete without |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tgross I kinda wish this flag was default "on", I've yet had a case where I haven't had to set this to true in my various nomad deployments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I agree. Let me float that by the rest of the team to see if it's something we feel safe flipping before 1.0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
Postrun hooks for allocation runners don't currently block the registration of
terminal health with the servers, which is what allows system jobs to be
drained. So draining nodes with jobs that claim CSI volumes requires the
-ignore-system
job to ensure that the postrun hook for service jobs gets achance to execute.
This docs change is associated with #8580 inasmuch as that PR can't fully solve the node drain issue summarized in #8232 (comment)