Smarter node draining handling when using CSI plugins #11614
Labels
stage/accepted
Confirmed, and intend to work on. No timeline committment though.
theme/storage
type/enhancement
Projects
Milestone
Proposal
Currently, when performing a node drain on a client that has a CSI plugin running as a system task, you have to go through the workflow below in order to drain the node without leaving volumes attached to it:
Failure to perform the drain procedure in steps like this can result in volumes being left attached to a node causing jobs that were drained to not be able to reschedule.
There are probably multiple ways to address this scenario but I think ideally, the node draining logic would be able to tap into the CSI Plugin state to determine if there are tasks with volumes mounted on the node being drained and if so, wait for each of the volumes to be detached before spinning down the CSI plugin jobs registered to handle those volumes.
Use-cases
Being able to perform a single Node Drain and have jobs using CSI volumes be properly handled to ensure they are able to be rescheduled appropriately.
Attempted Solutions
As mentioned above, this can be worked around by performing the drain in multiple steps, first without draining system jobs, then draining system jobs after everything has migrated.
The text was updated successfully, but these errors were encountered: