Skip to content

Commit

Permalink
docs: document handling of local volumes on missing nodes
Browse files Browse the repository at this point in the history
This is intentionally a separate section because although it applies
to distributed provisioning, the same problem also arises when a CSI
driver handles provisioning of local volumes differently.
  • Loading branch information
pohly committed Dec 4, 2020
1 parent c517af4 commit 1aaef5f
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,6 +333,21 @@ instead implement a custom policy in a separate controller which sets
the "selected node" annotation to trigger local provisioning on the
desired node.

### Deleting local volumes after a node failure or removal

When a node with local volumes gets removed from a cluster before
deleting those volumes, the PV and PVC objects may still exist. It may
be possible to remove the PVC normally if the volume was not in use by
any pod on the node, but normal deletion of the volume and thus
deletion of the PV is not possible anymore because the CSI driver
instance on the node is not available or reachable anymore and therefore
Kubernetes cannot be sure that it is okay to remove the PV.

When an administrator is sure that the node is never going to come
back, then this situation can be resolved manually:
- force-delete objects: `kubectl delete pv <pv> --wait=false --grace-period=0 --force`
- remove all finalizers: `kubectl patch pv <pv> -p '{"metadata":{"finalizers":null}}`

## Community, discussion, contribution, and support

Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/).
Expand Down

0 comments on commit 1aaef5f

Please sign in to comment.