For debugging, it's possible to run the external-attacher on command line:
csi-attacher -kubeconfig ~/.kube/config -v 5 -csi-address /run/csi/socket
The external-attacher follows controller pattern and uses informers to watch for VolumeAttachment
and PersistentVolume
create/update/delete events. It filters out VolumeAttachment
instances with Attacher==<CSI driver name>
and processes these events in workqueues with exponential backoff. Real handling is deferred to Handler
interface.
Handler
interface has two implementations, trivial and real one.
Trivial handler will be used for CSI drivers that don't support ControllerPublish
calls and marks all VolumeAttachment
as attached. It does not use any finalizers. This attacher can also be used for testing.
"Real" attacher talks to CSI over socket (/run/csi/socket
by default, configurable by -csi-address
).
The attacher then:
-
Discovers the supported attacher name by
GetPluginInfo
calls. The attacher only processesVolumeAttachment
instances that haveAttacher==GetPluginInfoResponse.Name
. -
Uses
ControllerGetCapabilities
to find out if CSI driver supportsControllerPublish
calls. It degrades to trivial mode if not. -
Processes new/updated
VolumeAttachment
instances and attaches/detaches volumes:VolumeAttachment
withoutDeletionTimestamp
:- Ignore
VolumeAttachment
that wants to attach PV withDeletionTimestamp
. - A finalizer is added to
VolumeAttachment
instance to preserve the object after deletion so we can detach the volume. - A finalizer is added to referenced PV instance to preserve the PV. Attacher needs information from the PV to detach the volume.
- CSI
ControllerPublishVolume
is called. AttachmentMetadata
is saved toVolumeAttachment
.- On any error, the
VolumeAttachment
is re-queued with exponential backoff.
- Ignore
VolumeAttachment
withDeletionTimestamp
:- CSI
ControllerUnpublishVolume
is called. - A finalizer is removed from
VolumeAttachment
. At this point, the API server is going to delete this instance and "deletedVolumeAttachment
" event will be received.
- CSI
-
Processes deleted
VolumeAttachment
instances:- Pokes PV queue with name of the detached PV. This triggers removal of finalizer on PV, if needed.
-
Processes added/updated PV to remove finalizer on PVs:
- Ignore PVs that don't have DeletionTimestamp.
- Checks that the PV is not used by any
VolumeAttachment
instance. - Removes Attacher's finalizer on the PV if so.
- On any error, the PV is re-queued with exponential backoff.
Both PV queue and VolumeAttachment
queue run in parallel. To ensure that removal of PV finalizers work without races:
- The controller attaches PVs only when the PV has no DeletionTimestamp and has attacher's finalizer.
- The controller removes finalizer only from PVs that have DeletionTimestamp.
As consequence, the external-attacher must be available until all PVs that refer to the CSI driver are removed. Even fully detached PVs have external-attacher's finalizer that is removed only after the PV is marked for deletion.
Secondary cache and locks was considered to keep a map PV -> list of VolumeAttachments that use the PV. Attacher's finalizer could be removed from a PV immediately after the last VolumeAttachment was deleted. Keeping this map is either racy or requires long critical sections with complicated error recovery.
We use dep for management of vendor/
.
vendor/k8s.io
is manually copied from staging/
directory of work-in-progress API for CSI, namely kubernetes/kubernetes#54463.