This repository has been archived by the owner on Oct 22, 2024. It is now read-only.
ControllerPublishVolume on Node called twice with same VolumeID #103
Comments
|
Olev Kartau <notifications@github.com> writes:
I spotted a creation error in Node log right after cluster reference
test cycle, i.e. deployment steps up to _my-csi-app_ pod. It appears
that ControllerPublishVolume on the Node is called twice in a row
without much delay (almost immediately). The first call created
device:
...
I need to try with smaller Volume request, does the node create double volumes then.
Not sure why driver gets called twice in that case (something in master, or CSI ?).
Driver at that point could check for registered VolumeID, if already
created, should refuse.
This happens because the caller has a very low timeout for the gRPC
call. This has tripped up ceph-csi before:
kubernetes-csi/external-provisioner#131
When the call times out, it just gets retried until it
succeeds. According to the standard, that is okay, because each call
must be idempotent.
In practice that means that a driver must:
- serialize gRPC calls (see for example ceph/ceph-csi#92)
- detect when a request was already completed earlier and then
return without error
|
|
just for reference, here is master log of same, indeed Master calls it again after like, hm 2,8 ms ? |
|
anyway, the right thing to do is to look into local volume registry where volumeID gets stored, and return OK if that ID is already there. |
|
fix proposal in #106 |
|
fix in #106 was merged, we can close this |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
I spotted a creation error in Node log right after cluster reference test cycle, i.e. deployment steps up to my-csi-app pod. It appears that ControllerPublishVolume on the Node is called twice in a row without much delay (almost immediately). The first call created device:
immediately followed by other with same params which fails, as there is no more 8G space:
I need to try with smaller Volume request, does the node create double volumes then.
Not sure why driver gets called twice in that case (something in master, or CSI ?).
Driver at that point could check for registered VolumeID, if already created, should refuse.
The text was updated successfully, but these errors were encountered: