Skip to content
This repository has been archived by the owner on Oct 21, 2020. It is now read-only.

RBD provisioner use provisioner name as identitiy by default instead of random string #267

Merged
merged 1 commit into from
Aug 2, 2017

Conversation

cofyc
Copy link
Contributor

@cofyc cofyc commented Aug 1, 2017

What this PR does / why we need it:

In production, provisioner should not use random identity on start.

Because, provisioner Pod may crash or be recreated on different nodes, if a provisioner generates random identity string each time, then it will cannot delete old PVs it provisioned before. Example errors logs:

I0801 11:03:05.685653    1901 controller.go:1068] scheduleOperation[delete-pvc-e2be4095-76a8-11e7-88c1-000c291fbe71[e2c845d1-76a8-11e7-88c1-000c291fbe71]]
I0801 11:03:05.688484    1901 controller.go:1035] deletion of volume "pvc-e2be4095-76a8-11e7-88c1-000c291fbe71" ignored: ignored because identity annotation on PV does not match ours
...

Most users deployed their RBD provisioner without specifying -id flag. So I change the default behavior to use provisioner name as identity (like some other provisions, gluster/block, etc) instead of random string.

Special notes for your reviewer:

In this PR, I also add PROVISIONER_NAME environment variable support, like many other provisioners do.

support `PROVISIONER_NAME` envionment variable.

In production, provisioner should not use random identity on start.
Because, provisioner Pod may crash or be recreated on different nodes, if
a provisioner generates random identity string each time, then it will
cannot manage old PVs it provisioned before. In this commit, I change
the default behavior to use `provision.ProvisionerName` constant as
default identity instead of random UUID string.
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 1, 2017
@cofyc cofyc mentioned this pull request Aug 1, 2017
@cofyc
Copy link
Contributor Author

cofyc commented Aug 1, 2017

/cc @rootfs @wongma7

@k8s-ci-robot k8s-ci-robot requested a review from wongma7 August 1, 2017 11:30
@k8s-ci-robot
Copy link
Contributor

@cofyc: GitHub didn't allow me to request PR reviews from the following users: rootfs.

Note that only kubernetes-incubator members can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @rootfs @wongma7

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@kfox1111
Copy link

kfox1111 commented Aug 1, 2017

Hmmm.... Does id have to be unique within the same type of provisioner?

Say I have a statefulset of ceph rbd provisioners. If I use string for id of 'rbd', then multiple could handle the request, but maybe thats ok? If not, maybe using the statefulset's hostname is a better default?

@wongma7
Copy link
Contributor

wongma7 commented Aug 1, 2017

@kfox1111 id doesn't have to be unique, except in rare cases where only the provisioner that created a PV is capable of deleting it. By giving all provisioners serving the same class the same id (same prName), they will simply race to Delete PVs and everything should turn out fine.

I'm okay with using prName as the default.

On one hand, it just "makes sense" that the provisioner that created a PV should be responsible for deleting it, so the recommendation made when gluster/block and cephfs were created was to force the user to specify an ID.

On the other, users expect provisioners to behave the same as the upstream mega-controller, i.e. if a PV has Delete it should be deleted, it doesn't matter by which provisioner instance. And maintaining state via an ID argument is painful, plus impossible with a statefulset.

@kfox1111
Copy link

kfox1111 commented Aug 1, 2017

ok. sounds good to me.

Does the cephfs one have the same issue currently or is it defaulting to hostname?

@wongma7
Copy link
Contributor

wongma7 commented Aug 1, 2017

Yes cephfs has the same issue, it is defaulting to random UUID atm.

Copy link
Contributor

@wongma7 wongma7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. I am too scared to say this should be best practice though :)

@wongma7 wongma7 added area/ceph/rbd lgtm Indicates that a PR is ready to be merged. labels Aug 2, 2017
@wongma7 wongma7 merged commit a8de80c into kubernetes-retired:master Aug 2, 2017
cofyc added a commit to cofyc/external-storage that referenced this pull request Aug 2, 2017
…, and

support `PROVISIONER_NAME` envionment variable.

See kubernetes-retired#267.
@cofyc cofyc deleted the rbd_provisioner_name branch November 15, 2017 03:09
yangruiray pushed a commit to yangruiray/external-storage that referenced this pull request Jul 19, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/ceph/rbd cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants