Allow more control of the name of the directory created by the Access Point under Dynamic Provisioning

[jonathanrainer]

This is a continuation of #568, as that PR was opened incorrectly and messing around with it was much more trouble than just opening a new PR

Is this a bug fix or adding new feature?
This adds a new feature, mostly in response to discussions on Issue #543

What is this PR about? / Why do we need it?
At the moment if you use dynamic provisioning the name of the folder created on the EFS is the name of the generated PV however in some use cases this isn't desirable and leads to a lot of confusion. Under this scheme the name of the directory can be controlled by users using a new parameter subPathPattern. This allows users a higher level of control over the directories that are created as they can specify any combination of fixed strings and a set of predefined variables based on what the CSI Spec supports. To stop this causing problems with security, there is additionally a new featureFlag added called ensureUniqueDirectories which appends a UUID to the created structure so clashes will be avoided. This is true by default but allows users to explicitly opt into an unsafe mode if they wish to provide the guarantee that there will be not be clashes themselves rather than having the driver enforce it for them.

What testing is done?
I have added to the unit tests for the module and I believe this is sufficient because all that is being changed is the directory being added to the AccessPointOpts struct. There might want to be some E2E testing about what happens if that directory already exists but I'm happy to take pointers on that as this is my first PR to the repo.

fixes #543

[k8s-ci-robot]

Hi @jonathanrainer. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jonathanrainer]

Right, sorry about the PR wrangling, but now we're back I can continue the conversation from #568.

@iAnomaly, @wolffberg - This is where the PR continues. @wolffberg I've addressed your points about validating the created EFS path by explicitly validating it before we send it off the API and have added tests to that effect.

@wongma7 - I had a long think about your comments re: security and think I've come up with a compromise. My general view is always that things should be secure by default but should allow you to turn off those controls in the knowledge that you're explicitly opting in to having to manage those guarantees yourself. So to that end I've introduced another parameter ensureUniqueDirectories which adds a UUID to the end of any directory path specified to ensure that directories are unique. This is on by default so I imagine most users won't notice. You can turn this off to get the behaviour originally specified but throughout the README's and examples it makes it clear that then you are opting in to the responsibility to manage the guarantees yourself. This way it's not a surprise for users and it would require you to make a bad change in two places for something bad to happen.

I hope this is satisfactory and if not I'm more than happy to have a discussion about it. Further I realise some of the Go may not be 100% correct as I'm fairly new to the language and some of the most common idioms still elude me so feel free to comment away but hopefully this is a much better proposition than that proposed previously.

[iAnomaly]

This is awesome @jonathanrainer, really appreciate all your time on this and all the cases you covered with tests. I went looking for the omitted basePath, empty subPathPattern case (which achieves the AWS EFS Console default behavior of just / as the root path for the Access Point) and sure enough you've got that in there.

Thanks again, this will be an awesome flexible feature once merged.

[jonathanrainer]

/assign wongma7

[wongma7]

/ok-to-test

I will not have time to review for a while but in principle yes I want this merged, thanks for patience

[thomaspaulin]

I've run into a use case where these changes would be highly desirable. Thanks team!

[jonathanrainer]

Just to prove this actually works, I spun this up on an EKS cluster with the following StorageClass

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: "2022-06-12T15:46:35Z"
  labels:
    provisioning-type: dynamic
  name: efs-dynamic
  resourceVersion: "16864"
  uid: 983866ac-c731-45ac-a22d-1d1edaf7dbda
parameters:
  basePath: ""
  directoryPerms: "777"
  ensureUniqueDirectory: "false"
  fileSystemId: fs-05427a7b4af6c1fcc
  gidRangeEnd: "2000"
  gidRangeStart: "1000"
  provisioningMode: efs-ap
  subPathPattern: /dynamic/${.PVC.name}/foo
provisioner: efs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: Immediate

Which then generated the following on disk:

So my Unit Test predictions were good and it also showed the value of all the error handling as I got the StorageClass wrong a few times and every time my error handling directed me to the issue quickly.

[Ashley-wenyizha]

When we release this new feature and customer upgraded to adopt this feature, controller can not really find the subPathPattern or EnsureUniqueDirectory in its existing volume param.

From the logic here, it seems that will still use PV name for access point directory like before correct?

So theoretically this should not break any existing customer use cases

[jonathanrainer]

Yes that's correct, once this is released a customer has to explicitly opt into it for it to become active, by setting a subPath pattern. This was one of the key reasons for using a newly defined key so that this behaviour is an enhancement rather than replacement for what the driver already does.

ensureUniqueDirectory is the same, setting it on its own won't do anything, only when the combination of subPathPattern and ensureUniqueDirectory are set does it work. If neither are set then the code behaves exactly as it was before, with respect to the behaviour of basePath etc. So yes when this releases unless a customer actively changes their StorageClass to turn this new behaviour on they won't notice any difference.

[Ashley-wenyizha]

Hi Jonathan @jonathanrainer,

Thanks for the patience and apologize about the delay, this PR lgtm, could you give it a rebase and squash into 1-3 commits? We can merge this afterwards and you can continue to revise PR 732

thanks!

[harshad-jpmc]

bump!

[Ashley-wenyizha]

Hi Jonathan @jonathanrainer,

this PR lgtm, could you give it a pull + rebase and squash into 1-3 commits? We can merge this afterwards and you can continue to revise PR 732

thanks!

[jonathanrainer]

@Ashley-wenyizha Hi, sorry for the delay, real life stuff got a bit on top of me! But have done what you wanted now and this is ready to merge as and when you'd like :)

[jonathanrainer]

@Ashley-wenyizha FYI I had to make a small change to the sanity tests because the EFS API Request won't accept paths more than 100 characters but the sanity attempts to use a volume name of 128 characters. In this case I think it's fine because this is the mechanism by which you'd get around exactly this problem but if you think there's a better solution then let me know

[Ashley-wenyizha]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ashley-wenyizha, iAnomaly, jonathanrainer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Ashley-wenyizha]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[joshuaganger]

This is fantastic. Any idea what version we can expect this feature to be available in?

Thanks for all the work that went into this.

[joebowbeer]

@jonathanrainer It looks like the new properties (subPathPattern, ensureUniqueDirectory) were released with helm chart 2.4.9, but not in the driver appVersion 1.6.0 that this helm chart uses.

When will these properties be supported in a released appVersion?

Also, note the typo in the helm docs: s/ensureUniqueDirectories/ensureUniqueDirectory/