Support Subvolume{Group} pinning in ceph csi

[humblec]

Describe the feature you'd like to have

Subvolumes and subvolume groups can be automatically pinned to ranks according
to policies. This can help distribute load across MDS ranks in predictable and
stable ways.

Pinning is configured by::
$ ceph fs subvolumegroup pin <vol_name> <group_name> <pin_type> <pin_setting>
or for subvolumes::
$ ceph fs subvolume pin <vol_name> <group_name> <pin_type> <pin_setting>

in Ceph CSI we have to do pinning for the subvolumegroup. The pin_type may be
one of export, distributed, or random.

So, for example, setting a distributed pinning strategy on a subvolume group::

$ ceph fs subvolumegroup pin cephfilesystem-a csi distributed 1

Will enable distributed subtree partitioning policy for the "csi" subvolume
group. This will cause every subvolume within the group to be automatically
pinned to one of the available ranks on the file system.

Additional Ref # ceph/ceph#43896

Cc @nixpanic

[phlogistonjohn]

Hi, @humblec I am assuming you'd prefer to access this via the api rather than cli if possible. Would you like me to file an issue in go-ceph for API calls for these or would you prefer to do it?

[humblec]

Hi, @humblec I am assuming you'd prefer to access this via the api rather than cli if possible.

Hi @phlogistonjohn yeah, indeed its good to have the api support and keep away from cli.

Would you like me to file an issue in go-ceph for API calls for these or would you prefer to do it?

Anything works for me , please let me know if you are filing it.

[phlogistonjohn]

Done. ceph/go-ceph#611 :-)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

[humblec]

go-ceph v0.16.0 ( the recent release ) whcih is the base for 3.7 dont have this functionality. so , moving the milestone to 3.8 https://github.com/ceph/ceph-csi/milestone/15

[Madhu-1]

I don't think it's a good idea for us to create subvolumegroups at cephcsi. It should be the responsibility of the ceph admin to handle this task. Instead, cephcsi should only create subvolumes that correspond to the PVC/PV.
Additionally, we should avoid including pin details in the storageclass or clusterID configmap since a single subvolumegroup might be used by multiple storageclasses/clusterIDs. Pinning details are subject to change and should be considered a dynamic configuration.
Lastly, it's best if the ceph admin manages the creation and management of the filesystem and subvolumegroup.

[Rakshith-R]

+1

As discussed, I think we can close this issue and let admin/rook at higher level handle pinning

[Rakshith-R]

Closing this epic as per above comments.
subvolumegroup pinning support will be added Rook in upcoming releases.
In standalone ceph clusters, storage admins needs to do it.