feat: nutanix csi driver 3.0

[faiq]

What problem does this PR solve?:

Which issue(s) this PR fixes:
https://jira.nutanix.com/browse/D2IQ-100510

How Has This Been Tested?:

Tested by creating a cluster like this

$ clusterctl generate cluster faiqcsi3-take4   --from examples/capi-quick-start/nutanix-cluster-cilium-helm-addon.yaml   --kubernetes-version ${KUBERNETES_VERSION}   --worker-machine-count 3 |   kubectl apply --server-side -f -

using the following info

export NUTANIX_USER="<REDACTED>"
export NUTANIX_PASSWORD="<REDACTED>
export NUTANIX_STORAGE_CONTAINER_NAME="default"
export DOCKER_HUB_USERNAME="<REDACTED>"
export DOCKER_HUB_PASSWORD="<REDACTED>"
export NUTANIX_ENDPOINT="prismcentral.dev.ntnxsherlock.com"
export NUTANIX_PORT="9440"

export NUTANIX_PRISM_ELEMENT_CLUSTER_NAME="ganon"
export NUTANIX_INSECURE=false
export NUTANIX_SUBNET_NAME="sherlock_net"
export NUTANIX_MACHINE_TEMPLATE_IMAGE_NAME="ubuntu-2204-kube-v1.28.7.qcow2"
export CONTROL_PLANE_ENDPOINT_IP="10.40.142.X"
export KUBERNETES_VERSION="v1.28.7"

next get the kubeconfig:

clusterctl get kubeconfig faiqcsi3-take4 >> faiqcsi3-take4.conf

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx
    volumeMounts:
    - name: my-volume
      mountPath: /data
  volumes:
  - name: my-volume
    persistentVolumeClaim:
      claimName: my-pvc

apply that

$ kubectl --kubeconfig=faiqcsi3-take4.conf apply -f podpvc.yaml 

$ kubectl --kubeconfig=faiqcsi3-take4.conf get po 
NAME                                                 READY   STATUS              RESTARTS   AGE
cluster-autoscaler-faiqcsi3-take4-6ff4cbbd67-n2vdm   0/1     ContainerCreating   0          13m
my-pod                                               1/1     Running             0          11m

observe helmChartProxy

$ kubectl get helmchartproxy nutanix-csi-faiqcsi3-take4 -o yaml 
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
  creationTimestamp: "2024-04-17T19:53:09Z"
  finalizers:
  - helmchartproxy.addons.cluster.x-k8s.io
  generation: 1
  name: nutanix-csi-faiqcsi3-take4
  namespace: default
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    name: faiqcsi3-take4
    uid: f06bbe3b-0ed3-4bc6-8979-d7e368e0b6d2
  resourceVersion: "77210"
  uid: ae0956b0-c153-4c42-9186-24d1b37c617b
spec:
  chartName: nutanix-csi-storage
  clusterSelector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: faiqcsi3-take4
  namespace: ntnx-system
  options:
    enableClientCache: false
    install:
      createNamespace: true
    timeout: 10m0s
    upgrade:
      maxHistory: 10
  releaseName: nutanix-csi-storage
  repoURL: https://nutanix.github.io/helm-releases/
  valuesTemplate: |-
    # The Secret containing the credentials will be created by the handler.
    createPrismCentralSecret: false
  version: v3.0.0-beta.1912

Notice the version

Special notes for your reviewer:

[faiq]

For some reason the only storage container i'm able to access is called default

[faiq]

I see this from the controller logs

2024-04-17T20:23:25.063Z main.go:292: [WARN] Error getting the data from ConfigMap ntnx-cluster-configmap/ntnx-system : configmaps "ntnx-cluster-configmap" not found

it seems to configure something called categories https://github.com/nutanix-core/k8s-csi/blob/00151b610730c4e567cbee611891c9f4d03c4fee/cmd/init/main.go#L286. I'm not sure what is supposed to go in there.