IPPool CRD NV-IPAM

NV-IPAM updates: - Add support for IPPool CRD - Deprecate the configmap - Update templates with latest yamls configuration - Update documentation - Upgrade to v0.1.0 Signed-off-by: Fred Rolland <frolland@nvidia.com>
rollandf · Sep 21, 2023 · cd81508 · cd81508
1 parent b53f9d7
commit cd81508
Show file tree

Hide file tree

Showing 17 changed files with 294 additions and 157 deletions.
diff --git a/README.md b/README.md
@@ -219,12 +219,7 @@ spec:
   nvIpam:
     image: nvidia-k8s-ipam
     repository: ghcr.io/mellanox
-    version: v0.0.3
-    config: '{
-      "pools":  {
-        "my-pool": {"subnet": "192.168.0.0/24", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }'
+    version: v0.1.0
 ```
 
 Can be found at: `example/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-nvidia-ipam.yaml`
@@ -493,83 +488,6 @@ Specifying configuration either via Helm values when installing NVIDIA
 network operator, or by specifying them when directly creating NicClusterPolicy CR.
 These configurations eventually trigger the creation of a ConfigMap object in K8s.
 
-As an example, NVIDIA K8s IPAM plugin configuration is specified either via:
-
-__Helm values:__
-
-```yaml
-deployCR: true
-nvIpam:
-  deploy: true
-  image: nvidia-k8s-ipam
-  repository: ghcr.io/mellanox
-  version: v0.0.3
-  config: |-
-    {
-    "pools":  {
-      "rdma-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }
-```
-
-__NicClusterPolicy CR:__
-
-```yaml
-apiVersion: mellanox.com/v1alpha1
-kind: NicClusterPolicy
-metadata:
-  name: nic-cluster-policy
-spec:
-  nvIpam:
-    image: nvidia-k8s-ipam
-    repository: ghcr.io/mellanox
-    version: v0.0.3
-    config: '{
-      "pools":  {
-        "my-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }'
-```
-
-The configuration is then processed by the operator, eventually rendering and creating a _ConfigMap_, `nvidia-k8s-ipam-config`, within the
-namespace the operator was deployed. It contains the configuration for _nvidia k8s IPAM plugin_.
-
-For some advanced use-cases, it is desirable to provide such configurations at a later time.
-(e.g if network configuration is not known during Network Operator deployment time)
-
-To support this, it is possible to explicitly set such configuration to `nil` in Helm values
-or omit the `config` field of the relevant component while creating NicClusterPolicy CR.
-This will prevent Network Operator from
-creating such ConfigMaps, allowing the user to provide its own.
-
-Example (omitting nvidia k8s ipam config):
-
-__Helm values:__
-
-```yaml
-deployCR: true
-nvIpam:
-  deploy: true
-  image: nvidia-k8s-ipam
-  repository: ghcr.io/mellanox
-  version: v0.0.3
-  config: null
-```
-
-__NicClusterPolicy CR:__
-
-```yaml
-apiVersion: mellanox.com/v1alpha1
-kind: NicClusterPolicy
-metadata:
-  name: nic-cluster-policy
-spec:
-  nvIpam:
-    image: nvidia-k8s-ipam
-    repository: ghcr.io/mellanox
-    version: v0.0.3
-```
-
 > __Note__: It is the responsibility of the user to delete any existing configurations (ConfigMaps) if
 > they were already created by the Network Operator as well as deleting his own configuration when they
 > are no longer required.
diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml
@@ -242,6 +242,23 @@ rules:
   - get
   - list
   - watch
+- apiGroups:
+  - nv-ipam.nvidia.com
+  resources:
+  - ippools
+  verbs:
+  - create
+  - get
+  - list
+  - watch
+- apiGroups:
+  - nv-ipam.nvidia.com
+  resources:
+  - ippools/status
+  verbs:
+  - get
+  - patch
+  - update
 - apiGroups:
   - policy
   resources:

diff --git a/controllers/nicclusterpolicy_controller.go b/controllers/nicclusterpolicy_controller.go
@@ -74,6 +74,8 @@ type NicClusterPolicyReconciler struct {
 // +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=get;list;watch;create;update;patch;delete
 // +kubebuilder:rbac:groups=batch,resources=cronjobs,verbs=get;list;watch;create;update;patch;delete
 // +kubebuilder:rbac:groups=config.openshift.io,resources=proxies,verbs=get;list;watch
+// +kubebuilder:rbac:groups=nv-ipam.nvidia.com,resources=ippools,verbs=get;list;watch;create;
+// +kubebuilder:rbac:groups=nv-ipam.nvidia.com,resources=ippools/status,verbs=get;update;patch;
 
 // Reconcile is part of the main kubernetes reconciliation loop which aims to
 // move the current state of the cluster closer to the desired state.

diff --git a/deployment/network-operator/README.md b/deployment/network-operator/README.md
@@ -561,9 +561,9 @@ optionally deployed components:
 | `nvIpam.deploy`           | bool   | `false`            | Deploy NVIDIA IPAM Plugin                                                                |
 | `nvIpam.image`            | string | `nvidia-k8s-ipam`  | NVIDIA IPAM Plugin image name                                                            |
 | `nvIpam.repository`       | string | `ghcr.io/mellanox` | NVIDIA IPAM Plugin image repository                                                      |
-| `nvIpam.version`          | string | `v0.0.3`           | NVIDIA IPAM Plugin image version                                                         |
+| `nvIpam.version`          | string | `v0.1.0`           | NVIDIA IPAM Plugin image version                                                         |
 | `nvIpam.imagePullSecrets` | list   | `[]`               | An optional list of references to secrets to use for pulling any of the Plugin image     |
-| `nvIpam.config`           | string | `"{"pools": {"rdma-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}}}"` | Network pool configuration as described in [nvidia-k8s-ipam](https://github.com/Mellanox/nvidia-k8s-ipam), the default defines a single IP Pool named `"rdma-pool"`|
+| `nvIpam.config`           | string | Deprecated         | This field is ignored. Configuration is done by using IPPool CRD                                                             |
 
 ## Deployment Examples
 

diff --git a/deployment/network-operator/templates/mellanox.com_v1alpha1_nicclusterpolicy_cr.yaml b/deployment/network-operator/templates/mellanox.com_v1alpha1_nicclusterpolicy_cr.yaml
@@ -176,8 +176,5 @@ spec:
     repository: {{ .Values.nvIpam.repository }}
     version: {{ .Values.nvIpam.version }}
     imagePullSecrets: {{ include "network-operator.nvIpam.imagePullSecrets" . }}
-    {{- if .Values.nvIpam.config | empty | not }}
-    config: {{ .Values.nvIpam.config | quote }}
-    {{- end }}
   {{- end }}
 {{ end }}
diff --git a/deployment/network-operator/templates/role.yaml b/deployment/network-operator/templates/role.yaml
@@ -352,6 +352,23 @@ rules:
   - get
   - list
   - watch
+- apiGroups:
+  - nv-ipam.nvidia.com
+  resources:
+  - ippools
+  verbs:
+  - create
+  - get
+  - list
+  - watch
+- apiGroups:
+  - nv-ipam.nvidia.com
+  resources:
+  - ippools/status
+  verbs:
+  - get
+  - patch
+  - update
 - apiGroups:
   - policy
   resources:

diff --git a/deployment/network-operator/values.yaml b/deployment/network-operator/values.yaml
@@ -235,15 +235,8 @@ nvIpam:
   deploy: false
   image: nvidia-k8s-ipam
   repository: ghcr.io/mellanox
-  version: v0.0.3
+  version: v0.1.0
   # imagePullSecrets: []
-  # network pool configuration as described in https://github.com/Mellanox/nvidia-k8s-ipam
-  config: |-
-    {
-    "pools":  {
-      "rdma-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }
 
 secondaryNetwork:
   deploy: true

diff --git a/example/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-nvidia-ipam.yaml b/example/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-nvidia-ipam.yaml
@@ -61,9 +61,4 @@ spec:
   nvIpam:
     image: nvidia-k8s-ipam
     repository: ghcr.io/mellanox
-    version: v0.0.3
-    config: '{
-      "pools":  {
-        "my-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }'
+    version: v0.1.0
diff --git a/hack/release.yaml b/hack/release.yaml
@@ -52,4 +52,4 @@ IpamPlugin:
 nvIpam:
   image: nvidia-k8s-ipam
   repository: ghcr.io/mellanox
-  version: v0.0.3
+  version: v0.1.0
diff --git a/hack/templates/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-nvidia-ipam.template b/hack/templates/crs/mellanox.com_v1alpha1_nicclusterpolicy_cr-nvidia-ipam.template
@@ -62,8 +62,3 @@ spec:
     image: {{ .NvIPAM.Image }}
     repository: {{ .NvIPAM.Repository }}
     version: {{ .NvIPAM.Version }}
-    config: '{
-      "pools":  {
-        "my-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }'
diff --git a/hack/templates/values/values.template b/hack/templates/values/values.template
@@ -237,13 +237,6 @@ nvIpam:
   repository: {{ .NvIPAM.Repository }}
   version: {{ .NvIPAM.Version }}
   # imagePullSecrets: []
-  # network pool configuration as described in https://github.com/Mellanox/nvidia-k8s-ipam
-  config: |-
-    {
-    "pools":  {
-      "rdma-pool": {"subnet": "192.168.0.0/16", "perNodeBlockSize": 100, "gateway": "192.168.0.1"}
-      }
-    }
 
 secondaryNetwork:
   deploy: true