Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Mofed termination grace period param #443

Merged
merged 1 commit into from
Jan 4, 2023
Merged

Add Mofed termination grace period param #443

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions api/v1alpha1/nicclusterpolicy_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,12 @@ type OFEDDriverSpec struct {
CertConfig *ConfigMapNameReference `json:"certConfig,omitempty"`
// Optional: Custom package repository configuration for OFED container
RepoConfig *ConfigMapNameReference `json:"repoConfig,omitempty"`
// TerminationGracePeriodSeconds specifies the length of time in seconds
// to wait before killing the OFED pod on termination
// +optional
// +kubebuilder:default:=300
// +kubebuilder:validation:Minimum:=0
TerminationGracePeriodSeconds int64 `json:"terminationGracePeriodSeconds,omitempty"`
}

// NVPeerDriverSpec describes configuration options for NV Peer Memory driver
Expand Down
7 changes: 7 additions & 0 deletions bundle/manifests/mellanox.com_nicclusterpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,6 +475,13 @@ spec:
- initialDelaySeconds
- periodSeconds
type: object
terminationGracePeriodSeconds:
default: 300
description: TerminationGracePeriodSeconds specifies the length
of time in seconds to wait before killing the OFED pod on termination
format: int64
minimum: 0
type: integer
upgradePolicy:
description: Ofed auto-upgrade settings
properties:
Expand Down
1 change: 1 addition & 0 deletions bundle/manifests/nvidia-network-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ metadata:
"initialDelaySeconds": 10,
"periodSeconds": 20
},
"terminationGracePeriodSeconds": 300,
"upgradePolicy": {
"autoUpgrade": false,
"drain": {
Expand Down
9 changes: 1 addition & 8 deletions config/crd/bases/mellanox.com_hostdevicenetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: hostdevicenetworks.mellanox.com
spec:
Expand Down Expand Up @@ -98,9 +97,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
9 changes: 1 addition & 8 deletions config/crd/bases/mellanox.com_ipoibnetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: ipoibnetworks.mellanox.com
spec:
Expand Down Expand Up @@ -78,9 +77,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
9 changes: 1 addition & 8 deletions config/crd/bases/mellanox.com_macvlannetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we really need to have some makefile targets to get a pre-defined version of controller-gen/operator-sdk so all generate with same version

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a bug filed: #444

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version is defined here: https://github.com/Mellanox/network-operator/blob/master/Makefile#L248

The problem is that if you have an older version, it will not be updated unless you delete the older version from ./bin directory.

Copy link
Collaborator

@adrianchiris adrianchiris Jan 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack so it just making sure CRD are updated using up2date version. in that case i dont think we need the bug.

we could technically add a CI step to regenerated crds and make sure no diffs or simillar

creationTimestamp: null
name: macvlannetworks.mellanox.com
spec:
Expand Down Expand Up @@ -92,9 +91,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
23 changes: 15 additions & 8 deletions config/crd/bases/mellanox.com_nicclusterpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: nicclusterpolicies.mellanox.com
spec:
Expand Down Expand Up @@ -173,6 +172,7 @@ spec:
type: object
type: array
type: object
x-kubernetes-map-type: atomic
weight:
description: Weight associated with matching the corresponding
nodeSelectorTerm, in the range 1-100.
Expand Down Expand Up @@ -268,10 +268,12 @@ spec:
type: object
type: array
type: object
x-kubernetes-map-type: atomic
type: array
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
type: object
nvPeerDriver:
description: NVPeerDriverSpec describes configuration options for
Expand Down Expand Up @@ -353,6 +355,7 @@ spec:
required:
- key
type: object
x-kubernetes-map-type: atomic
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, `metadata.labels[''<KEY>'']`,
Expand All @@ -371,6 +374,7 @@ spec:
required:
- fieldPath
type: object
x-kubernetes-map-type: atomic
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
Expand All @@ -395,6 +399,7 @@ spec:
required:
- resource
type: object
x-kubernetes-map-type: atomic
secretKeyRef:
description: Selects a key of a secret in the pod's
namespace
Expand All @@ -415,6 +420,7 @@ spec:
required:
- key
type: object
x-kubernetes-map-type: atomic
type: object
required:
- name
Expand Down Expand Up @@ -470,6 +476,13 @@ spec:
- initialDelaySeconds
- periodSeconds
type: object
terminationGracePeriodSeconds:
default: 300
description: TerminationGracePeriodSeconds specifies the length
of time in seconds to wait before killing the OFED pod on termination
format: int64
minimum: 0
type: integer
upgradePolicy:
description: Ofed auto-upgrade settings
properties:
Expand Down Expand Up @@ -727,9 +740,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
1 change: 0 additions & 1 deletion config/rbac/role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
Expand Down
1 change: 1 addition & 0 deletions config/samples/mellanox.com_v1alpha1_nicclusterpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ spec:
image: mofed
repository: mellanox
version: 5.5-1.0.3.2
terminationGracePeriodSeconds: 300
e0ne marked this conversation as resolved.
Show resolved Hide resolved
startupProbe:
initialDelaySeconds: 10
periodSeconds: 20
Expand Down
1 change: 1 addition & 0 deletions deployment/network-operator/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,7 @@ imagePullSecrets:
| `ofedDriver.env` | list | `[]` | An optional list of [environment variables](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#envvar-v1-core) passed to the Mellanox OFED driver image |
| `ofedDriver.repoConfig.name` | string | `` | Private mirror repository configuration configMap name |
| `ofedDriver.certConfig.name` | string | `` | Custom TLS key/certificate configuration configMap name |
| `ofedDriver.terminationGracePeriodSeconds` | int | 300 | Mellanox OFED termination grace periods in seconds|
| `ofedDriver.startupProbe.initialDelaySeconds` | int | 10 | Mellanox OFED startup probe initial delay |
| `ofedDriver.startupProbe.periodSeconds` | int | 20 | Mellanox OFED startup probe interval |
| `ofedDriver.livenessProbe.initialDelaySeconds` | int | 30 | Mellanox OFED liveness probe initial delay |
Expand Down
9 changes: 1 addition & 8 deletions deployment/network-operator/crds/mellanox.com_hostdevicenetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: hostdevicenetworks.mellanox.com
spec:
Expand Down Expand Up @@ -98,9 +97,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
9 changes: 1 addition & 8 deletions deployment/network-operator/crds/mellanox.com_ipoibnetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: ipoibnetworks.mellanox.com
spec:
Expand Down Expand Up @@ -78,9 +77,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
9 changes: 1 addition & 8 deletions deployment/network-operator/crds/mellanox.com_macvlannetworks.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: macvlannetworks.mellanox.com
spec:
Expand Down Expand Up @@ -92,9 +91,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
23 changes: 15 additions & 8 deletions deployment/network-operator/crds/mellanox.com_nicclusterpolicies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
controller-gen.kubebuilder.io/version: v0.10.0
creationTimestamp: null
name: nicclusterpolicies.mellanox.com
spec:
Expand Down Expand Up @@ -173,6 +172,7 @@ spec:
type: object
type: array
type: object
x-kubernetes-map-type: atomic
weight:
description: Weight associated with matching the corresponding
nodeSelectorTerm, in the range 1-100.
Expand Down Expand Up @@ -268,10 +268,12 @@ spec:
type: object
type: array
type: object
x-kubernetes-map-type: atomic
type: array
required:
- nodeSelectorTerms
type: object
x-kubernetes-map-type: atomic
type: object
nvPeerDriver:
description: NVPeerDriverSpec describes configuration options for
Expand Down Expand Up @@ -353,6 +355,7 @@ spec:
required:
- key
type: object
x-kubernetes-map-type: atomic
fieldRef:
description: 'Selects a field of the pod: supports metadata.name,
metadata.namespace, `metadata.labels[''<KEY>'']`,
Expand All @@ -371,6 +374,7 @@ spec:
required:
- fieldPath
type: object
x-kubernetes-map-type: atomic
resourceFieldRef:
description: 'Selects a resource of the container: only
resources limits and requests (limits.cpu, limits.memory,
Expand All @@ -395,6 +399,7 @@ spec:
required:
- resource
type: object
x-kubernetes-map-type: atomic
secretKeyRef:
description: Selects a key of a secret in the pod's
namespace
Expand All @@ -415,6 +420,7 @@ spec:
required:
- key
type: object
x-kubernetes-map-type: atomic
type: object
required:
- name
Expand Down Expand Up @@ -470,6 +476,13 @@ spec:
- initialDelaySeconds
- periodSeconds
type: object
terminationGracePeriodSeconds:
default: 300
description: TerminationGracePeriodSeconds specifies the length
of time in seconds to wait before killing the OFED pod on termination
format: int64
minimum: 0
type: integer
upgradePolicy:
description: Ofed auto-upgrade settings
properties:
Expand Down Expand Up @@ -727,9 +740,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
1 change: 1 addition & 0 deletions deployment/network-operator/templates/mellanox.com_v1alpha1_nicclusterpolicy_cr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ spec:
name: {{ .Values.ofedDriver.repoConfig.name }}
{{- end }}
imagePullSecrets: {{ include "network-operator.ofed.imagePullSecrets" . | nindent 4 }}
terminationGracePeriodSeconds: {{ .Values.ofedDriver.terminationGracePeriodSeconds }}
startupProbe:
initialDelaySeconds: {{ .Values.ofedDriver.startupProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.ofedDriver.startupProbe.periodSeconds }}
Expand Down
1 change: 1 addition & 0 deletions deployment/network-operator/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ ofedDriver:
# env:
# - name: EXAMPLE_ENV_VAR
# value: example_env_var_value
terminationGracePeriodSeconds: 300
# Private mirror repository configuration
repoConfig:
name: ""
Expand Down
2 changes: 1 addition & 1 deletion manifests/stage-ofed-driver/0050_ofed-driver-ds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ spec:
failureThreshold: 1
periodSeconds: {{ .CrSpec.ReadinessProbe.PeriodSeconds }}
# unloading OFED modules can take more time than default terminationGracePeriod (30 sec)
terminationGracePeriodSeconds: 120
terminationGracePeriodSeconds: {{ .CrSpec.TerminationGracePeriodSeconds }}
volumes:
- name: run-mlnx-ofed
hostPath:
Expand Down