Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H2O Driverless AI Helm - Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update #20

Open
bmarolleau opened this issue Jan 17, 2019 · 1 comment

Comments

@bmarolleau
Copy link

Hello,
Here is the error I get when deploying on ICP 3.1.1 (no pb with ICP 2.x) due to the GPU management by K8s which changed with the latest versions of K8s. Symptom: POD scheduling error at helm install.
With ICP 3.1 & 3.1.1 (and K8s version >= 11) nvidia.com/gpu should be used instead of alpha.kubernetes.io/nvidia-gpu,
Here is a modified helm chart that works on my environment:
The critical part is in the Helm templates/deployment.yaml file in the requests/limits lines:

resources:
            limits:
            {{- if and (eq (.Capabilities.KubeVersion.Major|int) 1) (lt (.Capabilities.KubeVersion.Minor|int) 11) }}
              alpha.kubernetes.io/nvidia-gpu: {{ .Values.resources.limits.gpu }}
            {{- else }}
              nvidia.com/gpu: {{ .Values.resources.limits.gpu }}
            {{- end }}
              memory: {{ .Values.resources.limits.memory }}
            requests:
            {{- if and (eq (.Capabilities.KubeVersion.Major|int) 1) (lt (.Capabilities.KubeVersion.Minor|int) 11) }}
              alpha.kubernetes.io/nvidia-gpu: {{ .Values.resources.requests.gpu }}
            {{- else }}
              nvidia.com/gpu: {{ .Values.resources.requests.gpu }}
            {{- end }}
              memory: {{ .Values.resources.requests.memory }}

Here is the modified file to be placed in the templates folder of the helm chart, as an example:
deployment.zip

@bmarolleau bmarolleau changed the title Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update DriverlessAI Helm - Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update Jan 17, 2019
@bmarolleau bmarolleau changed the title DriverlessAI Helm - Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update H2O Driverless AI Helm - Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update Jan 17, 2019
@CreatureDev
Copy link

This issue will be fixed in the latest release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants