Affinity values result in error

[jpds]

I normally set the following in my charts for affinity settings:

affinity: |
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        topologyKey: kubernetes.io/hostname
        labelSelector:
          matchLabels:
            app: {{ template "chart-name.fullname" . }}
            release: {{ .Release.Name }}

Setting something like this in flux for a chart's values: results in:

flux/flux-7fd44b4c74-r2jz7[flux]: ts=2019-03-13T10:55:11.325105707Z caller=daemon.go:609 component=daemon event="Sync: 9efbf69, production-namespace:helmrelease/production-app" logupstream=false
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.356647556Z caller=chartsync.go:525 component=chartsync error="Release production-app: values have diverged due to manual Chart release"
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.356721436Z caller=release.go:139 component=release info="processing release production-app" action=UPDATE options="{DryRun:false ReuseName:false}" timeout=300s
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.458389512Z caller=release.go:214 component=release error="Chart upgrade release failed: production-app: &status.statusError{Code:2, Message:\"failed to create patch: failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore struct from: string\", Details:[]*any.Any(nil)}"
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.465620376Z caller=chartsync.go:370 component=chartsync warning="Failed to upgrade chart" namespace=production-namespace name=production-app error="rpc error: code = Unknown desc = failed to create patch: failed to get versionedObject: unable to convert unstructured object to apps/v1beta2, Kind=Deployment: cannot restore struct from: string"
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.465698444Z caller=operator.go:214 component=operator info="Successfully synced 'production-namespace/production-app'"
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:11.466009822Z caller=operator.go:169 component=operator debug="Processing next work queue job ..."
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: I0313 10:55:11.466242       7 event.go:221] Event(v1.ObjectReference{Kind:"HelmRelease", Namespace:"production-namespace", Name:"production-app", UID:"5aea4958-44e1-11e9-a0f1-02dab9d7b196", APIVersion:"flux.weave.works/v1beta1", ResourceVersion:"1440711", FieldPath:""}): type: 'Normal' reason: 'ChartSynced' Chart managed by HelmRelease processed successfully
flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T10:55:13.085039689Z caller=chartsync.go:247 component=chartsync info="Start of releasesync"

Even if I put topologyKey, app and release in strings.

After deleting the release, flux outputs:

flux/flux-helm-operator-7fb864d794-2wrnk[flux-helm-operator]: ts=2019-03-13T11:12:41.333578114Z caller=release.go:186 component=release error="Chart release failed: production-app: &status.statusError{Code:2, Message:\"release production-app failed: Deployment in version \\\"v1beta2\\\" cannot be handled as a Deployment: v1beta2.Deployment.Spec: v1beta2.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Affinity: readObjectStart: expect { or n, but found \\\", error found in #10 byte of ...|ffinity\\\":\\\"podAntiAff|..., bigger context ...|se\\\":\\\"production-app\\\"}},\\\"spec\\\":{\\\"affinity\\\":\\\"podAntiAffinity:\\\\n  preferredDuringSchedulingIgnor|...\", Details:[]*any.Any(nil)}"

[squaremo]

What's the chart you're using? If it's an internal chart, perhaps you could suggest the minimal conditions necessary to reproduce the problem (my guess is: a chart that inlines a string as YAML in a template)

[jpds]

This is indeed an internal chart which was made with helm create with v2.10.0 with no changes. It does use toYaml in the deployment template.

[jpds]

I do have the same issue with the above affinity settings with the public nginx-ingress chart for its controller.affinity value.

[squaremo]

I do have the same issue with the above affinity settings with the public nginx-ingress chart

Perfect, we can use that to try and reproduce the problem.

[hiddeco]

You are declaring your affinity rules as a so called 'literal block' (using a pipe |), this causes Helm to interpret it as a literal string when it is being cast to the toYaml function.

You will also note this when trying to replicate this issue using the helm command, when you know what to look for, as this will give an error message that looks like the following one (note the " before podAntiAff):

$ helm install stable/nginx-ingress --name nginx-ingress -f values.yaml
Error: release nginx-ingress failed: Deployment in version "v1beta1" cannot be handled as a Deployment: v1beta1.Deployment.Spec: v1beta1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Affinity: readObjectStart: expect { or n, but found ", error found in #10 byte of ...|ffinity":"podAntiAff|..., bigger context ...|,"release":"nginx-ingress"}},"spec":{"affinity":"podAntiAffinity:\n  preferredDuringSchedulingIgnor|...`

---

To summarize: the following HelmRelease works for me:

---
apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
  name: nginx-ingress
  namespace: default
spec:
  releaseName: nginx-ingress
  chart:
    repository: https://kubernetes-charts.storage.googleapis.com/
    name: nginx-ingress
    version: 1.4.0
  values:
    controller:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              topologyKey: kubernetes.io/hostname
              labelSelector:
                matchLabels:
                  app: nginx-ingress
                  release: nginx-ingres

and generates the following Deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "2"
    flux.weave.works/antecedent: default:helmrelease/nginx-ingress
  creationTimestamp: "2019-04-15T18:47:57Z"
  generation: 3
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.4.0
    component: controller
    heritage: Tiller
    release: nginx-ingress
  name: nginx-ingress-controller
  namespace: default
  resourceVersion: "375616"
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/nginx-ingress-controller
  uid: fc91c396-5fae-11e9-b762-b031d855523e
spec:
<snip>
  template:
<snip>
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchLabels:
                  app: nginx-ingress
                  release: nginx-ingress
              topologyKey: kubernetes.io/hostname
            weight: 100

[jpds]

I thought I'd tried it without the pipe and got the same error but this is working perfectly for me now. Thanks @hiddeco!