YAML anchors and aliases return error on List types

[rkr-kununu]

Describe the bug

There appears to be a bug regarding how kyaml deals with yaml anchors and alises. This issue effects kustomize v3.9.2 - v4.0.1

Files that can reproduce the issue

kustomization.yaml

resources:
- test.yaml

test.yaml

apiVersion: v1
kind: List
items:
- apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: test
  spec:
    selector:
      matchLabels:
        app: test
    template:
      metadata:
        labels:
          app: test
      spec:
        containers:
        - &base
          name: test
          image: ubuntu:20.04
          imagePullPolicy: Always
        initContainers:
        - <<: *base
          name: test2
- apiVersion: batch/v1beta1
  kind: CronJob
  metadata:
    name: testcron
  spec:
    suspend: true
    jobTemplate:
      spec:
        ttlSecondsAfterFinished: 60  # prune the logs after 60 seconds
        backoffLimit: 1  # Only try once
        template:
          spec:
           restartPolicy: Never
           containers:
           - <<: *base
             name: testcron

Expected output

Using kustomize v3.9.2, I can run the above code successfully via kustomize build . --enable_kyaml=false

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - image: ubuntu:20.04
        imagePullPolicy: Always
        name: test
      initContainers:
      - image: ubuntu:20.04
        imagePullPolicy: Always
        name: test2
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: testcron
spec:
  jobTemplate:
    spec:
      backoffLimit: 1
      template:
        spec:
          containers:
          - image: ubuntu:20.04
            imagePullPolicy: Always
            name: testcron
          restartPolicy: Never
      ttlSecondsAfterFinished: 60
  suspend: true

Actual output

Using kustomize v4.0.1:

Error: map[string]interface {}{"apiVersion":"apps/v1", "kind":"Deployment", "metadata":map[string]interface {}{"name":"test"}, "spec":map[string]interface {}{"selector":map[string]interface {}{"matchLabels":map[string]interface {}{"app":"test"}}, "template":map[string]interface {}{"metadata":map[string]interface {}{"labels":map[string]interface {}{"app":"test"}}, "spec":map[string]interface {}{"containers":[]interface {}{map[string]interface {}{"image":"ubuntu:20.04", "imagePullPolicy":"Always", "name":"test"}}, "initContainers":[]interface {}{map[interface {}]interface {}{"image":"ubuntu:20.04", "imagePullPolicy":"Always", "name":"test2"}}}}}}: json: unsupported type: map[interface {}]interface {}

Kustomize version

{Version:kustomize/v4.0.1 GitCommit:516ff1fa56040adc0173ff6ece66350eb4ed78a9 BuildDate:2021-02-13T21:21:14Z GoOs:linux GoArch:amd64}

Platform

Linux

Additional context

It could be related to #3446 , but I don't understand how this is related to anchors and aliases.

[Shell32-Natsu]

Possibly related with #3500

[pst]

This issue affects the TektonCD release.yaml. Any Kustomize version after v3.8.10 will fail with the following error:

Error: map[string]interface {}{"apiVersion":"apiextensions.k8s.io/v1", "kind":"CustomResourceDefinition", "metadata":map[string]interface {}{"annotations":map[string]interface {}{"app.kubernetes.io/version":"v0.20.1", "catalog.kubestack.com/heritage":"kubestack.com/catalog/tektoncd", "catalog.kubestack.com/variant":"base"}, "labels":map[string]interface {}{"app.kubernetes.io/component":"controller", "app.kubernetes.io/instance":"default", "app.kubernetes.io/managed-by":"kubestack", "app.kubernetes.io/name":"tektoncd", "app.kubernetes.io/part-of":"tekton-pipelines", "pipeline.tekton.dev/release":"v0.20.1", "version":"v0.20.1"}, "name":"clustertasks.tekton.dev"}, "spec":map[string]interface {}{"conversion":map[string]interface {}{"strategy":"Webhook", "webhook":map[string]interface {}{"clientConfig":map[string]interface {}{"service":map[string]interface {}{"name":"tekton-pipelines-webhook", "namespace":"tekton-pipelines"}}, "conversionReviewVersions":[]interface {}{"v1beta1"}}}, "group":"tekton.dev", "names":map[string]interface {}{"categories":[]interface {}{"tekton", "tekton-pipelines"}, "kind":"ClusterTask", "plural":"clustertasks"}, "preserveUnknownFields":false, "scope":"Cluster", "versions":[]interface {}{map[string]interface {}{"name":"v1alpha1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object", "x-kubernetes-preserve-unknown-fields":true}}, "served":true, "storage":false, "subresources":map[string]interface {}{"status":map[string]interface {}{}}}, map[interface {}]interface {}{"name":"v1beta1", "schema":map[string]interface {}{"openAPIV3Schema":map[string]interface {}{"type":"object", "x-kubernetes-preserve-unknown-fields":true}}, "served":true, "storage":true, "subresources":map[string]interface {}{"status":map[string]interface {}{}}}}}}: json: unsupported type: map[interface {}]interface {}

The full YAML is linked from upstream above. But the root cause seems to be this spec.versions that's part of many of Tekton's CRDs:

spec:
  versions:
    - &version
      name: v1alpha1
      served: true
      storage: false
      schema:
        openAPIV3Schema:
          type: object
          # One can use x-kubernetes-preserve-unknown-fields: true
          # at the root of the schema (and inside any properties, additionalProperties)
          # to get the traditional CRD behaviour that nothing is pruned, despite
          # setting spec.preserveUnknownProperties: false.
          #
          # See https://kubernetes.io/blog/2019/06/20/crd-structural-schema/
          # See issue: https://github.com/knative/serving/issues/912
          x-kubernetes-preserve-unknown-fields: true
      # Opt into the status subresource so metadata.generation
      # starts to increment
      subresources:
        status: {}
    - !!merge <<: *version
      name: v1beta1
      storage: true

[Shell32-Natsu]

Confirmed this is because of the conversion to and from JSON. This should be fixed after #3271 is done.

[metasim]

Anyone figured out workarounds?

[metasim]

In case it's helpful to someone else, here's my (very limited) workaround: a generator plugin (wouldn't work as a transformer).

Assuming problematic.yaml is whatever's confusing kustomize, do the following:

Put this in ${XDG_CONFIG_HOME}/kustomize/plugin/some.org/v1/ExplodeAnchors/ExplodeAnchors

#!/usr/bin/env bash
# Workaround for https://github.com/kubernetes-sigs/kustomize/issues/3614
if ! [ -x "$(command -v yq)" ]; then
  echo "Error: please install yq."
  exit 1
fi

shift # skip config file. exec plugins pass 'argsOneLiner'
SRC="$@"

# Invoke on specified source yml
yq r -X "$SRC"

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

generators:
  - kust-3614-hack.yaml

kust-3614-hack.yaml:

apiVersion: some.org/v1
kind: ExplodeAnchors
metadata:
  name: explode-anchors
argsOneLiner: problematic.yaml

[KnVerey]

I'm consolidating all the issues related to YAML anchors on this one: #3675. Though they don't all fail with the same error message, they seem to all be related to the switch to kyaml. If you're encountering this issue and YAML anchor support is important to you, please chime in over there.

/close
/triage duplicate

[k8s-ci-robot]

@KnVerey: Closing this issue.

In response to this:

I'm consolidating all the issues related to YAML anchors on this one: #3675. Though they don't all fail with the same error message, they seem to all be related to the switch to kyaml. If you're encountering this issue and YAML anchor support is important to you, please chime in over there.

/close
/triage duplicate

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.