YAML Anchor support

[george-angel]

Starting from kustomize/v4.0.0 - there is no way for us to build manifests that contain yaml anchors. Previously we were able to workaround by setting --enable_kyaml=false.

Example:

proximo.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/path: /__/metrics
    prometheus.io/port: "8080"
    prometheus.io/scrape: "true"
  labels:
    app: &app proximo
    uw.systems/mirror: "true"
  name: *app
spec:
  ports:
  - name: ops
    port: 8080
    protocol: TCP
  - name: http
    port: 6868
    protocol: TCP
  selector:
    app: *app

kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - proximo.yaml

Trying to do a build:

$ kustomize build .
Error: considering field 'metadata/name' of object
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/path: /__/metrics
    prometheus.io/port: "8080"
    prometheus.io/scrape: "true"
  labels:
    app: &app proximo
    uw.systems/mirror: "true"
  name: *app
spec:
  ports:
  - name: ops
    port: 8080
    protocol: TCP
  - name: http
    port: 6868
    protocol: TCP
  selector:
    app: *app
: wrong Node Kind for metadata.name expected: ScalarNode was AliasNode: value: {*app}

Thank you.

[eddiezane]

/label anchors

[k8s-ci-robot]

@eddiezane: The label(s) /label anchors cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda

In response to this:

/label anchors

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[eddiezane]

@KnVerey to reply

[KnVerey]

Several issues have been opened about this YAML anchor regression, which affects v4+ universally and v3+ if kyaml is enabled. Although they don't all result in the same error message, it seems to me that they are all related to the switch from apimachinery to kyaml. To that end, I'm going to close the various other issues and consolidate on this one.

If a community member would be interested in diving in to restore YAML anchor support, we would be happy to accept the changes. That said, given the effort involved, it would be helpful to have more information about the purpose of the feature from those who have historically been using it. YAML anchors provide a way to reuse configuration snippets, but in the examples I've seen so far, the same result could be achieved with Kustomize-native mechanisms (patches, commonLabels, etc.). This is even more true now that the extremely powerful ReplacementsTranformer has merged. It would be helpful to have more information as to why and in which cases YAML anchors are being preferred.

If there isn't interest in contributing support, and effective results can be achieve within Kustomize's own mechanisms, we should instead consider simplifying by removing the feature entirely (as some other projects have chosen to do) and providing clear error messages to that effect.

If you use YAML anchors with Kustomize, please comment with more context to help us understand why this feature is important to you.

/triage under-consideration

[levsha]

Anchors are part of YAML specification. Just dropping anchors instead of properly implementing YAML parser will mean that it will be not correct to state that configuration is written in YAML.

[levsha]

@KnVerey can you please show how the example from #3614 can be solved with kustomize-native mechanisms?

[diarmuidie]

In my case I'm using YAML anchors not by choice, but because they are present in the the third party YAML that my customise is overlaying on. Knative is an example of this: https://github.com/knative/serving/blob/8a9bb8e5f2f6959d0de2119498e32102dc2df942/config/core/deployments/webhook.yaml#L100

Because I don't control the code it's not practical to remove the anchors.

[metasim]

This has been a major PITA for us (we used anchor/aliases to DRY in our infrastructure). Our workaround was to create a kustomize generator that looks something like this:

ExplodeAnchors

#!/usr/bin/env bash
# Workaround for https://github.com/kubernetes-sigs/kustomize/issues/3614
if ! [ -x "$(command -v yq)" ]; then
  echo "Error: please install yq."
  exit 1
fi

shift # skip config file. exec plugins pass 'argsOneLiner'
SRC="$@"

# Invoke on specified source yml
yq eval 'explode(.)' "$SRC"

When placed in your kustomize plugin path as kustomize/plugin/astraea.io/v1/ExplodeAnchors/ExplodeAnchors it can be invoked something like this:

kust-3614-hack.yaml

# This should hopefully go away once
# https://github.com/kubernetes-sigs/kustomize/issues/3614 is resolved
apiVersion: astraea.io/v1
kind: ExplodeAnchors
metadata:
  name: explode-anchors
argsOneLiner: <file to explode>.yaml

kustomization.yaml

generators:
  - kust-3614-hack.yaml

ref: #3614

[metasim]

If you use YAML anchors with Kustomize, please comment with more context to help us understand why this feature is important to you.

One use case that comes to mind: you have a pod with several containers, and each container needs to have the same secrets injected in each container but renamed from what they are in the Secrets file (meaning it's involved). Let's say this represents 20 or 30 lines of yaml per container, and if a change is made, it has to be made to each container. A scenario where it's easy to make a mistake, but hard to debug. With anchors/aliases we define it once in the first container and inject it into subsequent containers with a single line.

[pyohannes]

It's important for us, because it effectively breaks a bunch of legacy YAML files that we currently have to maintain.

[Thraganux]

We extensively use anchors in order to avoid copy paste errors. Removing support for this would break most of our deployments and partner teams deployments. Due to how many different services we support, removing these would involve a month of dev time to re-work at the very least.

[ykyr]

We also rely on the yaml anchoring. It helps to avoid a lot of copy&pasting and keep yamls smaller&cleaner

[metasim]

Given that YAML is a standard, it seems fair to at least document the level of compliance supported. That said, anchors and aliases were part of YAML 1.0.

I'm sure this is a naive question, but there exists a full featured Go YAML Parser, supporting anchors and aliases... why doesn't Kustomize use that?

[artyom-p]

Same for us, now we need to update all of the manifests we have....

[metasim]

Is there a process or mechanism short of a PR by which we can bump this in priority, e.g. votes, raise funds to fix it, find a sponsor, other support? Kustomize is all about YAML processing, and standards compliance in its interpretation seems essential for it to have a successful future. Thanks to Kustomize, we've benefitted from keeping our use of Helm to a minimum, enabling us to adjust manifests in a helpfully precise manner. I'm a big fan of Kustomize, but I worry if this issue (and any related to the YAML parser) linger it's going to send us back to non-declarative tools (a step backwards IMHO).

PS:

short of a PR

I say this as I'm not a Go developer.

[metasim]

I hope the above doesn't come across as unappreciative of the team behind this project! I really do think Kustomize is fantastic, and want to see it become the best that it can be, and appreciate all the hard work that's gone into it! :-)

[foobarbecue]

Anchors were the feature that made YAML usable, so I could have 2K line manifests instead of 20K line manifests. And now it breaks without any warning or workaround. Guess we're stuck using kustomize 3 .

As a stopgap, could we at least improve the error message output (mine was json: unsupported type: map[interface {}]interface {}) or just add a note on every error saying FYI kustomize doesn't support YAML anchors? Could have saved me weeks of trying to figure out what went wrong with my build pipeline.

[metasim]

@KnVerey Curious to know if there's been any further discussion on this within the Kustomization team?

[monopole]

I think this is fixable on the kyaml side. There are some complex edges cases here, so will try a few and see which is least troublesome.

[KnVerey]

/triage accepted

[RE-codes]

I am experiencing partial support for anchor/alias. Anchors/aliases are used throughout various teams in our org, and we are interested in using kustomize for a cross squad purpose, so it would be great to have the full yaml spec support to avoid confusion across teams with regard to "how much yaml can we use" and unwinding a lot of preexisting work.

Specifically, I am seeing that anchors for labels and namespace seem to work, but annotations and name are not recognized.

Additionally, the <<: *alias feature does not seem to work in any capacity. It gives an invalid type error map[interface()]interface{} as noted above by foobarbecue.

This is using {Version:kustomize/v4.3.0 GitCommit:cd17338759ef64c14307991fd25d52259697f1fb BuildDate:2021-08-24T19:04:58Z GoOs:darwin GoArch:amd64}.

I can show more details if interested, but given the time and scope of this issue I didn't want to be overly redundant.

Thanks!

[KnVerey]

The fix has been released in version 4.4.0. Please try it out and open a follow-up issue with any problems encountered.

[patricknelson]

FWIW, it looks like the windows version of Kustomize isn't yet available for this release for some reason... it seems to be missing from the more recent builds. I traced it back to PR #4027 (via commit bb02a76).

[patricknelson]

For anyone else using Windows, I found that a custom build of this latest version for Windows works and does support YAML anchors (whereas the latest public release for Windows 4.1.3 at time of writing does not).

Thankfully Go is super easy to use, but since I wasn't very familiar with it (and to save anyone else the trouble, including
future me), just install go 1.16.x, clone this repo down, cd kustomize and go build to get your custom build until the build pipeline for Windows releases gets fixed. 😕

Edit: The discussion relating to windows builds can be found #4028 (proposed fix in #4262).

[patricknelson]

p.s. Just following up: This functionality is now finally available in Windows via 4.4.1 and later thanks to #4262.