Breaking change after new engine (4.35.2 vs 4.40.3)

[lucasfcnunes]

Describe the bug

Breaking change after new engine (4.35.2 vs 4.40.3).
Same eval, different results: new unexpected null values and key merging.

Version of yq: 4.40.3
Operating system: linux
Installed via: docker and binary

Input Yaml

helmfile.yaml.gotmpl:

repositories:
  - name: repo-1
    url: https://repo-1.github.io/helm-charts
  - name: repo-2
    url: https://repo-2.github.io/helm-charts
templates:
  default-release: &default-release
    namespace: k8s_namespace
environments:
  default:
    values:
      - kubeContext: MY_CLUSTER
        releases:
          chart-1:
            version: v1
            <<: *default-release
          chart-2:
            version: v2
            <<: *default-release
---
templates:
  default: &default
    kubeContext: '{{ .Values.kubeContext }}'
    version: '{{`{{ .Values | get "releases" | get .Release.Name | get "version" }}`}}'
    namespace: '{{`{{ .Values | get "releases" | get .Release.Name | get "namespace" }}`}}'
releases:
  - chart: repo-1/chart-1
    name: chart-1
    <<: *default
  - chart: repo-2/chart-2
    name: chart-2
    <<: *default

Command
The command you ran:

yq eval '.environments[].values[].releases[] | {"name": key, "env": (parent | parent | parent | parent | key), "kubeContext": (parent | parent | .kubeContext), "namespace": .namespace} | [.]' -o json helmfile.yaml.gotmpl

Actual behavior

[
  {
    "name": "chart-1",
    "env": null,
    "default": null,
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[
  {
    "name": "chart-2",
    "env": null,
    "default": null,
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[]

Expected behavior

[
  {
    "name": "chart-1",
    "env": "default",
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[
  {
    "name": "chart-2",
    "env": "default",
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[]

Additional context
How to reproduce:

yq_breaking_change.sh:

#! /bin/bash

set -e

echo -e '>>> sample yaml file'

FILE=helmfile.yaml.gotmpl
cat << 'EOF' | tee $FILE
repositories:
  - name: repo-1
    url: https://repo-1.github.io/helm-charts
  - name: repo-2
    url: https://repo-2.github.io/helm-charts
templates:
  default-release: &default-release
    namespace: k8s_namespace
environments:
  default:
    values:
      - kubeContext: MY_CLUSTER
        releases:
          chart-1:
            version: v1
            <<: *default-release
          chart-2:
            version: v2
            <<: *default-release
---
templates:
  default: &default
    kubeContext: '{{ .Values.kubeContext }}'
    version: '{{`{{ .Values | get "releases" | get .Release.Name | get "version" }}`}}'
    namespace: '{{`{{ .Values | get "releases" | get .Release.Name | get "namespace" }}`}}'
releases:
  - chart: repo-1/chart-1
    name: chart-1
    <<: *default
  - chart: repo-2/chart-2
    name: chart-2
    <<: *default
EOF

YQ_ARGS=$(
cat <<EOF
eval '.environments[].values[].releases[] | {"name": key, "env": (parent | parent | parent | parent | key), "kubeContext": (parent | parent | .kubeContext), "namespace": .namespace} | [.]' -o json $FILE
EOF
)

echo -e '\n>>> yq args'
echo $YQ_ARGS

DOCKER_CMD="docker run --rm -v "${PWD}":/workdir/"

echo -e '\n>>> old output 4.35.2'
bash -c "$DOCKER_CMD mikefarah/yq:4.35.2 $YQ_ARGS"

echo -e '\n>>> new output 4.40.3'
bash -c "$DOCKER_CMD mikefarah/yq:4.40.3 $YQ_ARGS"

Output:

$ ./yq_breaking_change.sh
>>> sample yaml file
repositories:
  - name: repo-1
    url: https://repo-1.github.io/helm-charts
  - name: repo-2
    url: https://repo-2.github.io/helm-charts
templates:
  default-release: &default-release
    namespace: k8s_namespace
environments:
  default:
    values:
      - kubeContext: MY_CLUSTER
        releases:
          chart-1:
            version: v1
            <<: *default-release
          chart-2:
            version: v2
            <<: *default-release
---
templates:
  default: &default
    kubeContext: '{{ .Values.kubeContext }}'
    version: '{{`{{ .Values | get "releases" | get .Release.Name | get "version" }}`}}'
    namespace: '{{`{{ .Values | get "releases" | get .Release.Name | get "namespace" }}`}}'
releases:
  - chart: repo-1/chart-1
    name: chart-1
    <<: *default
  - chart: repo-2/chart-2
    name: chart-2
    <<: *default

>>> yq args
eval '.environments[].values[].releases[] | {"name": key, "env": (parent | parent | parent | parent | key), "kubeContext": (parent | parent | .kubeContext), "namespace": .namespace} | [.]' -o json helmfile.yaml.gotmpl

>>> old output 4.35.2
[
  {
    "name": "chart-1",
    "env": "default",
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[
  {
    "name": "chart-2",
    "env": "default",
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[]

>>> new output 4.40.3
[
  {
    "name": "chart-1",
    "env": null,
    "default": null,
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[
  {
    "name": "chart-2",
    "env": null,
    "default": null,
    "kubeContext": "MY_CLUSTER",
    "namespace": "k8s_namespace"
  }
]
[]

[mikefarah]

Thanks for raising - looking into this, it's is actually caused by the same issue as #1886. Will fix in the next release shortly.

[mikefarah]

Fixed in 4.40.4