Like e9762c6 (vendor: Update openshift/api to pick up
zz_generated.crd-manifests, 2024-04-30, openshift#1045), except I pulled
release-4.16 branches.
Catching up with the YAML getting shifted into a subdirectory [1]. I
followed the new docs from [2] to create pkg/dependencymagnet, and
then:
$ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus
$ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@190fec8c3f09de1290c0dcaa25556b4515d65fc6 # different from e9762c6's @master
$ go mod tidy
$ go mod vendor
$ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator)
$ sed -i 's/\(clusterversion\|clusteroperator\)/\1s/' hack/test-prerequisites.go
$ git checkout a90cec9 -- pkg/payload/render.go # Eran Cohen's DevPreviewNoUpgrade exclusion
$ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet pkg/featuregates go.* vendor
all using:
$ go version
go version go1.22.2 linux/amd64
A few points on the invocation:
* I usually use 'go get -u ...', e.g. a98232f (vendor: Bump
openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in
this case, that would pull in k8s.io/api@v0.30.0, and we want to
stick with 0.29 until our release-4.16 branch stops following this
development branch.
* I'm explicitly bumping library-go, because if I don't, builds fail on [3]:
Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty)
# github.com/openshift/library-go/pkg/manifest
vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets
# github.com/openshift/cluster-version-operator/pkg/featuregates
pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus
pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus
If I had been able to use -u, library-go would have automatically
floated to the tip commit anyway. But either way, I still needed to
make manual changes to keep up with [4] moving to the new
openshift/api/features package.
* And I want [5], but for some reason:
$ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@release-4.16
was getting me the older v0.0.0-20240517135010-e93e442c2b18. I'm
not clear on why. But requesting the library-go commit I wanted by
commit-hash instead of by branch-name got me the CustomNoUpgrade
fix.
* The DevPreviewNoUpgrade exclusion avoids bootstrap failures like [6]:
$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -xOz log-bundle-20240709194524/bootstrap/journals/bootkube.log | grep 'are not consistent so results would be unpredictable depending on apply order' | tail -n4
Jul 09 19:45:15 ip-10-0-155-191 kube-apiserver-render[74584]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values
Jul 09 19:45:15 ip-10-0-155-191 bootkube.sh[74561]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster
versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values
Jul 09 19:45:27 ip-10-0-155-191 kube-apiserver-render[75368]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_
01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values
Jul 09 19:45:27 ip-10-0-155-191 bootkube.sh[75345]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster
versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values
as the kube-apiserver-render command worries about the CVO having
rendered two ClusterVersion CRDs:
$ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpre
view/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -tvz | grep 'cvo-bootstrap/.*cluster-version-operator.*crd'
-rw-r--r-- core/core 42129 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml
-rw-r--r-- core/core 39010 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml
-rw-r--r-- core/core 7628 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusteroperators.crd.yaml
[1]: openshift/api#1814
[2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211
[3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79
[4]: openshift/api#1821
[5]: openshift/library-go#1759
Backported to 4.16 in openshift/library-go#1762
[6]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808