-
Notifications
You must be signed in to change notification settings - Fork 504
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add doc.go to each generated mainfest dir #1814
add doc.go to each generated mainfest dir #1814
Conversation
Hello @deads2k! Some important instructions when contributing to openshift/api: |
transitive lgtm from #1814 |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: deads2k The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
@deads2k: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
8ebde95
into
openshift:master
[ART PR BUILD NOTIFIER] This PR has been included in build ose-cluster-config-api-container-v4.16.0-202403180813.p0.g8ebde95.assembly.stream.el9 for distgit ose-cluster-config-api. |
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ go get -u github.com/openshift/api@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ go get -u github.com/openshift/api@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ go get -u github.com/openshift/api@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821
I followed the new docs from [2] to create pkg/dependencymagnet Skip render for DevPreviewNoUpgrade The Default ClusterVersion CRD is the one we want to render during bootstrap-render [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 Signed-off-by: Eran Cohen <eranco@redhat.com> Signed-off-by: Eran Cohen <eranco@redhat.com>
I followed the new docs from [2] to create pkg/dependencymagnet Skip render for DevPreviewNoUpgrade The Default ClusterVersion CRD is the one we want to render during bootstrap-render Update ./hack/test-prerequisites.go to pick up clusterversion crds from the new path config/v1/zz_generated.crd-manifests [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 Signed-off-by: Eran Cohen <eranco@redhat.com>
I followed the new docs from [2] to create pkg/dependencymagnet Skip render for DevPreviewNoUpgrade The Default ClusterVersion CRD is the one we want to render during bootstrap-render Update ./hack/test-prerequisites.go to pick up clusterversion crds from the new path config/v1/zz_generated.crd-manifests [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 Signed-off-by: Eran Cohen <eranco@redhat.com>
go get -u github.com/openshift/api@master go mod tidy go mod vendor git add -A go.* vendor Get featuregate definitions from api/features instead of api/config/v1 Updted library-go to latest This is required due to the FeatureSets changes in openshift/api Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet Skip render for DevPreviewNoUpgrade The Default ClusterVersion CRD is the one we want to render during bootstrap-render Update ./hack/test-prerequisites.go to pick up clusterversion crds from the new path config/v1/zz_generated.crd-manifests [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 Signed-off-by: Eran Cohen <eranco@redhat.com>
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. And I want [5]. [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821 [5]: openshift/library-go#1759
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ git checkout a90cec9 -- pkg/payload/render.go # Eran Cohen's DevPreviewNoUpgrade exclusion $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. And I want [5]. * The DevPreviewNoUpgrade exclusion avoids bootstrap failures like [6]: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -xOz log-bundle-20240709194524/bootstrap/journals/bootkube.log | grep 'are not consistent so results would be unpredictable depending on apply order' | tail -n4 Jul 09 19:45:15 ip-10-0-155-191 kube-apiserver-render[74584]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:15 ip-10-0-155-191 bootkube.sh[74561]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 kube-apiserver-render[75368]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 bootkube.sh[75345]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values as the kube-apiserver-render command worries about the CVO having rendered two ClusterVersion CRDs: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -tvz | grep 'cvo-bootstrap/.*cluster-version-operator.*crd' -rw-r--r-- core/core 42129 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml -rw-r--r-- core/core 39010 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml -rw-r--r-- core/core 7628 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusteroperators.crd.yaml [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821 [5]: openshift/library-go#1759 [6]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808
Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@master github.com/openshift/library-go@master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ sed -i 's/\(clusterversion\|clusteroperator\)/\1s/' hack/test-prerequisites.go $ git checkout a90cec9 -- pkg/payload/render.go # Eran Cohen's DevPreviewNoUpgrade exclusion $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. And I want [5]. * The DevPreviewNoUpgrade exclusion avoids bootstrap failures like [6]: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -xOz log-bundle-20240709194524/bootstrap/journals/bootkube.log | grep 'are not consistent so results would be unpredictable depending on apply order' | tail -n4 Jul 09 19:45:15 ip-10-0-155-191 kube-apiserver-render[74584]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:15 ip-10-0-155-191 bootkube.sh[74561]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 kube-apiserver-render[75368]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 bootkube.sh[75345]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values as the kube-apiserver-render command worries about the CVO having rendered two ClusterVersion CRDs: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -tvz | grep 'cvo-bootstrap/.*cluster-version-operator.*crd' -rw-r--r-- core/core 42129 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml -rw-r--r-- core/core 39010 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml -rw-r--r-- core/core 7628 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusteroperators.crd.yaml [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821 [5]: openshift/library-go#1759 [6]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808
Like e9762c6 (vendor: Update openshift/api to pick up zz_generated.crd-manifests, 2024-04-30, openshift#1045), except I pulled release-4.16 branches. Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@190fec8c3f09de1290c0dcaa25556b4515d65fc6 # different from e9762c6's @master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ sed -i 's/\(clusterversion\|clusteroperator\)/\1s/' hack/test-prerequisites.go $ git checkout a90cec9 -- pkg/payload/render.go # Eran Cohen's DevPreviewNoUpgrade exclusion $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet pkg/featuregates go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. * And I want [5], but for some reason: $ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@release-4.16 was getting me the older v0.0.0-20240517135010-e93e442c2b18. I'm not clear on why. But requesting the library-go commit I wanted by commit-hash instead of by branch-name got me the CustomNoUpgrade fix. * The DevPreviewNoUpgrade exclusion avoids bootstrap failures like [6]: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -xOz log-bundle-20240709194524/bootstrap/journals/bootkube.log | grep 'are not consistent so results would be unpredictable depending on apply order' | tail -n4 Jul 09 19:45:15 ip-10-0-155-191 kube-apiserver-render[74584]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:15 ip-10-0-155-191 bootkube.sh[74561]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 kube-apiserver-render[75368]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_ 01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 bootkube.sh[75345]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values as the kube-apiserver-render command worries about the CVO having rendered two ClusterVersion CRDs: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpre view/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -tvz | grep 'cvo-bootstrap/.*cluster-version-operator.*crd' -rw-r--r-- core/core 42129 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml -rw-r--r-- core/core 39010 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml -rw-r--r-- core/core 7628 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusteroperators.crd.yaml [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821 [5]: openshift/library-go#1759 Backported to 4.16 in openshift/library-go#1762 [6]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808
Like e9762c6 (vendor: Update openshift/api to pick up zz_generated.crd-manifests, 2024-04-30, openshift#1045), except I pulled release-4.16 branches. Catching up with the YAML getting shifted into a subdirectory [1]. I followed the new docs from [2] to create pkg/dependencymagnet, and then: $ emacs pkg/featuregates/featuregates.go pkg/featuregates/featurechangestopper_test.go # configv1.FeatureGateUpgradeStatus -> features.FeatureGateUpgradeStatus $ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@190fec8c3f09de1290c0dcaa25556b4515d65fc6 # different from e9762c6's @master $ go mod tidy $ go mod vendor $ sed -i 's|config/v1/0000_00_cluster-version-operator|config/v1/zz_generated.crd-manifests/0000_00_cluster-version-operator|g' $(git grep -l 0000_00_cluster-version-operator) $ sed -i 's/\(clusterversion\|clusteroperator\)/\1s/' hack/test-prerequisites.go $ git checkout a90cec9 -- pkg/payload/render.go # Eran Cohen's DevPreviewNoUpgrade exclusion $ git add -A Dockerfile* hack/test-prerequisites.go pkg/dependencymagnet pkg/featuregates go.* vendor all using: $ go version go version go1.22.2 linux/amd64 A few points on the invocation: * I usually use 'go get -u ...', e.g. a98232f (vendor: Bump openshift/api to pick up signatureStores, 2023-12-04, openshift#994), but in this case, that would pull in k8s.io/api@v0.30.0, and we want to stick with 0.29 until our release-4.16 branch stops following this development branch. * I'm explicitly bumping library-go, because if I don't, builds fail on [3]: Building github.com/openshift/cluster-version-operator (v1.0.0-1203-g258de3f7-dirty) # github.com/openshift/library-go/pkg/manifest vendor/github.com/openshift/library-go/pkg/manifest/manifest.go:32:35: undefined: configv1.FeatureSets # github.com/openshift/cluster-version-operator/pkg/featuregates pkg/featuregates/featuregates.go:92:32: undefined: configv1.FeatureGateUpgradeStatus pkg/featuregates/featuregates.go:97:33: undefined: configv1.FeatureGateUpgradeStatus If I had been able to use -u, library-go would have automatically floated to the tip commit anyway. But either way, I still needed to make manual changes to keep up with [4] moving to the new openshift/api/features package. * And I want [5], but for some reason: $ go get github.com/openshift/api@release-4.16 github.com/openshift/library-go@release-4.16 was getting me the older v0.0.0-20240517135010-e93e442c2b18. I'm not clear on why. But requesting the library-go commit I wanted by commit-hash instead of by branch-name got me the CustomNoUpgrade fix. * The DevPreviewNoUpgrade exclusion avoids bootstrap failures like [6]: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpreview/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -xOz log-bundle-20240709194524/bootstrap/journals/bootkube.log | grep 'are not consistent so results would be unpredictable depending on apply order' | tail -n4 Jul 09 19:45:15 ip-10-0-155-191 kube-apiserver-render[74584]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:15 ip-10-0-155-191 bootkube.sh[74561]: F0709 19:45:15.447834 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 kube-apiserver-render[75368]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_ 01_clusterversions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values Jul 09 19:45:27 ip-10-0-155-191 bootkube.sh[75345]: F0709 19:45:27.669093 1 render.go:75] --rendered-manifest-files, are not consistent so results would be unpredictable depending on apply order: "/assets/manifests/0000_00_cluster-version-operator_01_cluster versions-DevPreviewNoUpgrade.crd.yaml" and "/assets/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml" both set CustomResourceDefinition.apiextensions.k8s.io/clusterversions.config.openshift.io in ns/, but have different values as the kube-apiserver-render command worries about the CVO having rendered two ClusterVersion CRDs: $ curl -s https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808/artifacts/e2e-aws-ovn-techpre view/ipi-install-install/artifacts/log-bundle-20240709194524.tar | tar -tvz | grep 'cvo-bootstrap/.*cluster-version-operator.*crd' -rw-r--r-- core/core 42129 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-DevPreviewNoUpgrade.crd.yaml -rw-r--r-- core/core 39010 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusterversions-Default.crd.yaml -rw-r--r-- core/core 7628 2024-07-09 12:45 log-bundle-20240709194524/rendered-assets/openshift/cvo-bootstrap/manifests/0000_00_cluster-version-operator_01_clusteroperators.crd.yaml [1]: openshift/api#1814 [2]: openshift/api@06baaa4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R211 [3]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-images/1785396933666279424#1:build-log.txt%3A74-79 [4]: openshift/api#1821 [5]: openshift/library-go#1759 Backported to 4.16 in openshift/library-go#1762 [6]: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cluster-version-operator/1045/pull-ci-openshift-cluster-version-operator-master-e2e-aws-ovn-techpreview/1810753819311607808
fixes problem for @p0lyn0mial