-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WINC-950: WMCO bundle upgrade testing #39585
Conversation
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
2 similar comments
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
2 similar comments
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
1 similar comment
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
1 similar comment
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
Issues in openshift/release go stale after 30d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle stale |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
pod-security.kubernetes.io/enforce=privileged | ||
# Deploy the bundle which will fail to come up due to it using the operator image via a tag that is not able to | ||
# be made public. The csv needs to be patched to point the image via the tag that is pullable by the job. | ||
operator-sdk run bundle --timeout=10m --security-context-config restricted -n $WMCO_NS "$PREVIOUS_BUNDLE" \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will this hit the 10 min timeout every time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't you patch the CSV first?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The timeout will not be hit as there is an issue in operator-sdk which will not respect the timeout parameter correctly. operator-framework/operator-sdk#6440
operator-sdk run bundle --timeout=10m --security-context-config restricted -n $WMCO_NS "$OLD_BUNDLE" \ | ||
|| oc get csv -n $WMCO_NS |awk {'print $1'} | tail -n1 | xargs oc patch csv -n $WMCO_NS --type='json' -p="[{\"op\": \"replace\", \"path\": \"/spec/install/spec/deployments/0/spec/template/spec/containers/0/image\", \"value\": \"$OLD_OPERATOR\"}]" | ||
sleep 10 | ||
oc delete deployment -n $WMCO_NS windows-machine-config-operator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I though the deployment automatically gets deleted + re-created by olm after patching the csv? Or is this done to speed up the process
pod-security.kubernetes.io/enforce=privileged | ||
# Deploy the bundle which will fail to come up due to it using the operator image via a tag that is not able to | ||
# be made public. The csv needs to be patched to point the image via the tag that is pullable by the job. | ||
operator-sdk run bundle --timeout=10m --security-context-config restricted -n $WMCO_NS "$PREVIOUS_BUNDLE" \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't you patch the CSV first?
sleep 10 | ||
# Delete the deployment which will then be recreated by the subscription controller with the correct image | ||
oc delete deployment -n $WMCO_NS windows-machine-config-operator | ||
sleep 5m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need the sleep given the wait below?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I dont remember and don't think its actually required. Removing and hoping it works in the pj-rehearse
# be made public. The csv needs to be patched to point the image via the tag that is pullable by the job. | ||
operator-sdk run bundle --timeout=10m --security-context-config restricted -n $WMCO_NS "$PREVIOUS_BUNDLE" \ | ||
|| oc get csv -n $WMCO_NS |awk {'print $1'} | tail -n1 | xargs oc patch csv -n $WMCO_NS --type='json' \ | ||
-p="[{\"op\": \"replace\", \"path\": \"/spec/install/spec/deployments/0/spec/template/spec/containers/0/image\", \"value\": \"$PREVIOUS_OPERATOR\"}]" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where does $PREVIOUS_BUNDLE
and $PREVIOUS_OPERATOR
get set?
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
export WMCO_NS=openshift-windows-machine-config-operator | ||
operator-sdk run bundle-upgrade --timeout=10m --security-context-config restricted -n $WMCO_NS "$OO_BUNDLE" | ||
sleep 5m | ||
oc wait --timeout=5m --for condition=Available -n openshift-windows-machine-config-operator deployment windows-machine-config-operator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oc wait --timeout=5m --for condition=Available -n openshift-windows-machine-config-operator deployment windows-machine-config-operator | |
oc wait --timeout=5m --for condition=Available -n $WMCO_NS deployment windows-machine-config-operator |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
Adds a job that upgrades from the WMCO released for the previous OpenShift minor version, to the WMCO being tested. This is being marked as not required. Once the job is fully functional, after changes to the WMCO repo, this job will be made required and the already existing upgrade job will be removed. This job is following this documented workflow: https://docs.ci.openshift.org/docs/how-tos/testing-operator-sdk-operators/#operator-upgrade-testing Because of undocumented limitations of the ci-operator the "old-bundle" needs to be patched to use the proper image. The "old bundle" points to an operator image via a tag that is not able to be made public. This is why both old-bundle and old-wmco need to be imported through the base-images section of the config. The job file was generated from the config file through `make update`. This job is split into 4 test steps 1) Install WMCO built for the previous OCP minor version. 2) Run initial tests against that WMCO using the `upgrade-test-setup` make target, creating Windows Nodes. 3) Upgrade WMCO to the version which is built from the code this job is testing, this causes those Windows nodes to be upgraded from the previous version to the current version. 4) Run tests using the `upgrade-test` make target, testing that the Node upgrade was successful.
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
/pj-rehearse pull-ci-openshift-windows-machine-config-operator-master-vsphere-e2e-upgrade |
/pj-rehearse ack |
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aravindhp, saifshaikh48 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@sebsoto: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
@sebsoto: Updated the following 2 configmaps:
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Adds a job that upgrades from the WMCO released for the previous
OpenShift minor version, to the WMCO being tested. This is being marked
as not required. Once the job is fully functional, after changes to the
WMCO repo, this job will be made required and the already existing
upgrade job will be removed.
This job is split into 4 test steps
upgrade-test-setup
make target, creating Windows Nodes.testing, this causes those Windows nodes to be upgraded from the
previous version to the current version.
upgrade-test
make target, testing that the Nodeupgrade was successful.