Makefile target for consecutive updates #2443
Conversation
kubevirt-bot
added
release-note
Pull Request Test Coverage Report for Build 7288479913
💛 - Coveralls |
| upgrade $INITIAL_VERSION $MID_VERSION $OO_MID_BUNDLE | ||
| upgrade $MID_VERSION $TARGET_VERSION $OO_LAST_BUNDLE |
There was a problem hiding this comment.
I'm not sure about this implementation.
let's say we're doing upgrade from x-->y-->z
It's over kill to run all the test and checks after both x-->y and y-->z, first because these upgrades are tested elsewhere, but also because what will be more interesting, for example, is to start a VM after the x, and validate it after z. In addition, some of the checks are specific for version x to version y upgrade, e.g. the removal of TTO.
I think that the script should:
- check that the installation of x is ok
- start a vm
- upgrade to y
- check that the upgrade was done. Check that the VM is still running. check the operator condition
- upgrade to z
- check all the above (in 4) + run the func tests and all the relevant checks.
There was a problem hiding this comment.
Makes sense. I put in all that within the function because I wasn't sure how much is enough. Does b06a052 seem more appropriate in terms of scope?
|
Kudos, SonarCloud Quality Gate passed!
|
|
hco-e2e-upgrade-operator-sdk-sno-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-operator-sdk-aws, ci/prow/hco-e2e-upgrade-operator-sdk-sno-azure, ci/prow/okd-hco-e2e-operator-sdk-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-prev-operator-sdk-sno-azure lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-operator-sdk-sno-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-operator-sdk-azure lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-operator-sdk-aws In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with /lifecycle stale |
kubevirt-bot
added
the
lifecycle/stale
|
Stale issues rot after 30d of inactivity. If this issue is safe to close now please do so with /lifecycle rotten |
kubevirt-bot
added
lifecycle/rotten
|
/remove-lifecycle rotten |
kubevirt-bot
removed
the
lifecycle/rotten
Signed-off-by: Dharmit Shah <shahdharmit@gmail.com>
Signed-off-by: Dharmit Shah <shahdharmit@gmail.com>
Signed-off-by: Dharmit Shah <shahdharmit@gmail.com>
dharmit
force-pushed
the
fix-cnv-20547
branch
from
b06a052
to
922d0db
Compare
|
hco-e2e-upgrade-prev-operator-sdk-sno-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-operator-sdk-sno-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
okd-hco-e2e-operator-sdk-gcp lane succeeded. |
dharmit
force-pushed
the
fix-cnv-20547
branch
from
51db5fd
to
6429e7f
Compare
|
| # dependent images, new CRDs...). | ||
| # - the script then performs various validations against the upgraded version. | ||
|
|
||
| MAX_STEPS=$(( $(grep -c "Msg " "$0") - 2)) # subtract self line and the function name |
There was a problem hiding this comment.
We need to re-think about it. This is not true, as we're calling the Msg function for the upgrade function, so the number here is not right.
Let's fix this in a future PR.
| sleep 5 | ||
| INITIAL_BOOTTIME=$(check_uptime 10 60) | ||
|
|
||
| Msg "HCO deployOVS annotation and OVS state in CNAO CR before the upgrade" |
There was a problem hiding this comment.
This is common to all the Msg calls in this function:
Let's add some text to distinguish between the two calls to this function. So we could find from the message what upgrade this log is related to (e.g. [va.b.c => vx.y.z] or any other text that will mark where we are in the process).
Let's do it in a future PR.
| PREVIOUS_OVS_STATE=$(${CMD} get networkaddonsconfigs cluster -o jsonpath='{.spec.ovs}') | ||
|
|
||
| # Before starting the upgrade, make sure the CSV is installed properly. | ||
| Msg "Read the CSV to make sure the deployment is done" |
| HCO_SUBSCRIPTION=$(${CMD} get subscription -n ${HCO_NAMESPACE} -o name -l operators.coreos.com/community-kubevirt-hyperconverged.${HCO_NAMESPACE}=) | ||
| OLD_INSTALL_PLAN=$(${CMD} -n "${HCO_NAMESPACE}" get "${HCO_SUBSCRIPTION}" -o jsonpath='{.status.installplan.name}') | ||
|
|
||
| Msg "Perform the upgrade, using operator-sdk" |
| ## Verify the subscription has changed to the new version | ||
| # currentCSV: kubevirt-hyperconverged-operator.v100.0.0 | ||
| # installedCSV: kubevirt-hyperconverged-operator.v100.0.0 | ||
| Msg "Verify the subscription's currentCSV and installedCSV have moved to the new version" |
| ${CMD} wait deployment ${HCO_DEPLOYMENT_NAME} --for condition=Available -n ${HCO_NAMESPACE} --timeout="1200s" | ||
| ${CMD} wait deployment ${HCO_WH_DEPLOYMENT_NAME} --for condition=Available -n ${HCO_NAMESPACE} --timeout="1200s" | ||
|
|
||
| Msg "operator conditions during upgrade" |
| ./hack/retry.sh 30 60 "${CMD} get ${HCO_SUBSCRIPTION} -n ${HCO_NAMESPACE} -o yaml | grep currentCSV | grep v${T_VERSION}" | ||
| ./hack/retry.sh 2 30 "${CMD} get ${HCO_SUBSCRIPTION} -n ${HCO_NAMESPACE} -o yaml | grep installedCSV | grep v${T_VERSION}" | ||
|
|
||
| Msg "Verify the hyperconverged-cluster deployment is using the new image" |
| SEARCH_PHRASE="${OPENSHIFT_BUILD_NAMESPACE}/pipeline" | ||
| ./hack/retry.sh 60 30 "${CMD} get -n ${HCO_NAMESPACE} deployment ${HCO_DEPLOYMENT_NAME} -o jsonpath=\"{ .spec.template.spec.containers[0].image }\" | grep ${SEARCH_PHRASE}" | ||
|
|
||
| Msg "Wait that cluster is operational after upgrade" |
| timeout 20m bash -c 'export CMD="${CMD}";exec ./hack/check-state.sh' | ||
|
|
||
| # Make sure the CSV is installed properly. | ||
| Msg "Read the CSV to make sure the deployment is done" |
| ./hack/retry.sh 30 10 "${CMD} get ${CSV} -n ${HCO_NAMESPACE} -o jsonpath='{ .spec.version }' | grep ${T_VERSION}" | ||
|
|
||
| echo "----- Pod after upgrade" | ||
| Msg "Verify that the hyperconverged-cluster Pod is using the new image" |
| Msg "Verify that the hyperconverged-cluster Pod is using the new image" | ||
| ./hack/retry.sh 10 30 "CMD=${CMD} HCO_NAMESPACE=${HCO_NAMESPACE} ./hack/check_pod_upgrade.sh" | ||
|
|
||
| Msg "Verify new operator version reported after the upgrade" |
| Msg "Verify new operator version reported after the upgrade" | ||
| ./hack/retry.sh 15 30 "CMD=${CMD} HCO_RESOURCE_NAME=${HCO_RESOURCE_NAME} HCO_NAMESPACE=${HCO_NAMESPACE} TARGET_VERSION=${T_VERSION} hack/check_hco_version.sh" | ||
|
|
||
| Msg "Ensure that HCO got upgraded" |
| fi | ||
| done | ||
|
|
||
| Msg "operator conditions after upgrade" |
|
|
||
| OUTPUT_DIR=${OUTPUT_DIR} dump_sccs_after | ||
|
|
||
| Msg "make sure that the VM is still running, after the upgrade" |
|
/retest |
|
/unhold |
kubevirt-bot
removed
the
do-not-merge/hold
|
hco-e2e-upgrade-prev-operator-sdk-sno-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-kv-smoke-azure, ci/prow/hco-e2e-upgrade-prev-operator-sdk-sno-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
hco-e2e-upgrade-prev-operator-sdk-sno-aws lane succeeded. |
|
@hco-bot: Overrode contexts on behalf of hco-bot: ci/prow/hco-e2e-upgrade-prev-operator-sdk-sno-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/override-bot |
|
hco-e2e-operator-sdk-sno-aws lane passed hco-e2e-upgrade-operator-sdk-aws lane passed /lgtm |
|
@nunnatsa: Overrode contexts on behalf of nunnatsa: ci/prow/hco-e2e-upgrade-operator-sdk-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/override ci/prow/hco-e2e-operator-sdk-sno-azure |
|
@nunnatsa: Overrode contexts on behalf of nunnatsa: ci/prow/hco-e2e-operator-sdk-sno-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@dharmit: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
hco-e2e-kv-smoke-gcp lane passed |
|
@nunnatsa: Overrode contexts on behalf of nunnatsa: ci/prow/hco-e2e-kv-smoke-azure In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
What this PR does / why we need it:
Adds a Makefile target for consecutive operator updates, which enables HCO for OLM's fail forward upgrade mechanism.
Reviewer Checklist
Jira Ticket:
Release note: