Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e-upgrades: add basic tests with new rpm-ostree version #4784

Closed
wants to merge 1 commit into from

Conversation

HuijingHei
Copy link
Member

Test new rpm-ostree with kernel override, initramfs args, initramfs-etc, layering, overrides.
See #4776

ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
ci/prow/kola/utils.sh Fixed Show fixed Hide fixed
@HuijingHei HuijingHei force-pushed the upgrade-test branch 2 times, most recently from 7e8f254 to e3b0be9 Compare January 19, 2024 07:49
Test new rpm-ostree with kernel override, initramfs args,
initramfs-etc, layering, overrides.
See coreos#4776
@HuijingHei
Copy link
Member Author

/retest

Copy link

openshift-ci bot commented Jan 19, 2024

@HuijingHei: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/fcos-e2e de79076 link true /test fcos-e2e
ci/prow/kola-upgrade de79076 link true /test kola-upgrade

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@HuijingHei
Copy link
Member Author

Confused about the error error waiting for pod: Get "https://api-int.build02.gcp.ci.openshift.org:6443/api/v1/namespaces/ci-op-m1kkigtc/pods/kola-upgrade-kola-upgrades?timeout=1m0s": tls: failed to verify certificate: x509: certificate signed by unknown authority, does this mean can not reach https://koji.fedoraproject.org/koji/buildinfo?buildID=2302642 ?

Copy link
Member

@jlebon jlebon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

Three things from my side:

  1. IIUC, the first boot is already working from the new rpm-ostree (via --oscontainer passed to kola), but I think what we want is to create state from a node on the old rpm-ostree and then update on the new rpm-ostree.
  2. Confusingly, IIUC the second boot is actually working from the old rpm-ostree, which is an interesting test (it verifies that state created by a new rpm-ostree does not break an old one). But a more natural test is upgrading to another update with the new rpm-ostree so that you can verify that new features are working correctly.
  3. Totally OK if you want to pursue this via the Prow harness. I personally think making this part of the other kola tests covered by CoreOS CI would make it more accessible and easier to maintain.

To fix 1, you'd probably want in the first boot to rollback to e.g. what's in testing-devel right now before creating state. To fix 2, you can do it the container way (export current commit to a container image, then create a new container image with some arbitrary change in it -- see e.g. this test) or the OSTree way (create a synthetic commit and rebase onto it -- see e.g. this test)

@HuijingHei
Copy link
Member Author

  1. IIUC, the first boot is already working from the new rpm-ostree (via --oscontainer passed to kola), but I think what we want is to create state from a node on the old rpm-ostree and then update on the new rpm-ostree.

Yes, this is to add some basic tests with new rpm-ostree (via --oscontainer) in the old build fcos-stable, then update using new rpm-ostree. But seems there is problem with https, should consider about your option 3 about kolainst.

  1. Confusingly, IIUC the second boot is actually working from the old rpm-ostree, which is an interesting test (it verifies that state created by a new rpm-ostree does not break an old one). But a more natural test is upgrading to another update with the new rpm-ostree so that you can verify that new features are working correctly.
  2. Totally OK if you want to pursue this via the Prow harness. I personally think making this part of the other kola tests covered by CoreOS CI would make it more accessible and easier to maintain.

To fix 1, you'd probably want in the first boot to rollback to e.g. what's in testing-devel right now before creating state. To fix 2, you can do it the container way (export current commit to a container image, then create a new container image with some arbitrary change in it -- see e.g. this test) or the OSTree way (create a synthetic commit and rebase onto it -- see e.g. this test)

Will look at it, thanks a lot @jlebon for your quick reply!

Maybe we can have 2 tests about the upgrades, does this make sense?

  • According to your suggestion: Do some basic tests (like kernel override, initramfs args, initramfs-etc, layering, overrides) in the old builds like fcos-stable, then upgrade new rpm-ostree, then upgrade to another update.
  • Upgrade rpm-ostree in old builds like fcos-stable, then do some basic tests (see above) using the new rpm-ostree, then upgrade to another update.

@jlebon
Copy link
Member

jlebon commented Jan 23, 2024

Maybe we can have 2 tests about the upgrades, does this make sense?

* According to your [suggestion](https://github.com/coreos/rpm-ostree/issues/4776): Do some basic tests (like kernel override, initramfs args, initramfs-etc, layering, overrides) in the old builds like fcos-stable, then upgrade new rpm-ostree, then upgrade to another update.

* Upgrade rpm-ostree in old builds like fcos-stable, then do some basic tests (see above) using the new rpm-ostree, then upgrade to another update.

I.e. in one test we do the mutations on top of old rpm-ostree and then upgrade, and in the other test, we upgrade first and then do the mutations? The first one sounds more interesting to me. What do you have in mind on what the second one is intended to catch?

@HuijingHei
Copy link
Member Author

Maybe we can have 2 tests about the upgrades, does this make sense?

* According to your [suggestion](https://github.com/coreos/rpm-ostree/issues/4776): Do some basic tests (like kernel override, initramfs args, initramfs-etc, layering, overrides) in the old builds like fcos-stable, then upgrade new rpm-ostree, then upgrade to another update.

* Upgrade rpm-ostree in old builds like fcos-stable, then do some basic tests (see above) using the new rpm-ostree, then upgrade to another update.

I.e. in one test we do the mutations on top of old rpm-ostree and then upgrade, and in the other test, we upgrade first and then do the mutations? The first one sounds more interesting to me. What do you have in mind on what the second one is intended to catch?

The second is to avoid regression of fedora-silverblue/issue-tracker#523, which upgrade rpm-ostree on old build, but hit issue when remove package.

@HuijingHei
Copy link
Member Author

Close this as impl this in #4806

@HuijingHei HuijingHei closed this Jan 30, 2024
@HuijingHei HuijingHei deleted the upgrade-test branch March 4, 2024 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants