OCPBUGS-8703: Backport switchkernel 4.13

[cgwalters]

Manual cherry pick of #3580 since it conflicted on vendored code.

---

daemon: Clean up switchKernel a bit

De-duplicate calls to canonicalizeKernelType to make the
logic easier to read. Also add a few comments.

(cherry picked from commit b75c7af)

---

vendor: Bump coreos/rpm-ostree-client-go

In prep for usage in MCD.

(cherry picked from commit cae67a6)

---

daemon: Make switchKernel less stateful

This is prep for fixing RHEL9 upgrades while maintaining kernel-rt.

Previously the switchKernel logic tried to carefully handle
all 4 cases (default -> default, default -> rt, rt -> default, rt -> rt).

But, the last one (rt -> rt) was not quite right because
the previous rpm-ostree rebase command already preserved the previous
kernel. In fact it was pretty expensive to do things this way
because we'd e.g. regenerate the initramfs twice.

To say this another way: when doing a RHEL9 update, it's actually
the first rpm-ostree rebase command which fails before we
even get to switchKernel.

And the reason is due to the introduction of a new -core subpackage;
xref https://issues.redhat.com/browse/OCPBUGS-8113

So here's the new logic to handle this:

• Before we do the rebase operation to the new OS, we detect
  any previous overrides of any packages starting with kernel-rt
  and we remove them. Notably this avoids hardcoding any specific
  kernel subpackages; we just remove everything starting with
  kernel-rt which should be more robust to subpackage changes
  in the future.
• Consequently the rebase operation will hence start out by deploying the
  stock image i.e. with throughput kernel (though note we are
  carefully preserving other local overrides)
• The switchKernel function now longer needs to take the previous
  machineconfig state into account (except for logging).
  Instead, we just detect if the target is RT, and if so we then we
  apply the latest packages.

This significantly simplifies the logic in switchKernel, and will
help fix RHEL9 upgrades.

(cherry picked from commit 8ac5bee)

---

[openshift-ci]

@cgwalters: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

Backport switchkernel 4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@cgwalters: This pull request references Jira Issue OCPBUGS-8703, which is invalid:

• expected Jira Issue OCPBUGS-8703 to depend on a bug targeting a version in 4.14.0 and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Manual cherry pick of #3580 since it conflicted on vendored code.

---

daemon: Clean up switchKernel a bit

De-duplicate calls to canonicalizeKernelType to make the
logic easier to read. Also add a few comments.

(cherry picked from commit b75c7af)

---

vendor: Bump coreos/rpm-ostree-client-go

In prep for usage in MCD.

(cherry picked from commit cae67a6)

---

daemon: Make switchKernel less stateful

This is prep for fixing RHEL9 upgrades while maintaining kernel-rt.

Previously the switchKernel logic tried to carefully handle
all 4 cases (default -> default, default -> rt, rt -> default, rt -> rt).

But, the last one (rt -> rt) was not quite right because
the previous rpm-ostree rebase command already preserved the previous
kernel. In fact it was pretty expensive to do things this way
because we'd e.g. regenerate the initramfs twice.

To say this another way: when doing a RHEL9 update, it's actually
the first rpm-ostree rebase command which fails before we
even get to switchKernel.

And the reason is due to the introduction of a new -core subpackage;
xref https://issues.redhat.com/browse/OCPBUGS-8113

So here's the new logic to handle this:

• Before we do the rebase operation to the new OS, we detect
  any previous overrides of any packages starting with kernel-rt
  and we remove them. Notably this avoids hardcoding any specific
  kernel subpackages; we just remove everything starting with
  kernel-rt which should be more robust to subpackage changes
  in the future.
• Consequently the rebase operation will hence start out by deploying the
  stock image i.e. with throughput kernel (though note we are
  carefully preserving other local overrides)
• The switchKernel function now longer needs to take the previous
  machineconfig state into account (except for logging).
  Instead, we just detect if the target is RT, and if so we then we
  apply the latest packages.

This significantly simplifies the logic in switchKernel, and will
help fix RHEL9 upgrades.

(cherry picked from commit 8ac5bee)

---

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@cgwalters: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

OCPBUGS-8703: Backport switchkernel 4.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cgwalters]

/jira refresh

[cgwalters]

(That said, in practice it may make the most sense to just land this as part of a cherry-pick of #3485 )

[openshift-ci-robot]

@cgwalters: This pull request references Jira Issue OCPBUGS-8703, which is valid. The bug has been moved to the POST state.

6 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.13.0) matches configured target version for branch (4.13.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
• dependent bug Jira Issue OCPBUGS-8113 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
• dependent Jira Issue OCPBUGS-8113 targets the "4.14.0" version, which is one of the valid target versions: 4.14.0
• bug has dependents

Requesting review from QA contact:
/cc @rioliu-rh

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rioliu-rh]

/label cherry-pick-approved

[sinnykumari]

/lgtm
/label backport-risk-assessed
/test e2e-gcp-op
/test e2e-gcp-rt

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, sinnykumari

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cgwalters,sinnykumari]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@cgwalters: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-vsphere-ovn	c844f7a	link	false	/test okd-scos-e2e-vsphere-ovn
ci/prow/e2e-alibabacloud-ovn	c844f7a	link	false	/test e2e-alibabacloud-ovn
ci/prow/e2e-gcp-rt	c844f7a	link	false	/test e2e-gcp-rt
ci/prow/okd-scos-e2e-gcp-ovn-upgrade	c844f7a	link	false	/test okd-scos-e2e-gcp-ovn-upgrade
ci/prow/e2e-gcp-rt-op	c844f7a	link	false	/test e2e-gcp-rt-op

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[openshift-ci-robot]

@cgwalters: Jira Issue OCPBUGS-8703: All pull requests linked via external trackers have merged:

• openshift/machine-config-operator#3595

Jira Issue OCPBUGS-8703 has been moved to the MODIFIED state.

In response to this:

Manual cherry pick of #3580 since it conflicted on vendored code.

---

daemon: Clean up switchKernel a bit

De-duplicate calls to canonicalizeKernelType to make the
logic easier to read. Also add a few comments.

(cherry picked from commit b75c7af)

---

vendor: Bump coreos/rpm-ostree-client-go

In prep for usage in MCD.

(cherry picked from commit cae67a6)

---

daemon: Make switchKernel less stateful

This is prep for fixing RHEL9 upgrades while maintaining kernel-rt.

Previously the switchKernel logic tried to carefully handle
all 4 cases (default -> default, default -> rt, rt -> default, rt -> rt).

But, the last one (rt -> rt) was not quite right because
the previous rpm-ostree rebase command already preserved the previous
kernel. In fact it was pretty expensive to do things this way
because we'd e.g. regenerate the initramfs twice.

To say this another way: when doing a RHEL9 update, it's actually
the first rpm-ostree rebase command which fails before we
even get to switchKernel.

And the reason is due to the introduction of a new -core subpackage;
xref https://issues.redhat.com/browse/OCPBUGS-8113

So here's the new logic to handle this:

• Before we do the rebase operation to the new OS, we detect
  any previous overrides of any packages starting with kernel-rt
  and we remove them. Notably this avoids hardcoding any specific
  kernel subpackages; we just remove everything starting with
  kernel-rt which should be more robust to subpackage changes
  in the future.
• Consequently the rebase operation will hence start out by deploying the
  stock image i.e. with throughput kernel (though note we are
  carefully preserving other local overrides)
• The switchKernel function now longer needs to take the previous
  machineconfig state into account (except for logging).
  Instead, we just detect if the target is RT, and if so we then we
  apply the latest packages.

This significantly simplifies the logic in switchKernel, and will
help fix RHEL9 upgrades.

(cherry picked from commit 8ac5bee)

---

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.