helm: revert parts of CoreDNS Helm chart packaging

[burgerdev]

Context

We are trying to move CoreDNS from a cluster addon managed by kubeadm to a Helm chart managed alongside the Constellation microservices.

After #3236, upgrading a cluster reverted CoreDNS to the version installed by kubeadm. This required #3353 as a workaround - the cluster pretends it does not have CoreDNS to fool kubeadm, details in that PR's description.

This workaround had an unexpected consequence, though: kubeadm has a preflight check for CoreDNS, which now starts failing. Since this check is done in the upgrade-agent, which is part of the image, it does not benefit from the --ignores added in #3353 and fails. Now the kubeadm upgrade step is never executed, which makes us miss important migrations in the worst case.

Proposed change(s)

Extend the migration to two upgrades.

1. upgrade-agent ignores CoreDNS problems in 2.18+
2. CoreDNS is installed as Helm chart in 2.19+

Additional info

• AB#4627

Checklist

• Run the E2E tests that are relevant to this PR's changes
  • [ ] GCP upgrade
  • GCP upgrade
  • GCP e2e
• Add labels (e.g., for changelog category)
• Is PR title adequate for changelog?
• Link to Milestone

[netlify]

✅ Deploy Preview for constellation-docs canceled.

Name	Link
🔨 Latest commit	4526107
🔍 Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/66eae6570ef23a0008a79126

[elchead]

we should probably add a TODO(v2.19) to the helm chart installation / add a ticket. but I think you have this under control @burgerdev

[burgerdev]

Note to reviewers:

There are three commits,

• revert of helm: manage CoreDNS addon as Helm chart #3236
• revert of helm: fix kubeadm bugs caused by CoreDNS installation #3353
• cherry-pick the upgrade-agent parts of helm: fix kubeadm bugs caused by CoreDNS installation #3353

Limiting the diff to the last one should be sufficient.

we should probably add a TODO(v2.19) to the helm chart installation / add a ticket. but I think you have this under control @burgerdev

I'll add an AB ticket, thanks for the reminder.

[3u13r]

While the upgrade test is still failing, the code itself LGTM

[burgerdev]

Interesting - failing upgrades were not even a symptom of the issue addressed here, and this branch is rebased on #3361. The failure is also during initial cluster setup at v2.17.0, which should be sufficiently tested by now. Let's see what the retry does.

[msanft]

Thanks for explaining this so thoroughly!

[github-actions]

Coverage report

Package	Old	New	Trend
bootstrapper/internal/kubernetes	37.60%	37.70%	↗️
bootstrapper/internal/kubernetes/k8sapi	11.80%	11.80%	🚧
cli/internal/cmd	41.10%	41.10%	🚧
internal/constellation	18.50%	18.60%	↗️
internal/constellation/helm	34.50%	33.60%	↘️
internal/kubernetes/kubectl	8.20%	6.90%	↘️
terraform-provider-constellation/internal/provider	3.50%	3.50%	🚧