Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KubeOne 1.8.0 removes the kured DaemonSet, keep other kured resources #3196

Open
judge-red opened this issue May 14, 2024 · 1 comment
Open
Assignees
Labels
customer-request kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.
Milestone

Comments

@judge-red
Copy link

judge-red commented May 14, 2024

What happened?

During the upgrade to KubeOne 1.8.0, with the unattended-upgrades addon both pre-upgrade and post-upgrade and no other changes either, the kured DaemonSet got removed but the rest of the kured resources (ServiceAccount, ClusterRole, ClusterRoleBinding, Role, RoleBinding) were kept.

Expected behavior

It's unclear from the changelog and upgrade docs, whether the intention is to orphan the installed kured or to remove it, so not sure which is the bug, the removal of the DaemonSet or that the other resources are left behind. If I had to guess, the changelog and upgrade docs seem to hint at complete removal, as they urge for the installation of kured using the upstream Helm chart.

I think either way is fine, but it needs to be clear what happens (i.e. the docs must be more explicit), and it must be either completely orphaned or be left completely untouched.

How to reproduce the issue?

  • Install a Kubeone 1.7 cluster with the unattended-upgrades addon enabled
  • Upgrade to KubeOne 1.8
  • Notice the kured DaemonSet is missing, other resources are still there

What KubeOne version are you using?

1.8.0

Provide your KubeOneCluster manifest here (if applicable)

apiVersion: kubeone.k8c.io/v1beta2
kind: KubeOneCluster
name: <...>
versions:
  kubernetes: 1.27.13
features:
  openidConnect:
    enable: true
    config:
      <...>
addons:
  enable: true
  addons:
    - name: backups-restic
      params:
        <...>
    - name: ccm-openstack
      params:
        CCM_CONCURRENT_SERVICE_SYNCS: "10"
    - name: cluster-autoscaler
    - name: unattended-upgrades
cloudProvider:
  external: true
  cloudConfig: |
    <...>
clusterNetwork:
  cni:
    external: {}
  kubeProxy:
    skipInstallation: true
helmReleases:
  <...>

What cloud provider are you running on?

OpenStack

What operating system are you running in your cluster?

Ubuntu 22.04

Additional information

Those who wish to install the upstream Helm chart of kured will first have to clean up the left behind resources, here's the required commands for your convenience:

kubectl delete clusterrole kured
kubectl delete clusterrolebinding kured
kubectl -n kube-system delete role kured
kubectl -n kube-system delete rolebinding kured
kubectl -n kube-system delete serviceaccount kured
@judge-red judge-red added kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management. labels May 14, 2024
@kron4eg kron4eg added this to the KubeOne 1.8 milestone May 15, 2024
@kron4eg
Copy link
Member

kron4eg commented May 15, 2024

Thanks for the report @judge-red! Yeah, I can confirm this. It's due to how kubectl apply --prune works (it started to skip non-namespaced resources).

I'll think how to fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-request kind/bug Categorizes issue or PR as related to a bug. sig/cluster-management Denotes a PR or issue as being assigned to SIG Cluster Management.
Projects
None yet
Development

No branches or pull requests

3 participants