✨ Add KCP feature to clusterctl alpha rollout #6858

tobiasgiese · 2022-07-07T14:42:33Z

Signed-off-by: Tobias Giese tobias.giese@mercedes-benz.com

What this PR does / why we need it:

This PR implements the sub-commands rollout, pause and resume for clusterctl alpha rollout

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Part of #3439 (replaces #6857)

tobiasgiese · 2022-07-07T16:06:06Z

/retest

tests were successful on Prow

tobiasgiese · 2022-07-07T16:27:33Z

/test pull-cluster-api-e2e-informing-main

flaky CC test

tobiasgiese · 2022-07-07T19:42:22Z

cmd/clusterctl/client/alpha/rollout_restarter.go

@@ -53,3 +65,8 @@ func setRestartedAtAnnotation(proxy cluster.Proxy, name, namespace string) error
 	patch := client.RawPatch(types.MergePatchType, []byte(fmt.Sprintf("{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"cluster.x-k8s.io/restartedAt\":\"%v\"}}}}}", time.Now().Format(time.RFC3339))))
 	return patchMachineDeployment(proxy, name, namespace, patch)
 }
+
+func setRolloutAfter(proxy cluster.Proxy, name, namespace string) error {
+	patch := client.RawPatch(types.MergePatchType, []byte(fmt.Sprintf("{\"spec\":{\"rolloutAfter\":\"%v\"}}", time.Now().Format(time.RFC3339))))


Another idea to handle issues regarding different timezones or async local times: we could use for rolloutAfter the creation timestamp + a few seconds of the most recent control plane machine. In this case the rolling update should start in any case.

CAPI is checking if the machines creation timestamp is before rolloutAfter and rolloutAfter is before the reconciliation time.

cluster-api/util/collections/machine_filters.go

Line 176 in 03f62b1

return machine.CreationTimestamp.Before(rolloutAfter) && rolloutAfter.Before(reconciliationTime)

Regarding #6858 (comment)

AFAIK kubectl rollout restart has no rolloutAfter alike field. I think that the rollout will be performed immediately.

And btw, e.g. Deployment has also a restartedAt field like our MachineDeployment.

❯ kk rollout restart deploy coredns deployment.apps/coredns restarted ❯ kk get deploy coredns -oyaml | grep restartedAt kubectl.kubernetes.io/restartedAt: "2022-07-18T12:09:53+02:00"

Same for Statefulsets

❯ kk rollout restart sts csi-resizer-cinder statefulset.apps/csi-resizer-cinder restarted ❯ kk get sts csi-resizer-cinder -oyaml | grep restartedAt kubectl.kubernetes.io/restartedAt: "2022-07-18T12:11:06+02:00"

chrischdi

Thanks for this PR :-)

All in all it looks pretty good, only some nits :-)
Still want to / have to play around with it.

cmd/clusterctl/client/alpha/rollout.go

cmd/clusterctl/client/alpha/rollout_restarter_test.go

cmd/clusterctl/client/alpha/kubeadmcontrolplane.go

cmd/clusterctl/client/alpha/rollout_resumer.go

cmd/clusterctl/client/alpha/rollout_restarter.go

cmd/clusterctl/client/alpha/rollout_resumer.go

chrischdi · 2022-08-08T15:33:00Z

One question: Does making use of RolloutAfter limit the functionality for RolloutAfter.

E.g. a user may have already set RolloutAfter to a timestamp in the future (e.g. to enforce recreation of the control plane to renew certificates).

So what would happen if RolloutAfter is already set (and because of that we should not overwrite it?)

Edit: so I currently think we should not make use of RolloutAfter for clusterctl alpha rollout.

tobiasgiese · 2022-08-13T08:21:16Z

One question: Does making use of RolloutAfter limit the functionality for RolloutAfter.

AFAIK the RolloutAfter value will be overriden. Maybe we have to change the update behavior of the KCP? E.g., make use of the cluster.x-k8s.io/restartedAt for the KCP as well. In this case we can leave the RolloutAfter exclusively for the actual use case. This would also fit to #7053.

sbueringer · 2022-08-16T12:26:26Z

cc @ykakarap

ykakarap

This looks interesting to me. I will take a look a this.

sbueringer · 2023-01-20T11:22:51Z

/retest

ykakarap · 2023-01-21T00:53:08Z

/lgtm

vincepri

/approve
/hold
for @fabriziopandini's review

Great work thank you!

tobiasgiese · 2023-01-23T20:10:05Z

/hold cancel

tobiasgiese · 2023-01-23T20:10:37Z

/hold

sorry, overlooked your hold Vince

tobiasgiese · 2023-01-25T09:58:26Z

Will solve the merge conflict after the review from @fabriziopandini

fabriziopandini

Only one question, otherwise lgtm

fabriziopandini · 2023-01-25T11:06:32Z

hack/tools/go.mod

@@ -49,6 +49,8 @@ require (
 	github.com/blang/semver/v4 v4.0.0 // indirect
 	github.com/cespare/xxhash/v2 v2.1.2 // indirect
 	github.com/containerd/containerd v1.6.12 // indirect
+	github.com/coredns/caddy v1.1.0 // indirect
+	github.com/coredns/corefile-migration v1.0.18 // indirect


q for my understanding, why are there such changes in hack/tools/go.mod? this sems unrelated to the other changes in this PR

I am also not sure why these changes are in the PR. But go mod tidy and the Prow job want this change.

IIRC we found out why these changes happen @chrischdi, right? Do you still know why?

hack/tools/tilt-prepare uses "sigs.k8s.io/cluster-api/cmd/clusterct/*"

"sigs.k8s.io/cluster-api/cmd/clusterct/*" now uses "sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1beta1" (for the KubeadmControlPlane type)

"sigs.k8s.io/cluster-api/controlplane/kubeadm/api/v1beta1" uses "github.com/coredns/corefile-migration" (for the KCP webhook)

"github.com/coredns/corefile-migration" uses "github.com/coredns/caddy" (internally)

I came to the same conclusion as stefan 🏆

sbueringer · 2023-01-25T12:32:12Z

@tobiasgiese Can you please rebase the PR? I would merge then

Signed-off-by: Tobias Giese <tobias.giese@mercedes-benz.com>

tobiasgiese · 2023-01-25T12:35:39Z

@tobiasgiese Can you please rebase the PR? I would merge then

Done, thank you 👍🏻

sbueringer · 2023-01-25T12:53:08Z

Thank you very much!! (especially for the patience :))

/lgtm
/approve

k8s-ci-robot · 2023-01-25T12:53:14Z

LGTM label has been added.

Git tree hash: fad73e5eb2d6bfa576ff62c98f846f5987bce042

k8s-ci-robot · 2023-01-25T12:53:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer, vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sbueringer,vincepri]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tobiasgiese · 2023-01-25T12:55:05Z

/hold cancel

fabriziopandini · 2023-01-25T14:45:14Z

thanks for pushing this over the line @tobiasgiese!

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 7, 2022

k8s-ci-robot requested review from stmcginnis and ykakarap July 7, 2022 14:42

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from f77c1f3 to 6dd881d Compare July 7, 2022 14:47

tobiasgiese mentioned this pull request Jul 7, 2022

Implement clusterctl KCP rollout command #6857

Closed

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch 3 times, most recently from 34d4f49 to 4747856 Compare July 7, 2022 15:16

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch 2 times, most recently from 24f31ae to 95c0015 Compare July 7, 2022 18:07

tobiasgiese commented Jul 7, 2022

View reviewed changes

chrischdi reviewed Jul 18, 2022

View reviewed changes

cmd/clusterctl/client/alpha/rollout_restarter.go Outdated Show resolved Hide resolved

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from 95c0015 to c8c2742 Compare July 18, 2022 09:04

chrischdi reviewed Jul 18, 2022

View reviewed changes

cmd/clusterctl/client/alpha/rollout_resumer.go Show resolved Hide resolved

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from c8c2742 to 3e892ae Compare July 18, 2022 09:42

ykakarap reviewed Aug 18, 2022

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 14, 2022

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from 3e892ae to 86e02a0 Compare September 15, 2022 15:03

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 15, 2022

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from 86e02a0 to 2d38aa0 Compare September 15, 2022 15:07

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 15, 2022

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 28, 2022

vincepri approved these changes Jan 23, 2023

View reviewed changes

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 23, 2023

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 23, 2023

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jan 23, 2023

fabriziopandini reviewed Jan 25, 2023

View reviewed changes

Add KCP feature to clusterctl alpha rollout

4593c3c

Signed-off-by: Tobias Giese <tobias.giese@mercedes-benz.com>

tobiasgiese force-pushed the tobiasgiese/clusterctl-kcp-rollout branch from 6eb2e31 to 4593c3c Compare January 25, 2023 12:34

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 25, 2023

k8s-ci-robot requested review from fabriziopandini and sbueringer January 25, 2023 12:34

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 25, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 25, 2023

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 25, 2023

k8s-ci-robot merged commit 33d0e8f into kubernetes-sigs:main Jan 25, 2023

k8s-ci-robot added this to the v1.4 milestone Jan 25, 2023

tobiasgiese deleted the tobiasgiese/clusterctl-kcp-rollout branch January 25, 2023 13:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ Add KCP feature to clusterctl alpha rollout #6858

✨ Add KCP feature to clusterctl alpha rollout #6858

tobiasgiese commented Jul 7, 2022 •

edited

Loading

tobiasgiese commented Jul 7, 2022 •

edited

Loading

tobiasgiese commented Jul 7, 2022

tobiasgiese Jul 7, 2022 •

edited

Loading

tobiasgiese Jul 18, 2022 •

edited

Loading

tobiasgiese Jul 18, 2022

chrischdi left a comment

chrischdi commented Aug 8, 2022 •

edited

Loading

tobiasgiese commented Aug 13, 2022 •

edited

Loading

sbueringer commented Aug 16, 2022

ykakarap left a comment

sbueringer commented Jan 20, 2023

ykakarap commented Jan 21, 2023

vincepri left a comment

tobiasgiese commented Jan 23, 2023

tobiasgiese commented Jan 23, 2023 •

edited

Loading

tobiasgiese commented Jan 25, 2023

fabriziopandini left a comment

fabriziopandini Jan 25, 2023

tobiasgiese Jan 25, 2023 •

edited

Loading

sbueringer Jan 25, 2023 •

edited

Loading

chrischdi Jan 25, 2023

sbueringer commented Jan 25, 2023

tobiasgiese commented Jan 25, 2023

sbueringer commented Jan 25, 2023

k8s-ci-robot commented Jan 25, 2023

k8s-ci-robot commented Jan 25, 2023

tobiasgiese commented Jan 25, 2023

fabriziopandini commented Jan 25, 2023

✨ Add KCP feature to clusterctl alpha rollout #6858

✨ Add KCP feature to clusterctl alpha rollout #6858

Conversation

tobiasgiese commented Jul 7, 2022 • edited Loading

tobiasgiese commented Jul 7, 2022 • edited Loading

tobiasgiese commented Jul 7, 2022

tobiasgiese Jul 7, 2022 • edited Loading

Choose a reason for hiding this comment

tobiasgiese Jul 18, 2022 • edited Loading

Choose a reason for hiding this comment

tobiasgiese Jul 18, 2022

Choose a reason for hiding this comment

chrischdi left a comment

Choose a reason for hiding this comment

chrischdi commented Aug 8, 2022 • edited Loading

tobiasgiese commented Aug 13, 2022 • edited Loading

sbueringer commented Aug 16, 2022

ykakarap left a comment

Choose a reason for hiding this comment

sbueringer commented Jan 20, 2023

ykakarap commented Jan 21, 2023

vincepri left a comment

Choose a reason for hiding this comment

tobiasgiese commented Jan 23, 2023

tobiasgiese commented Jan 23, 2023 • edited Loading

tobiasgiese commented Jan 25, 2023

fabriziopandini left a comment

Choose a reason for hiding this comment

fabriziopandini Jan 25, 2023

Choose a reason for hiding this comment

tobiasgiese Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

sbueringer Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

chrischdi Jan 25, 2023

Choose a reason for hiding this comment

sbueringer commented Jan 25, 2023

tobiasgiese commented Jan 25, 2023

sbueringer commented Jan 25, 2023

k8s-ci-robot commented Jan 25, 2023

k8s-ci-robot commented Jan 25, 2023

tobiasgiese commented Jan 25, 2023

fabriziopandini commented Jan 25, 2023

tobiasgiese commented Jul 7, 2022 •

edited

Loading

tobiasgiese commented Jul 7, 2022 •

edited

Loading

tobiasgiese Jul 7, 2022 •

edited

Loading

tobiasgiese Jul 18, 2022 •

edited

Loading

chrischdi commented Aug 8, 2022 •

edited

Loading

tobiasgiese commented Aug 13, 2022 •

edited

Loading

tobiasgiese commented Jan 23, 2023 •

edited

Loading

tobiasgiese Jan 25, 2023 •

edited

Loading

sbueringer Jan 25, 2023 •

edited

Loading