[Edge case] Stale deployment.kubernetes.io/desired-replicas annotation block rollout

[himanshu-kun]

How to categorize this issue?

/area robustness
/kind bug
/priority 3

What happened:

A case was seen where there was a single machineSet under the machineDeployment with deployment.kubernetes.io/desired-replicas not up-to-date with machinedeloyment.spec.replicas , but machineSet.spec.Replicas ==machineDeployment.spec.replicas

This led to scaling being detected for this machineDeployment in every reconcile but scale not actually happening.

Later when the consumer triggered a rollout (machineDeployment started referring new machineClass) , the rollout never starts as rollout logic is called after scaling logic

What you expected to happen:

• deployment.kubernetes.io/desired-replicas to re-sync
• rollout not to get blocked

How to reproduce it (as minimally and precisely as possible):

I am not sure how this situation was reached. Ideally , on any machineSet update , all the machineSets are updated, which should update deployment.kubernetes.io/desired-replicas also.

But these set of events could lead to this situation:

Anything else we need to know?:

Environment:
mcm 0.49.0

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• Others:

[himanshu-kun]

Post grooming discussion

• desired-replicas-anno should be placed on machineDeployment and not on machineSet , as it increases number of KAPI calls and possibilities of edge cases , Need to look at k/k code on why they decided to do this?