Automated cherry pick of #1838: WaitForPodsReady: Reset the requeueState while reconciling

[tenzen-y]

Cherry pick of #1838 on release-0.6.
#1838: WaitForPodsReady: Reset the requeueState while reconciling
For details on the cherry pick process, see the cherry pick requests page.

WaitForPodsReady: Fix a bug that the requeueState isn't reset.

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	41a8ac3
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/65f32d4a0bcbb10008222200

[tenzen-y]

/assign @alculquicondor

[tenzen-y]

2024-03-14T17:03:16.805955051Z ERROR controller-runtime.test-env envtest/server.go:317 unable to start the controlplane {"tries": 0, "error": "timeout waiting for process kube-apiserver to start"}

Failed to start the control-plane.

/test pull-kueue-test-integration-release-0-6

[tenzen-y]

2024-03-14T17:03:16.805955051Z ERROR controller-runtime.test-env envtest/server.go:317 unable to start the controlplane {"tries": 0, "error": "timeout waiting for process kube-apiserver to start"}

Failed to start the control-plane.

/test pull-kueue-test-integration-release-0-6

Wait, it seems that other errors happened:

2024-03-14T17:07:38.313021161Z ERROR controller/controller.go:329 Reconciler error {"controller": "rayjob", "controllerGroup": "ray.io", "controllerKind": "RayJob", "RayJob": {"name":"test-job","namespace":"core-l2gkb"}, "namespace": "core-l2gkb", "name": "test-job", "reconcileID": "7e6da7c5-2f19-4364-af73-462799c71413", "error": "Operation cannot be fulfilled on workloads.kueue.x-k8s.io "rayjob-test-job-715ab": StorageError: invalid object, Code: 4, Key: /registry/kueue.x-k8s.io/workloads/core-l2gkb/rayjob-test-job-715ab, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 609c5df0-dfe8-41e7-bd0e-94889424872f, UID in object meta: "}

[tenzen-y]

I couldn't find any existing flaky test issues.

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/1843/pull-kueue-test-integration-release-0-6/1768321538177634304

[alculquicondor]

Precondition failed: UID in precondition:

I have seen these kind of errors when the server is terminating or objects are being deleted while there is still another API call for an update in flight.

[alculquicondor]

/lgtm
/approve

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 11619245f48f9536e7c42b494d6250ebea2880a6

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [alculquicondor,tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tenzen-y]

Precondition failed: UID in precondition:

I have seen these kind of errors when the server is terminating or objects are being deleted while there is still another API call for an update in flight.

I see. Thank you for letting me know.

[alculquicondor]

/kind bug