[release-0.6] WaitForPodsReady: Reset .status.requeueState once the workload is deactivated

[tenzen-y]

What type of PR is this?

/kind bug

What this PR does / why we need it:

If the Workload is deactivated, the workload resets the requeueState to avoid a race condition between the kueue-scheduler and the workload controller in the case of the deactivated workload.

When the workload is deactivated by exceeding the backoffLimitCount, the workload is added The workload is deactivated due to exceeding the maximum number of re-queuing retries. I will introduce a dedicated reason in another PR to focus on fixing the bug.

Which issue(s) this PR fixes:

Fixes #2174

Special notes for your reviewer:

This PR is cherry-pick of #2219

The primary difference from #2219 is the criteria to deduce that the Workload is deactivated due to exceeding the backoffLimitCount.

In the v0.7 implementation, the deactivated workload doesn't have the .status.requeuesState.requeueAt since the requeueAt should be reset when the workload was requeued (Requeued=true condition).

// The deactivation reason could be deduced as the maximum number of re-queuing retries if the workload met all criteria below:
//  1. The waitForPodsReady feature is enabled, which means that it has "PodsReady" condition.
//  2. The workload has already exceeded the PodsReadyTimeout.
//  3. The workload already has been re-queued previously, which means it doesn't have the requeueAt field.
//  4. The number of re-queued has already reached the waitForPodsReady.requeuingBackoffLimitCount.

But, in the v0.6 implementation, an empty requeueAt is dropped from the criteria since we never reset the requeueAt and Requeued condition, a new feature for the v0.7.

// The deactivation reason could be deduced as the maximum number of re-queuing retries if the workload met all criteria below:
//  1. The waitForPodsReady feature is enabled, which means that it has "PodsReady" condition.
//  2. The workload has already exceeded the PodsReadyTimeout.
//  3. The number of re-queued has already reached the waitForPodsReady.requeuingBackoffLimitCount.

Does this PR introduce a user-facing change?

Fix a bug that causes the reactivated Workload to be immediately deactivated even though it doesn't exceed the backoffLimit.

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	5609e61
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-kueue/deploys/6654bde9352691000886a8b2

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• pkg/controller/OWNERS [tenzen-y]
• test/OWNERS [tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[alculquicondor]

still WIP?

[tenzen-y]

still WIP?

@alculquicondor This PR is ready for review right now.
PTAL.

[alculquicondor]

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: 89e36512bbe041155d132d044739c277a459ee6c