remove unnecessary patch requests when PodReadinessGate is enabled.

[M00nF1sh]

Description

This CR improves performance when a optional feature called PodReadinessGate is enabled by removing unnecessary patch requests to APIServer.

The workflow of PodReadinessGate ideally works as below:

1. when PodReadinessGate feature is enabled, the controller injects readinessGates into pod during pod creation(e.g. target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c), if a pod will be used as backend for a TargetGroup.
2. After a new pod is registered into targetGroup, we'll patch pod to modify the PodCondition to be {"type": "target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c", "status": "False", "reason": "Elb.RegistrationInProgress", "message": "Target registration is in progress"}
3. After a new pod became healthy in targetGroup, we'll patch pod to modify the PodCondition to be {"type": "target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c", "status": "True"}

This bug is for the 3rd step above, the patch were generated to be {"type": "target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c", "status": "True"}, where "reason" & "message" is omitted from patch, thus APIServer will patch the object to be {"type": "target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c", "status": "True", "reason": "Elb.RegistrationInProgress", "message": "Target registration is in progress"}.

Later, if there is a reconcile for the same service triggered again, our logic decides that "reason" and "message" is still not "empty", and will try to patch existing pods again. This bug didn't impact functionality since ReadinessGate only checks a condition's status and ignores "reason" and "message", but it might causes performance issues due to these extra unnecessary Pod patch requests.

This PR modify the patch generation to correct generate the patch to be {"type": "target-health.elbv2.k8s.aws/k8s-echoserv-echoserv-3b3889c42c", "status": "True", "reason": null, "message": null}

TODO: we can further optimize this performance by move pod status updates into a dedicated work queue to prevent it block the main reconcile threads.

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the docs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: M00nF1sh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [M00nF1sh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov-commenter]

Codecov Report

Patch coverage: 93.33% and project coverage change: +0.03% 🎉

Comparison is base (074beee) 55.73% compared to head (a6decce) 55.76%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3373      +/-   ##
==========================================
+ Coverage   55.73%   55.76%   +0.03%     
==========================================
  Files         149      149              
  Lines        8839     8827      -12     
==========================================
- Hits         4926     4922       -4     
+ Misses       3575     3571       -4     
+ Partials      338      334       -4     

Files Changed	Coverage Δ
pkg/targetgroupbinding/resource_manager.go	14.18% <93.33%> (-0.79%)	⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[johngmyers]

/lgtm