[release-v1.34] Auto pick #2658: Prototype: IP pool controller

[caseydavenport]

Cherry pick of #2658 on release-v1.34.

#2658: Prototype: IP pool controller

Original PR Body below

Description

More capable IP pool management controller in the operator.

It is meant to enable a few key use-cases:

• Installing multiple IP pools at install time via the operator.
• Incorporating IP pools not created via the operator into the operator's logic.
• Allow modification of IP pools post-install-time via the operator API.

The standard install-time flow is this:

• The core controller still gets the Installation and performs defaulting on it. It then waits for IP pools to exist.
• The new IP pool controller gets the installation, defaults and/or extracts IP pools from it, and creates them.
• The core controller sees the new pools and continues.

The IP pool controller will refuse to update existing IP pools, or delete IP pools, if the Calico API server is not running. This ensures we have proper validation and defaulting enforced, while allowing a small window using the crd.projectcalico.org API to bootstrap the cluster.

On upgrade, the IP pool controller will attempt to take over control of any IP pools in the cluster that match those in the Installation resource. This allows the operator to assume control of IP pools it previously had created. If there are IP pools present that are NOT in the Installation (and not owned by the controller), they will be ignored by the controller.

Other changes in this PR:

• Add a "Name" field to allow configuration of IP pool name.

For PR author

• Tests for change.
• If changing pkg/apis/, run make gen-files
• If changing versions, run make gen-versions

This PR includes a handful of termination / uninstall changes that are made necessary by the new FV tests added for the IP pool controller exposing an existing bug that is fixed by:

• Adjustments to the termination logic in this PR
• This API server change: Use distinct UIDs for projectcalico.org/v3 APIs projectcalico/calico#8586

For PR reviewers

A note for code reviewers - all pull requests must have the following:

• Milestone set according to targeted release.
• Appropriate labels:
  • kind/bug if this is a bugfix.
  • kind/enhancement if this is a a new feature.
  • enterprise if this PR applies to Calico Enterprise only.