feat(k8s): discovery uses EndpointSlices instead of Endpoints

[andrewazores]

Welcome to Cryostat! 👋

Before contributing, make sure you have:

• Read the contributing guidelines
• Linked a relevant issue which this PR resolves
• Linked any other relevant issues, PR's, or documentation, if any
• Resolved all conflicts, if any
• Rebased your branch PR on top of the latest upstream main branch
• Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
• Signed all commits using a GPG signature

To recreate commits with GPG signature git fetch upstream && git rebase --force --gpg-sign upstream/main

---

Fixes: #717

Description of the change:

Replaces the Kubernetes API discovery mechanism's usage of the older Endpoints with the newer EndpointSlices. This should be functionally identical to the end Cryostat user and should record the same results in the database, it's only a slight internal logic change.

For early review and testing purposes, this PR currently retains the old Endpoints-based implementation and adds the new EndpointSlices-based implementation as a separate discovery plugin/realm, which is enabled by a different config property. This is so that it's possible to enable both at the same time on the same Cryostat instance and verify that they behave identically. Once that is validated then I will clean up this PR so that the new implementation simply replaces the old one as an internal implementation detail.

Motivation for the change:

This is a newer Kubernetes API feature and may have some performance benefits (fewer discovery update events?).~~ It opens the door for future improvements like using the EndpointSlices' readiness property to either gate target discoveryability, or to include that readiness data in the target record.~~ The EndpointSlices' "conditions" are embedded into the target definition as Cryostat annotations, ie. the target's readiness state can be determined.

How to manually test:

1. Get a Kubernetes or OpenShift cluster
2. Check out WIP EndpointSlices discovery cryostat-helm#226
3. Create namespaces cryostat, apps1, and apps2. In each, use the Operator's Makefile to make sample_app to install a sample application to each namespace
4. kubectl scale -n apps2 deployment quarkus-test --replicas=2
5. Install Cryostat, using an image built from this PR: $ helm install --set core.image.repository=quay.io/andrewazores/cryostat --set core.image.tag=endpoint-slices-4 --set core.route.enabled=true --set core.discovery.kubernetes.namespaces='{cryostat,apps1,apps2}' --set core.discovery.kubernetes.enabled=false --set core.discovery.kubernetes2.enabled=true cryostat ./charts/cryostat (if not using OpenShift, turn off the core.route.enabled switch)
6. Wait for Cryostat to be ready, then open its UI and go to Topology
7. Verify that Cryostat discovers all the targets as usual, but using the new EndpointSlices discovery. When using the new discovery mechanism there should also be Cryostat annotations visible in the Topology view indicating the slices' conditions, ie readiness/serving/terminating:

8. Flip back to the old discovery mechanism: helm upgrade --reuse-values --set core.discovery.kubernetes.enabled=true --set core.discovery.kubernetes2.enabled=false cryostat ./charts/cryostat
9. Wait for Cryostat to be ready, then reopen the UI and verify that the Topology view looks essentially the same
10. Use the kubectl scale command on different sample applications in each namespace to test the behaviour of both the old and new implementations. Both should behave the same way with target lost/found events.