The operator keeps on altering keyspace replications at each reconcile

[adejanovski]

It looks like k8ssandra-operator could be altering keyspace replication a little too often and even without justification in some cases:

2024-01-29T15:55:26.639Z	INFO	keyspace reaper_db already exists in cluster alex2 but has wrong replication, altering it	{"controller": "k8ssandracluster", "controllerGroup": "k8ssandra.io", "controllerKind": "K8ssandraCluster", "K8ssandraCluster": {"name":"alex2","namespace":"alex-n50118bo"}, "namespace": "alex-n50118bo", "name": "alex2", "reconcileID": "e8b816df-eb31-4c73-8716-ea6512d3f8d3", "K8ssandraCluster": "alex-n50118bo/alex2", "CassandraDatacenter": "alex-n50118bo/alex2-dc1", "K8SContext": ""}

This happens when two clusters coexist in the same namespace, using dc names overrides which makes them both have a dc1 dc for example, while the cassdc objects are named cluster1-dc1 and cluster2-dc1.
The DC label will be dc1 for pods from both clusters and the following piece of code will misbehave as it's not filtering the pods on the cluster name as well:

func (r *defaultManagementApiFacade) fetchDatacenterPods() ([]corev1.Pod, error) {
	podList := &corev1.PodList{}
	labels := client.MatchingLabels{cassdcapi.DatacenterLabel: r.dc.DatacenterName()}
	if err := r.k8sClient.List(r.ctx, podList, labels); err != nil {
		return nil, err
	} else {
		pods := r.filterPods(podList.Items, func(pod corev1.Pod) bool {
			status := r.getCassandraContainerStatus(pod)
			return status != nil && status.Ready
		})
		if len(pods) == 0 {
			err = fmt.Errorf("no pods in READY state found in datacenter %v", r.dc.Name)
			return nil, err
		}
		return pods, nil
	}
}