Upgrade kube-state-metrics to latest when kube_namespace_annotations is enabled

[vijay-veeranki]

Now, Prometheus is upgraded to the latest version, but kube-state-metrics sub chart latest version removed kube_namespace_annotations metrics. So we downgraded the version to v1.7.0 to use kube-state-metrics as some users and cloud-platform set up alerts using kube_namespace_annotations

kube-state-metrics:
  image:
    tag: v1.7.0

There is an issue raised to fix this
kubernetes/kube-state-metrics#941

Once this is available, we need to upgrade to use the latest image.
Upgrade to v2.2.0

Definition of done:

Existing alerts work as expected and "kube_namespace_annotations" work as well, as our (cloud-platform) team is using it in application alerts.

[AntonyBishop]

Reviewed. Issue still unresolved so story is still blocked.

[AntonyBishop]

Reviewed. Issue still unresolved so story is still blocked.

[AntonyBishop]

Reviewed. Issue still unresolved so story is still blocked.

[AntonyBishop]

Reviewed. Issue still unresolved so story is still blocked.

[davidread]

Important not to forget to unpin this tag when the issue mentioned is resolved

[vijay-veeranki]

Comment here says this is not fixed in release v2.2.0

kubernetes/kube-state-metrics#941 (comment)

[vijay-veeranki]

kube-state-metrics is failing in "live" cluster as it is running on K8's 1.19.

Removed blocked on this issue and move to this sprint to work on this.

[vijay-veeranki]

Upgraded EKS Prometheus to the latest version, which got kube-state-metrics subchart v2.2.0 with fix in it
ministryofjustice/cloud-platform-infrastructure#1419
ministryofjustice/cloud-platform-terraform-monitoring#111