filter out last applied config deploy annotations

[tariq1890]

This is the near-term fix for the issue reported in #854.

[tariq1890]

cc @lilic @brancz

[lilic]

This would just filter out the annotation_kubectl_kubernetes_io_last_applied_configuration label from the kube_deployment_annotations metric, but this label will be present in all the resources that can be created/updated with the kubectl/oc apply command. The apply command is the one that sets this annotation, see https://kubernetes.io/docs/tasks/manage-kubernetes-objects/declarative-config/#how-to-create-objects

For example I am seeing it in the namespace as well:

kube_namespace_annotations{namespace="kube-system",annotation_kubectl_kubernetes_io_last_applied_configuration="{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"name\":\"kube-system\"}}\n"} 1

So we sadly need to filter this out from all the kube_*_annotations metrics.

[tariq1890]

Thanks for the review @lilic .

[lilic]

I think we wanted this to go in the 1.7 branch? Should we do that first or cherry-pick it out?

[brancz]

I changed the base branch. This needs a rebase now.

[lilic]

lgtm after rebase

[lilic]

So we (@brancz and I) found a few more annotations that kubernetes applies in the metrics, so we decided to just revert the annotations feature altogether. We cannot know all of the kubernetes annotations and what might potently be in there so reverting the feature might be best right now. Will do a PR for the revert.

We should rethink the annotation metrics feature later on, will open an issue for this.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tariq1890

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tariq1890]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@tariq1890: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[2rs2ts]

Might I suggest using a prefix you define for the annotations to add? Like kubernetes.io/kube-state-metrics/<tag name>?

I'd recommend drawing some inspiration from Datadog's autodiscovery annotations: https://docs.datadoghq.com/agent/autodiscovery/integrations/?tab=kubernetespodannotations#configuration