kube_pod_container_resource_[limits|requests] metrics not being exported

[jimmyseto]

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened:
the following metrics are being exported when they should not be:
kube_pod_container_resource_limits_cpu_bytes
kube_pod_container_resource_limits_memory_bytes
kube_pod_container_resource_requests_cpu_cores
kube_pod_container_resource_requests_memory_bytes

the following metrics are not being exported when it they should be:
kube_pod_container_resource_limits
kube_pod_container_resource_requests

What you expected to happen:
kube_pod_container_resource_limits and kube_pod_container_resource_requests to replace:
kube_pod_container_resource_limits_cpu_bytes
kube_pod_container_resource_limits_memory_bytes
kube_pod_container_resource_requests_cpu_cores
kube_pod_container_resource_requests_memory_bytes

How to reproduce it (as minimally and precisely as possible):
install kube-state-metrics from https://github.com/kubernetes/kube-state-metrics/tree/master/examples/standard. poll the endpoint. observe unexpected metrics mentioned above. observe missing metrics mentioned above

Anything else we need to know?:
expectations set per https://github.com/kubernetes/kube-state-metrics/releases/tag/v2.0.0-alpha

Environment:

• Kubernetes version (use kubectl version):
  Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0+d4cacc0", GitCommit:"d4cacc0", GitTreeState:"clean", BuildDate:"2020-10-01T19:23:04Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

• Kube-state-metrics image version
  quay.io/coreos/kube-state-metrics:v2.0.0-alpha

[jimmyseto]

@lilic - fyi. this is a bug that should probably be fixed in v2.0.0 as well. what do you think?

[lilic]

They were added, its the CHANGELOG that is wrong my apologies it was a lot of changes to document. Good catch! They were just reworked and marked as experimental see #1168

Will adjust the changelog when doing the next alpha prerelease. Thanks again!! 🎉

[jimmyseto]

They were added, its the CHANGELOG that is wrong my apologies it was a lot of changes to document. Good catch! They were just reworked and marked as experimental see #1168

Will adjust the changelog when doing the next alpha prerelease. Thanks again!! 🎉

hmm. ok. i was looking at #1004, which does remove them. but, i guess they were added back via #1168. so, what is the expected/desired behavior for v2.0.0? should both metric formats be exported for the "well-known resources" (cpu, memory, storage, ephemeral_storage)? at a quick glance of the latter PR, perhaps the desired behavior is, for the "well-known" metrics, to only export the kube_pod_container_resource_[limits|requests] metrics, but not the generic kube_pod_container_resource_[limits|requests] metrics. and, for resources outside the "well-known" set, export them using the generic kube_pod_container_resource_[limits|requests] metrics. is that correct? if so, i think that jives with what i'm observing above.

thanks in advance for the clarification.

[lilic]

Yes exactly that, hoping the new CHANGELOG clarifies that so users don't think they are not same metrics, hence the marking from STABLE to experimental https://github.com/kubernetes/kube-state-metrics/pull/1253/files#diff-4ac32a78649ca5bdd8e0ba38b7006a1eR27

[jimmyseto]

Yes exactly that, hoping the new CHANGELOG clarifies that so users don't think they are not same metrics, hence the marking from STABLE to experimental https://github.com/kubernetes/kube-state-metrics/pull/1253/files#diff-4ac32a78649ca5bdd8e0ba38b7006a1eR27

thanks. would something like this be more accurate?

Removed deprecated stable metrics #1004. kube_pod_container_resource_requests and kube_pod_container_resource_limits for cpu, memory, storage, and ephemeral_storage resources have been replaced with EXPERIMENTAL metrics #1168.

kube_pod_container_resource_requests_cpu_cores
kube_pod_container_resource_limits_cpu_cores
kube_pod_container_resource_requests_memory_bytes
kube_pod_container_resource_limits_memory_bytes
kube_pod_container_resource_requests_storage_bytes
kube_pod_container_resource_limits_stroage_bytes
kube_pod_container_resource_requests_ephemeral_storage_bytes
kube_pod_container_resource_limits_ephemeral_storage_bytes

[lilic]

@jimmyseto sounds good, can you comment the suggestion the PR? Thanks!

[jimmyseto]

@jimmyseto sounds good, can you comment the suggestion the PR? Thanks!

comment posted to PR.

thanks, @lilic . i think we are good after that! closing issue.

[yli186]

@lilic
Hi sorry hopping on the thread, since this is similar topic.
Since we have recently upgraded our cluster to 1.18 from 1.14, so we are trying out the v2.0.0-alpha.2 version. But we were not able to get any scrape for kube_pod_container_resource_limits, while kube_pod_container_resource_requests is scraping fine for most of the pod (for some stateful pods was also missing, still looking into the actual cause).
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
We have also checked on our old set-up of k8s 1.14 with version v1.9.3, both metrics are scraping fine.

Not sure if this is a bug on a similar note or any pointer would be appreciated, thanks!

[jimmyseto]

@lilic
Hi sorry hopping on the thread, since this is similar topic.
Since we have recently upgraded our cluster to 1.18 from 1.14, so we are trying out the v2.0.0-alpha.2 version. But we were not able to get any scrape for kube_pod_container_resource_limits, while kube_pod_container_resource_requests is scraping fine for most of the pod (for some stateful pods was also missing, still looking into the actual cause).
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
We have also checked on our old set-up of k8s 1.14 with version v1.9.3, both metrics are scraping fine.

Not sure if this is a bug on a similar note or any pointer would be appreciated, thanks!

@yli186 - i'm observing this as well. can you create an issue for this? it looks like a regression introduced as part of a revert done between alpha.1 and alpha.2 (see #1278). i can work on creating a PR for this today.

[yli186]

Thank you @jimmyseto, created #1292