-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Directly emit container ready time metric #2119
Comments
/triage accepted |
The container level metric should already be available: kube-state-metrics/internal/store/pod.go Line 1342 in 02417fb
|
Hi @dgrisonnet, thanks for looking into this. Unfortunately, I believe the metric you pointed to is actually at the pod level, representing the time that all containers are ready (ContainersReady). From the comments in the api:
|
Correct, the name got me. We should probably base |
@bbdouglas I am curious how you currently calculate this with promQL? |
@abhiraut Here is the query I came up with. Since it's looking back, you have to manually set the maximum age that you expect a pod to be up. Here I have assumed no pod lives for more than 1 day.
|
thanks ! |
What would you like to be added:
It would be great to have a metric for the container ready time in seconds to be emitted directly. There is currently a boolean gauge
kube_pod_container_status_ready
, which emits whether the container is ready or not, but that requires some computation to get at the time when the container flipped to the ready state. I'm interested in learning the amount of time it took between when the container started and when it was ready, and that would be simpler and more efficient to measure ifkube-state-metrics
emitted the ready time directly.There was a similar metric added at the pod level (#1465), but this would be at the container level. In the pods that I am tracking, there are many containers with wildly varying ready times, so it is helpful for debugging and optimization purposes to know how long each container takes to get ready.
Why is this needed:
Similar to the pod-level ready time metric (#1465), I'd like to measure the ready time of each individual container within my pod. This is helpful for tracking startup-times at a finer level of granularity than the whole pod, especially when a pod has many containers.
It is possible to use the existing boolean
kube_pod_container_status_ready
boolean to calculate this by looking at a series of data points and choosing the first point in time when that flag flips from false to true, but in practice that can be very resource intensive for Prometheus to calculate if there are a large number of pods/containers.Describe the solution you'd like
I would ideally like to see a new metric analogous to
kube_pod_status_ready_time
emitted at the container granularity.Additional context
I'm not that familiar with the internals of the Kubernetes API, but unfortunately it does not look like ContainerStatus has the same breadth of information as PodCondition, which includes a LastTransitionTime. So this might not be a simple addition.
The text was updated successfully, but these errors were encountered: