-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update nomad metrics to include information about Job status #4036
Comments
Hi, thanks for the question. These metrics should already be enabled, look for the metric name nomad.nomad.job_summary. We will be sure to update our telemetry documentation to include this metric as well. |
Thanks. I'll ensure I have the correct settings (https://www.nomadproject.io/docs/agent/configuration/telemetry.html) by Monday and update this ticket. |
I can't find it:
These are the TaskGroup summaries. I'm looking for Job Status, the higher level one (e.g. Running, Dead):
For the moment I'm exporting this myself with a custom Prometheus exporter: func (nc *NomadCollector) Collect(ch chan<- prometheus.Metric) {
jobs, _, err := nc.client.Jobs().List(&api.QueryOptions{})
if err != nil {
logError(err)
return
}
for _, job := range jobs {
ch <- prometheus.MustNewConstMetric(
nc.nomadJobStatus,
prometheus.GaugeValue,
1.0,
job.ID,
job.Name,
job.Type,
job.Status,
)
} |
We'd like above metrics exposed as well. Made a separate exporter for batch and parameterized jobs, scraping the HTTP API. 🐙 |
master...sevagh:feat/job-metrics |
Some accidental churn as I deleted my old fork of Nomad, but my pull request to implement this feature request is as of yet unmerged, and according to some other users here, I think several people are running custom Nomad exporters to expose this info (which this PR would help eliminate): |
Hey there Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this. Thanks! |
This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem 👍 |
See #6003 for another PR to try and fix this. |
The documentation at https://www.nomadproject.io/docs/telemetry/metrics.html#tagged-metrics mentions these are under |
Those docs seem to be for the unreleased 0.10 version where my PR is merged. |
Is there a way to display docs for released versions by default? I find this very confusing :( |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Hello,
We noticed that the metrics endpoint (scraped by Prometheus) exposes Task Group Summaries per Job's TaskGroup: https://github.com/hashicorp/nomad/blob/master/api/jobs.go#L691
However there are no metrics exposed for the job status (i.e. https://github.com/hashicorp/nomad/blob/master/api/jobs.go#L691)
Is that something I should compose by some aggregation on Task Group Summaries, or could this be a useful metric to add in a PR?
The text was updated successfully, but these errors were encountered: