[#171281178] Document metric exporting to prometheus #343
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
46bit
force-pushed
the
document-metric-exporting-to-prometheus
branch
from
5f6d8f9 to
da7f9c7
Compare
added 3 commits
September 15, 2020 11:47
This commit adds a new entry to the Tech Docs sidebar entitled "Monitoring backing services". This page describes how to access metrics and logs for tenants to understand the state of their backing services. This content is all moved over from the pre-existing "Monitoring apps" documentation. While there may be some broken anchor links, I am not clear that our Tech Docs provide any options for addressing that. To try and mitigate user confusion I've ensured that both sections of documentation link to each other. I have done this because the "Monitoring apps" section was unwieldy and disordered. It contains a description of every backing service metric shown by `paas-admin`, and that's way too much content to have inline. For now I'm not sure where else to put it. In the next commit I will be adding new content describing how to get backing service metrics exported to Prometheus. This split should give that content a good place to live.
This commit adds a short section of documentation:
- Inform users that they can export backing service metrics to
Prometheus (or by implication, other systems compatible with the
Prometheus exposition format)
- Explain that we already exported Postgres and MySQL metrics in the
https://github.com/alphagov/paas-prometheus-exporter
- Explain we can provide Elasticsearch and Redis metrics on request.
Elasticsearch has to be setup manually [1]. Redis is already
deployed [2], but it costs money that we can't recharge yet. To
control costs we want to keep an eye on who is using it.
- Mention that in theory Prometheus can be run on PaaS, and how, but
emphasise that we don't know if it's completely solved. This can
be expanded later if/when we can provide more info.
[1] https://team-manual.cloud.service.gov.uk/support/shipping_elasticsearch_metrics_to_tenants/
[2] https://github.com/alphagov/paas-prometheus-endpoints/tree/main/src/redis
The list of service metrics available in paas-admin hadn't been updated to include Elasticsearch. This commit adds it to the list. I'm not even going to try and add a "Metrics definitions" section. That info massively pollutes the backing service monitoring page already. It doesn't belong there.
46bit
force-pushed
the
document-metric-exporting-to-prometheus
branch
from
da7f9c7 to
9369706
Compare
We now show this information inside paas-admin, above the metric graphs. It could be worth having this information visible, but it enormously pollutes the metrics pages of our tech docs. All the definitions were pretty similar to the ones in paas-admin.
46bit
force-pushed
the
document-metric-exporting-to-prometheus
branch
from
9369706 to
0207270
Compare
|
Now that I've removed the metrics definitions, the "Monitoring backing services" page is quite short. I suggest we keep it that way, as the "Monitoring apps" section is huge. When the metrics endpoints for Redis/etc become generally available there'll be more information to put on the page. I think we need to be able to have separate pages where necessary--for instance, instructions for setting up Logit. These long sets of instructions pollute the pages and make it impossible to find anything by scrolling. |
added 2 commits
September 15, 2020 12:11
Now there's a Prometheus section below this heading was no longer clear enough to explain what it was for.
46bit
force-pushed
the
document-metric-exporting-to-prometheus
branch
from
1b4ac72 to
d032cbc
Compare
|
The Monitoring Apps docs could use some love, but I'm not going there right now. I've filed #344 for one of the most obvious issues with them. |
paroxp
approved these changes
Sep 15, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Does three main things:
paas-adminSee the commit messages for more details.
How to review
Who can review
Not @46bit