Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Add archive specific metrics #22

Merged
merged 1 commit into from
Oct 17, 2024

Conversation

spali
Copy link
Contributor

@spali spali commented Oct 9, 2024

They are basically free, as already provided by the actual command results.
I find them actually much more interesting than the current sizes of the whole repo... as they allow to conclude to some potential problems or trends...
i.e. :

  • compare archive file counts.... if they differ extrem ... someone deleted everything by accident or by malicious code.
  • allow to track data increase, compression and deduplication a bit different. Per archive can give different insights.
  • duration changes can also show potential problems.

for sure, this metrics only makes sense if you have one repo per backup and the archives represents snapshots of the same data.... but this was anyway a assumption for using last archive data per repo i.e. in borg_last_backup_timestamp.

Sorry for not providing the actual dashboard to this metrics, as I didn't had time to make them proper for the PR.

@maxim-mityutko
Copy link
Owner

Those are indeed interesting use cases!
Please add tests:

def test_registry(self, registry):
result = metrics.create_metrics(registry=registry)._names_to_collectors
assert "borg_total_backups" in result
assert "borg_total_chunks" in result
assert "borg_total_compressed_size" in result
assert "borg_total_size" in result
assert "borg_total_deduplicated_compressed_size" in result
assert "borg_total_deduplicated_size" in result
assert "borg_last_backup_timestamp" in result
@pytest.mark.parametrize(
"metric, repo, expect",
[
("borg_total_backups", "/borg/backup-1", 2.0),
("borg_total_backups", "/borg/backup-3", 0.0),
("borg_total_chunks", "/borg/backup-1", 3505.0),
("borg_total_compressed_size", "/borg/backup-1", 3965903861.0),
("borg_total_size", "/borg/backup-1", 8446787072.0),
("borg_total_deduplicated_compressed_size", "/borg/backup-1", 537932015.0),
("borg_total_deduplicated_size", "/borg/backup-1", 1296544339.0),
("borg_total_deduplicated_size", "/borg/backup-2", 21296544339.0),
],
)
def test_individual_metrics(self, collect, metric, repo, expect):
registry = collect
actual = registry.get_sample_value(name=metric, labels={"repository": repo})
assert actual == expect

Would be good to see how this can be visualized as well, but this can also be done in the next release.
Again, thanks for contributing!

@spali
Copy link
Contributor Author

spali commented Oct 9, 2024

missed the tests again... this time I checked at least the coverage but that didn't change and the above tests don't check actual code lines.

@spali
Copy link
Contributor Author

spali commented Oct 9, 2024

not sure how you choosed the examples for the parameterized test... just added all new metrics for backup-1 and backup-2.
Hope you are ok with that.

@maxim-mityutko
Copy link
Owner

not sure how you choosed the examples for the parameterized test... just added all new metrics for backup-1 and backup-2. Hope you are ok with that.

The values are pretty much random.

Please black the files, otherwise the checks are failing.
https://github.com/maxim-mityutko/borgmatic-exporter/actions/runs/11263638182/job/31323139133?pr=22

@maxim-mityutko maxim-mityutko changed the title add archive specific metrics [feature] Add archive specific metrics Oct 17, 2024
@maxim-mityutko maxim-mityutko merged commit 5e86df6 into maxim-mityutko:master Oct 17, 2024
2 of 3 checks passed
@maxim-mityutko
Copy link
Owner

Thanks! Will release this shortly.

@maxim-mityutko
Copy link
Owner

@spali I have releases the v0.2.7, do you by any chance have the updated Grafana dashboard that exposes these metrics?

@spali
Copy link
Contributor Author

spali commented Oct 18, 2024

No, didn't had time to prep some dashboards. But they may be also very individual, especially for alerting.

@maxim-mityutko
Copy link
Owner

Makes sense.
I've updated the existing dashboard though to include new metrics and changed the structure a little bit.

@spali spali deleted the new_metric branch November 19, 2024 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants