Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(trie): add Prometheus metrics for tries #2720

Closed
wants to merge 2 commits into from

Conversation

qdm12
Copy link
Contributor

@qdm12 qdm12 commented Aug 3, 2022

Changes

⚠️ this is just production code changes. I have most test code locally, but would like some feedback before continuing updating tests and adding more deltas for reviewers.

💁 this PR shall be the foundation to accommodate several future additional metrics (such as number of key bytes, number of value bytes for all tries, number of encodings, disk operations etc.)

Changes consist essentially in:

  • 3 metric packages, each with a Prometheus and a Noop implementation (for tests, instead of using 'any' mocks)
    • internal/runtime/metrics/roothash
    • internal/runtime/metrics/proof
    • internal/state/metrics
  • Add trie Die() method to clear metrics (add nodes removed). This is used when the trie is no longer needed and would be GC. We kind of need this to keep track of the total number of nodes kept in memory, especially for state tries.

The alternative implementation would be to return statistics from trie operations (Put, Delete etc.) and handle all the metrics in callers (lib/runtime/wasmer/imports.go and dot/state/tries.go). That makes a bit more sense so we handle all the metricing in callers, since those metrics are not specific to a single trie. On the other hand, that looks like it would require a lot more code changes 😢

Tests

Issues

#2229

Primary Reviewer

@timwu20

@qdm12 qdm12 changed the base branch from development to qdm12/dot-core/handlecodesub-wasmer-only August 3, 2022 15:09
- Remove `wasmerInstanceFunc` function type
- Add error wrapping context within `handleCodeSubstitution`
- Adding sentinel errors for wasm decompression
- Fix and improve `Test_Service_handleCodeSubstitution`
- Add runtime instance gomock mock
@qdm12 qdm12 force-pushed the qdm12/dot-core/handlecodesub-wasmer-only branch from d5ae2f6 to 9979cd8 Compare August 3, 2022 15:09
@qdm12 qdm12 force-pushed the qdm12/trie/prometheus branch from 3e2b123 to 38fb9c5 Compare August 3, 2022 15:10
- `internal/runtime/metrics/roothash`
- `internal/runtime/metrics/proof`
- `internal/state/metrics`
Comment on lines -25 to -27
if t == nil {
t = trie.NewEmptyTrie()
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to self: ensure no test NewTrieState(nil) remain

@qdm12 qdm12 force-pushed the qdm12/trie/prometheus branch from 38fb9c5 to cf909d8 Compare August 3, 2022 17:32
Base automatically changed from qdm12/dot-core/handlecodesub-wasmer-only to development August 6, 2022 16:06
@dimartiro dimartiro mentioned this pull request May 24, 2023
27 tasks
@danforbes danforbes closed this Aug 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants