Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transaction pool metrics are broken #11070

Open
Longarithm opened this issue Apr 15, 2024 · 1 comment
Open

Transaction pool metrics are broken #11070

Longarithm opened this issue Apr 15, 2024 · 1 comment

Comments

@Longarithm
Copy link
Member

During #10543 tx pool size and count metrics were showing continuous growth: https://nearinc.grafana.net/goto/6axQo--IR?orgId=1
Something is broken with computation using "deltas" - updating metrics with txs inserted/deleted for chunks. With naive metrics computation, sizes were around zero, as expected. Code Longarithm/nearcore@master...more-metrics-4

@aborg-dev
Copy link
Contributor

I think the root cause is that shard_id label is broken now for this metric, as instead of numeric values 0..5 we now have string values "s0.v3", "s1.v3", ... (see more info here https://near.zulipchat.com/#narrow/stream/308695-nearone.2Fprivate/topic/.5Bnode.5D.20Oncall.2030.20Mar.20-.205.20Apr.202024/near/431322538).

Can you try specifying a concrete label and see if this fixes the problem? Also, does the linked graph collect data from RPC nodes? For them the TX pool is sometimes growing without cleanup in some corner-cases (when they have validator keys but are not actively validating blocks).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants