feat: add network tag to metrics #12733

virajbhartiya · 2024-11-28T05:03:35Z

Add network tag to metrics

virajbhartiya · 2024-11-28T05:04:56Z

@rvagg I am trying to setup a Grafana dashboard on my device to test this out. Can you please just let me know if I am going in the right direction with this PR

rvagg · 2024-11-28T05:59:13Z

Yep, I think this is the right direction. It's unfortunate that you need to add it to all of the views but that looks like the trickiest bit here.

You should be able to register the value at startup, like in here:

lotus/cmd/lotus/daemon.go

Lines 211 to 216 in 07f2f69

    
           ctx, _ := tag.New(context.Background(), 
        
           	tag.Insert(metrics.Version, build.NodeBuildVersion), 
        
           	tag.Insert(metrics.Commit, build.CurrentCommit), 
        
           	tag.Insert(metrics.NodeType, "chain"), 
        
           ) 
        
           // Register all metric views

And in here:

lotus/cmd/lotus-miner/run.go

Lines 58 to 63 in 07f2f69

    
           ctx, _ := tag.New(lcli.DaemonContext(cctx), 
        
           	tag.Insert(metrics.Version, build.MinerBuildVersion), 
        
           	tag.Insert(metrics.Commit, build.CurrentCommit), 
        
           	tag.Insert(metrics.NodeType, "miner"), 
        
           ) 
        
           // Register all metric views

Currently I have grafana setup for my node and I'm scraping with prometheus, in my prometheus.yml scape_configs I have:

  - job_name: lotus-mainnet
    scrape_interval: 10s
    metrics_path: '/debug/metrics'
    static_configs:
      - targets: ['localhost:1234']
        labels:
          network: 'mainnet'
  - job_name: lotus-calibnet
    scrape_interval: 10s
    metrics_path: '/debug/metrics'
    static_configs:
      - targets: ['localhost:1235']
        labels:
          network: 'calibnet'

you can see my mainnet and calibnet tags being inserted there so I can do what you're doing here, but after the fact.

Then in grafana, when I explore metrics I can see the labels showing up that get reported for that metric type. All of mine have network but some have additional tags. In this one, network is there, but prometheus has also added instance and job, but the metric itself (block/failure) also has its own custom tag, from here:

lotus/chain/sub/incoming.go

Lines 466 to 473 in 07f2f69

    
           func recordFailure(ctx context.Context, metric *stats.Int64Measure, failureType string) { 
        
           	ctx, _ = tag.New( 
        
           		ctx, 
        
           		tag.Upsert(metrics.FailureType, failureType), 
        
           	) 
        
           	stats.Record(ctx, metric.M(1)) 
        
           }

virajbhartiya · 2024-11-29T06:58:27Z

We could get the network name from build/buildconstants/params.go we'll have to split the + to use it

cmd/lotus-miner/run.go

cmd/lotus/daemon.go

Co-authored-by: Rod Vagg <rod@vagg.org>

Kubuxu · 2024-12-04T14:09:42Z

@rvagg I am trying to setup a Grafana dashboard on my device to test this out. Can you please just let me know if I am going in the right direction with this PR

You can do "curl localhost:1234/debug/metrics` (IIRC) to fetch these metrics to check

rvagg · 2024-12-05T06:04:21Z

good hint 👌

$ curl -s http://localhost:1234/debug/metrics | grep block_failure
# HELP lotus_block_failure Counter for block validation failures
# TYPE lotus_block_failure counter
lotus_block_failure{failure_type=""} 193
lotus_block_failure{failure_type="unknown_miner"} 3557

so @virajbhartiya you should be able to compile a node, run it, do this curl and confirm that all of the metrics that you're editing have the network tag on them.

cmd/lotus-miner/run.go

rvagg · 2024-12-05T06:06:31Z

Code seems good to go other than the stray "strings" in a file that looks like it was manually edited but not run through go fmt, just accepting my PR suggestions I guess? Well now there's a new suggestion that should do it.

If you could compile and run this and confirm that the tags appear on that /debug/metrics output that would be helpful @virajbhartiya . Otherwise I think this is good to go.

virajbhartiya · 2024-12-10T02:14:09Z

curl -s http://localhost:1234/debug/metrics | grep network
lotus_info{commit="+git.1ab14eae2.dirty",network="calibrationnet",node_type="chain",version="1.31.1-dev"} 1
lotus_peer_count{network=""} 13
lotus_pubsub_prune{network=""} 80
lotus_pubsub_recv_rpc{network=""} 60
lotus_pubsub_rejected{network=""} 1
lotus_pubsub_send_rpc{network=""} 28
lotus_rcmgr_allow_conn{direction="inbound",network="",use_fd="true"} 1
lotus_rcmgr_allow_conn{direction="outbound",network="",use_fd="false"} 191
lotus_rcmgr_allow_conn{direction="outbound",network="",use_fd="true"} 165
lotus_rcmgr_allow_mem{network=""} 385
lotus_rcmgr_allow_peer{network="",peer_id=""} 357
lotus_rcmgr_allow_proto{network="",proto="/fil/chain/xchg/0.0.1"} 20
lotus_rcmgr_allow_proto{network="",proto="/fil/hello/1.0.0"} 76
lotus_rcmgr_allow_proto{network="",proto="/fil/kad/calibrationnet/kad/1.0.0"} 47
lotus_rcmgr_allow_proto{network="",proto="/ipfs/id/1.0.0"} 58
lotus_rcmgr_allow_proto{network="",proto="/ipfs/id/push/1.0.0"} 27
lotus_rcmgr_allow_proto{network="",proto="/ipfs/ping/1.0.0"} 4
lotus_rcmgr_allow_proto{network="",proto="/libp2p/autonat/1.0.0"} 2
lotus_rcmgr_allow_proto{network="",proto="/meshsub/1.1.0"} 58
lotus_rcmgr_allow_stream{direction="inbound",network="",peer_id=""} 139
lotus_rcmgr_allow_stream{direction="outbound",network="",peer_id=""} 163
lotus_rcmgr_allow_svc{network="",svc="libp2p.autonat"} 2
lotus_rcmgr_allow_svc{network="",svc="libp2p.identify"} 85
lotus_rcmgr_allow_svc{network="",svc="libp2p.ping"} 4
lotus_splitstore_miss{network=""} 926055

rvagg · 2024-12-11T04:35:01Z

lotus_pubsub_send_rpc{network=""} 28
lotus_rcmgr_allow_conn{direction="inbound",network="",use_fd="true"} 1
lotus_rcmgr_allow_conn{direction="outbound",network="",use_fd="false"} 191
lotus_rcmgr_allow_conn{direction="outbound",network="",use_fd="true"} 165
lotus_rcmgr_allow_mem{network=""} 385
lotus_rcmgr_allow_peer{network="",peer_id=""} 357

they're empty! something's going wrong for them not to be set in the main context .. I wonder if that context isn't being propagated through these subcomponents, or maybe set too late

rvagg · 2024-12-11T04:42:39Z

@virajbhartiya can you dump a full output from that curl, I assume this is just partial? We only have the one correct one for lotus_info but that's reported not long after the ctx is initialised. Perhaps we have context breaks for some of these and I don't know if that's intentional or not, or how hard it is to find them. If they're all missing except for lotus_info then maybe something else is going on in the fx direct-injection tooling not to be passing on the context.

rvagg · 2024-12-11T05:05:06Z

This might not be too hard to solve actually - for all of the metrics that don't fill out network, see if you can trace their contexts. It looks like we don't have too many in this boat:

$ git grep context.Background | grep -v '_test.*.go' | wc -l 
140

Even just hunting them down individually might not be hard. e.g. SplitstoreMiss is here:

lotus/blockstore/splitstore/splitstore.go

Line 415 in 12d76bd

stats.Record(s.ctx, metrics.SplitstoreMiss.M(1))

A quick search for s.ctx.*= shows it up here:

lotus/blockstore/splitstore/splitstore.go

Line 269 in 12d76bd

ss.ctx, ss.cancel = context.WithCancel(context.Background())

(Why it doesn't take a context argument is another question! But we'll leave that aside).

So in this case, you can put your tag on that context right where it's initialised.

The rcmgr metrics might not be hard either: if you look in node/modules/lp2p/rcmgr.go you'll find blocks like this:

lotus/node/modules/lp2p/rcmgr.go

Lines 195 to 209 in 12d76bd

    
           func (r rcmgrMetrics) BlockConn(dir network.Direction, usefd bool) { 
        
           	ctx := context.Background() 
        
           	if dir == network.DirInbound { 
        
           		ctx, _ = tag.New(ctx, tag.Upsert(metrics.Direction, "inbound")) 
        
           	} else { 
        
           		ctx, _ = tag.New(ctx, tag.Upsert(metrics.Direction, "outbound")) 
        
           	} 
        
           	if usefd { 
        
           		ctx, _ = tag.New(ctx, tag.Upsert(metrics.UseFD, "true")) 
        
           	} else { 
        
           		ctx, _ = tag.New(ctx, tag.Upsert(metrics.UseFD, "false")) 
        
           	} 
        
           	stats.Record(ctx, metrics.RcmgrBlockConn.M(1)) 
        
           }

Already setting up tags, just put the network tag value there too.

virajbhartiya · 2024-12-11T10:02:26Z

mainnet

curl -s http://localhost:1234/debug/metrics | grep network
lotus_info{commit="+git.78e0bf5d6.dirty",network="mainnet",node_type="chain",version="1.32.1-dev"} 1
lotus_peer_count{network=""} 0
lotus_pubsub_prune{network=""} 2
lotus_pubsub_recv_rpc{network=""} 126
lotus_pubsub_send_rpc{network=""} 6
lotus_rcmgr_allow_conn{direction="outbound",network="mainnet",use_fd="false"} 299
lotus_rcmgr_allow_conn{direction="outbound",network="mainnet",use_fd="true"} 226
lotus_rcmgr_allow_mem{network=""} 2349
lotus_rcmgr_allow_peer{network="mainnet",peer_id=""} 524
lotus_rcmgr_allow_proto{network="mainnet",proto="/chain/ipfs/bitswap/1.2.0"} 388
lotus_rcmgr_allow_proto{network="mainnet",proto="/fil/hello/1.0.0"} 249
lotus_rcmgr_allow_proto{network="mainnet",proto="/fil/kad/testnetnet/kad/1.0.0"} 153
lotus_rcmgr_allow_proto{network="mainnet",proto="/ipfs/id/1.0.0"} 301
lotus_rcmgr_allow_proto{network="mainnet",proto="/ipfs/id/push/1.0.0"} 25
lotus_rcmgr_allow_proto{network="mainnet",proto="/ipfs/ping/1.0.0"} 1
lotus_rcmgr_allow_proto{network="mainnet",proto="/libp2p/autonat/1.0.0"} 10
lotus_rcmgr_allow_proto{network="mainnet",proto="/meshsub/1.1.0"} 288
lotus_rcmgr_allow_stream{direction="inbound",network="mainnet",peer_id=""} 813
lotus_rcmgr_allow_stream{direction="outbound",network="mainnet",peer_id=""} 622
lotus_rcmgr_allow_svc{network="mainnet",svc="libp2p.autonat"} 10
lotus_rcmgr_allow_svc{network="mainnet",svc="libp2p.identify"} 321
lotus_rcmgr_allow_svc{network="mainnet",svc="libp2p.ping"} 1
lotus_rcmgr_block_peer{network="mainnet",peer_id=""} 1
lotus_splitstore_miss{network="mainnet"} 2.280991e+06

virajbhartiya · 2024-12-11T10:04:11Z

Calibration net (ran this a bit longer)

curl -s http://localhost:1234/debug/metrics | grep network
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.01"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.05"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.1"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.3"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.6"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="0.8"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="1"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="2"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="3"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="4"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="5"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="6"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="8"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="10"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="13"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="16"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="20"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="25"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="30"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="40"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="50"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="65"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="80"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="100"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="130"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="160"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="200"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="250"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="300"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="400"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="500"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="650"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="800"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="1000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="2000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="3000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="4000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="5000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="7500"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="10000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="20000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="50000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="100000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="250000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="500000"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="1e+06"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet",le="+Inf"} 1
lotus_api_request_duration_ms_sum{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet"} 0
lotus_api_request_duration_ms_count{api="lotus-daemon",endpoint="ChainHead",network="calibrationnet"} 1
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.01"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.05"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.1"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.3"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.6"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="0.8"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="1"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="2"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="3"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="4"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="5"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="6"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="8"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="10"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="13"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="16"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="20"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="25"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="30"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="40"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="50"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="65"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="80"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="100"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="130"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="160"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="200"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="250"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="300"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="400"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="500"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="650"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="800"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="1000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="2000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="3000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="4000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="5000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="7500"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="10000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="20000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="50000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="100000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="250000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="500000"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="1e+06"} 2
lotus_api_request_duration_ms_bucket{api="lotus-daemon",endpoint="SyncState",network="calibrationnet",le="+Inf"} 2
lotus_api_request_duration_ms_sum{api="lotus-daemon",endpoint="SyncState",network="calibrationnet"} 0
lotus_api_request_duration_ms_count{api="lotus-daemon",endpoint="SyncState",network="calibrationnet"} 2
lotus_block_failure{failure_type="",network=""} 4
lotus_block_received{network=""} 17
lotus_block_success{network=""} 13
lotus_block_validation_ms_bucket{network="",le="0.01"} 0
lotus_block_validation_ms_bucket{network="",le="0.05"} 0
lotus_block_validation_ms_bucket{network="",le="0.1"} 0
lotus_block_validation_ms_bucket{network="",le="0.3"} 0
lotus_block_validation_ms_bucket{network="",le="0.6"} 0
lotus_block_validation_ms_bucket{network="",le="0.8"} 0
lotus_block_validation_ms_bucket{network="",le="1"} 0
lotus_block_validation_ms_bucket{network="",le="2"} 0
lotus_block_validation_ms_bucket{network="",le="3"} 0
lotus_block_validation_ms_bucket{network="",le="4"} 0
lotus_block_validation_ms_bucket{network="",le="5"} 0
lotus_block_validation_ms_bucket{network="",le="6"} 3
lotus_block_validation_ms_bucket{network="",le="8"} 7
lotus_block_validation_ms_bucket{network="",le="10"} 9
lotus_block_validation_ms_bucket{network="",le="13"} 9
lotus_block_validation_ms_bucket{network="",le="16"} 10
lotus_block_validation_ms_bucket{network="",le="20"} 20
lotus_block_validation_ms_bucket{network="",le="25"} 24
lotus_block_validation_ms_bucket{network="",le="30"} 28
lotus_block_validation_ms_bucket{network="",le="40"} 31
lotus_block_validation_ms_bucket{network="",le="50"} 32
lotus_block_validation_ms_bucket{network="",le="65"} 32
lotus_block_validation_ms_bucket{network="",le="80"} 32
lotus_block_validation_ms_bucket{network="",le="100"} 32
lotus_block_validation_ms_bucket{network="",le="130"} 32
lotus_block_validation_ms_bucket{network="",le="160"} 32
lotus_block_validation_ms_bucket{network="",le="200"} 32
lotus_block_validation_ms_bucket{network="",le="250"} 32
lotus_block_validation_ms_bucket{network="",le="300"} 32
lotus_block_validation_ms_bucket{network="",le="400"} 32
lotus_block_validation_ms_bucket{network="",le="500"} 32
lotus_block_validation_ms_bucket{network="",le="650"} 32
lotus_block_validation_ms_bucket{network="",le="800"} 32
lotus_block_validation_ms_bucket{network="",le="1000"} 32
lotus_block_validation_ms_bucket{network="",le="2000"} 32
lotus_block_validation_ms_bucket{network="",le="3000"} 36
lotus_block_validation_ms_bucket{network="",le="4000"} 36
lotus_block_validation_ms_bucket{network="",le="5000"} 36
lotus_block_validation_ms_bucket{network="",le="7500"} 36
lotus_block_validation_ms_bucket{network="",le="10000"} 36
lotus_block_validation_ms_bucket{network="",le="20000"} 36
lotus_block_validation_ms_bucket{network="",le="50000"} 36
lotus_block_validation_ms_bucket{network="",le="100000"} 36
lotus_block_validation_ms_bucket{network="",le="250000"} 36
lotus_block_validation_ms_bucket{network="",le="500000"} 36
lotus_block_validation_ms_bucket{network="",le="1e+06"} 36
lotus_block_validation_ms_bucket{network="",le="+Inf"} 36
lotus_block_validation_ms_sum{network=""} 11559.000000000004
lotus_block_validation_ms_count{network=""} 36
lotus_chain_node_height{network=""} 2.219501e+06
lotus_chain_node_height_expected{network=""} 2.2195e+06
lotus_chain_node_worker_height{network=""} 2.219501e+06
lotus_info{commit="+git.be5d79a88",network="calibrationnet",node_type="chain",version="1.32.1-dev"} 1
lotus_message_received{network=""} 2
lotus_message_success{network=""} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.01"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.05"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.1"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.3"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.6"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="0.8"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="1"} 0
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="2"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="3"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="4"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="5"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="6"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="8"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="10"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="13"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="16"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="20"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="25"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="30"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="40"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="50"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="65"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="80"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="100"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="130"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="160"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="200"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="250"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="300"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="400"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="500"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="650"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="800"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="1000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="2000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="3000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="4000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="5000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="7500"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="10000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="20000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="50000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="100000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="250000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="500000"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="1e+06"} 2
lotus_message_validation_ms_bucket{local="",message_valid="true",network="",le="+Inf"} 2
lotus_message_validation_ms_sum{local="",message_valid="true",network=""} 2.694
lotus_message_validation_ms_count{local="",message_valid="true",network=""} 2
lotus_mpool_add_ms_bucket{network="",le="0.01"} 0
lotus_mpool_add_ms_bucket{network="",le="0.05"} 0
lotus_mpool_add_ms_bucket{network="",le="0.1"} 0
lotus_mpool_add_ms_bucket{network="",le="0.3"} 0
lotus_mpool_add_ms_bucket{network="",le="0.6"} 0
lotus_mpool_add_ms_bucket{network="",le="0.8"} 0
lotus_mpool_add_ms_bucket{network="",le="1"} 0
lotus_mpool_add_ms_bucket{network="",le="2"} 2
lotus_mpool_add_ms_bucket{network="",le="3"} 2
lotus_mpool_add_ms_bucket{network="",le="4"} 2
lotus_mpool_add_ms_bucket{network="",le="5"} 2
lotus_mpool_add_ms_bucket{network="",le="6"} 2
lotus_mpool_add_ms_bucket{network="",le="8"} 2
lotus_mpool_add_ms_bucket{network="",le="10"} 2
lotus_mpool_add_ms_bucket{network="",le="13"} 2
lotus_mpool_add_ms_bucket{network="",le="16"} 2
lotus_mpool_add_ms_bucket{network="",le="20"} 2
lotus_mpool_add_ms_bucket{network="",le="25"} 2
lotus_mpool_add_ms_bucket{network="",le="30"} 2
lotus_mpool_add_ms_bucket{network="",le="40"} 2
lotus_mpool_add_ms_bucket{network="",le="50"} 2
lotus_mpool_add_ms_bucket{network="",le="65"} 2
lotus_mpool_add_ms_bucket{network="",le="80"} 2
lotus_mpool_add_ms_bucket{network="",le="100"} 2
lotus_mpool_add_ms_bucket{network="",le="130"} 2
lotus_mpool_add_ms_bucket{network="",le="160"} 2
lotus_mpool_add_ms_bucket{network="",le="200"} 2
lotus_mpool_add_ms_bucket{network="",le="250"} 2
lotus_mpool_add_ms_bucket{network="",le="300"} 2
lotus_mpool_add_ms_bucket{network="",le="400"} 2
lotus_mpool_add_ms_bucket{network="",le="500"} 2
lotus_mpool_add_ms_bucket{network="",le="650"} 2
lotus_mpool_add_ms_bucket{network="",le="800"} 2
lotus_mpool_add_ms_bucket{network="",le="1000"} 2
lotus_mpool_add_ms_bucket{network="",le="2000"} 2
lotus_mpool_add_ms_bucket{network="",le="3000"} 2
lotus_mpool_add_ms_bucket{network="",le="4000"} 2
lotus_mpool_add_ms_bucket{network="",le="5000"} 2
lotus_mpool_add_ms_bucket{network="",le="7500"} 2
lotus_mpool_add_ms_bucket{network="",le="10000"} 2
lotus_mpool_add_ms_bucket{network="",le="20000"} 2
lotus_mpool_add_ms_bucket{network="",le="50000"} 2
lotus_mpool_add_ms_bucket{network="",le="100000"} 2
lotus_mpool_add_ms_bucket{network="",le="250000"} 2
lotus_mpool_add_ms_bucket{network="",le="500000"} 2
lotus_mpool_add_ms_bucket{network="",le="1e+06"} 2
lotus_mpool_add_ms_bucket{network="",le="+Inf"} 2
lotus_mpool_add_ms_sum{network=""} 2
lotus_mpool_add_ms_count{network=""} 2
lotus_mpool_addts_ms_bucket{network="",le="0.01"} 1
lotus_mpool_addts_ms_bucket{network="",le="0.05"} 1
lotus_mpool_addts_ms_bucket{network="",le="0.1"} 1
lotus_mpool_addts_ms_bucket{network="",le="0.3"} 1
lotus_mpool_addts_ms_bucket{network="",le="0.6"} 1
lotus_mpool_addts_ms_bucket{network="",le="0.8"} 1
lotus_mpool_addts_ms_bucket{network="",le="1"} 1
lotus_mpool_addts_ms_bucket{network="",le="2"} 2
lotus_mpool_addts_ms_bucket{network="",le="3"} 2
lotus_mpool_addts_ms_bucket{network="",le="4"} 2
lotus_mpool_addts_ms_bucket{network="",le="5"} 2
lotus_mpool_addts_ms_bucket{network="",le="6"} 2
lotus_mpool_addts_ms_bucket{network="",le="8"} 2
lotus_mpool_addts_ms_bucket{network="",le="10"} 2
lotus_mpool_addts_ms_bucket{network="",le="13"} 2
lotus_mpool_addts_ms_bucket{network="",le="16"} 2
lotus_mpool_addts_ms_bucket{network="",le="20"} 2
lotus_mpool_addts_ms_bucket{network="",le="25"} 2
lotus_mpool_addts_ms_bucket{network="",le="30"} 2
lotus_mpool_addts_ms_bucket{network="",le="40"} 2
lotus_mpool_addts_ms_bucket{network="",le="50"} 2
lotus_mpool_addts_ms_bucket{network="",le="65"} 2
lotus_mpool_addts_ms_bucket{network="",le="80"} 2
lotus_mpool_addts_ms_bucket{network="",le="100"} 2
lotus_mpool_addts_ms_bucket{network="",le="130"} 2
lotus_mpool_addts_ms_bucket{network="",le="160"} 2
lotus_mpool_addts_ms_bucket{network="",le="200"} 2
lotus_mpool_addts_ms_bucket{network="",le="250"} 2
lotus_mpool_addts_ms_bucket{network="",le="300"} 2
lotus_mpool_addts_ms_bucket{network="",le="400"} 2
lotus_mpool_addts_ms_bucket{network="",le="500"} 2
lotus_mpool_addts_ms_bucket{network="",le="650"} 2
lotus_mpool_addts_ms_bucket{network="",le="800"} 2
lotus_mpool_addts_ms_bucket{network="",le="1000"} 2
lotus_mpool_addts_ms_bucket{network="",le="2000"} 2
lotus_mpool_addts_ms_bucket{network="",le="3000"} 2
lotus_mpool_addts_ms_bucket{network="",le="4000"} 2
lotus_mpool_addts_ms_bucket{network="",le="5000"} 2
lotus_mpool_addts_ms_bucket{network="",le="7500"} 2
lotus_mpool_addts_ms_bucket{network="",le="10000"} 2
lotus_mpool_addts_ms_bucket{network="",le="20000"} 2
lotus_mpool_addts_ms_bucket{network="",le="50000"} 2
lotus_mpool_addts_ms_bucket{network="",le="100000"} 2
lotus_mpool_addts_ms_bucket{network="",le="250000"} 2
lotus_mpool_addts_ms_bucket{network="",le="500000"} 2
lotus_mpool_addts_ms_bucket{network="",le="1e+06"} 2
lotus_mpool_addts_ms_bucket{network="",le="+Inf"} 2
lotus_mpool_addts_ms_sum{network=""} 1
lotus_mpool_addts_ms_count{network=""} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.01"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.05"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.1"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.3"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.6"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="0.8"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="1"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="2"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="3"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="4"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="5"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="6"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="8"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="10"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="13"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="16"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="20"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="25"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="30"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="40"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="50"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="65"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="80"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="100"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="130"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="160"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="200"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="250"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="300"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="400"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="500"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="650"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="800"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="1000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="2000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="3000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="4000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="5000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="7500"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="10000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="20000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="50000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="100000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="250000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="500000"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="1e+06"} 2
lotus_mpool_getbalance_ms_bucket{network="",le="+Inf"} 2
lotus_mpool_getbalance_ms_sum{network=""} 0
lotus_mpool_getbalance_ms_count{network=""} 2
lotus_mpool_getnonce_ms_bucket{network="",le="0.01"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="0.05"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="0.1"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="0.3"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="0.6"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="0.8"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="1"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="2"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="3"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="4"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="5"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="6"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="8"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="10"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="13"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="16"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="20"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="25"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="30"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="40"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="50"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="65"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="80"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="100"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="130"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="160"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="200"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="250"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="300"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="400"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="500"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="650"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="800"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="1000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="2000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="3000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="4000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="5000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="7500"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="10000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="20000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="50000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="100000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="250000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="500000"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="1e+06"} 5
lotus_mpool_getnonce_ms_bucket{network="",le="+Inf"} 5
lotus_mpool_getnonce_ms_sum{network=""} 0
lotus_mpool_getnonce_ms_count{network=""} 5
lotus_mpool_message_count{network=""} 0
lotus_peer_count{network=""} 41
lotus_pubsub_delivered{network=""} 129
lotus_pubsub_duplicate{network=""} 1070
lotus_pubsub_prune{network=""} 134
lotus_pubsub_recv_rpc{network=""} 1525
lotus_pubsub_rejected{network=""} 27
lotus_pubsub_send_rpc{network=""} 1560
lotus_rcmgr_allow_conn{direction="inbound",network="calibrationnet",use_fd="false"} 17
lotus_rcmgr_allow_conn{direction="inbound",network="calibrationnet",use_fd="true"} 5
lotus_rcmgr_allow_conn{direction="outbound",network="calibrationnet",use_fd="false"} 971
lotus_rcmgr_allow_conn{direction="outbound",network="calibrationnet",use_fd="true"} 982
lotus_rcmgr_allow_mem{network=""} 1311
lotus_rcmgr_allow_peer{network="calibrationnet",peer_id=""} 1963
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/f3/certexch/get/1/calibrationnet"} 10
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/fil/chain/xchg/0.0.1"} 26
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/fil/hello/1.0.0"} 244
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/fil/kad/calibrationnet/kad/1.0.0"} 230
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/ipfs/id/1.0.0"} 183
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/ipfs/id/push/1.0.0"} 149
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/ipfs/ping/1.0.0"} 22
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/libp2p/autonat/1.0.0"} 5
lotus_rcmgr_allow_proto{network="calibrationnet",proto="/meshsub/1.1.0"} 159
lotus_rcmgr_allow_stream{direction="inbound",network="calibrationnet",peer_id=""} 454
lotus_rcmgr_allow_stream{direction="outbound",network="calibrationnet",peer_id=""} 738
lotus_rcmgr_allow_svc{network="calibrationnet",svc="libp2p.autonat"} 5
lotus_rcmgr_allow_svc{network="calibrationnet",svc="libp2p.identify"} 317
lotus_rcmgr_allow_svc{network="calibrationnet",svc="libp2p.ping"} 22
lotus_rcmgr_block_peer{network="calibrationnet",peer_id=""} 11
lotus_splitstore_miss{network="calibrationnet"} 6.73232e+06
lotus_vm_applied{network=""} 52
lotus_vm_applyblocks_cron_bucket{network="",le="0.01"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="0.05"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="0.1"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="0.3"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="0.6"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="0.8"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="1"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="2"} 0
lotus_vm_applyblocks_cron_bucket{network="",le="3"} 3
lotus_vm_applyblocks_cron_bucket{network="",le="4"} 4
lotus_vm_applyblocks_cron_bucket{network="",le="5"} 5
lotus_vm_applyblocks_cron_bucket{network="",le="6"} 7
lotus_vm_applyblocks_cron_bucket{network="",le="8"} 9
lotus_vm_applyblocks_cron_bucket{network="",le="10"} 11
lotus_vm_applyblocks_cron_bucket{network="",le="13"} 11
lotus_vm_applyblocks_cron_bucket{network="",le="16"} 11
lotus_vm_applyblocks_cron_bucket{network="",le="20"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="25"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="30"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="40"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="50"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="65"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="80"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="100"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="130"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="160"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="200"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="250"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="300"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="400"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="500"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="650"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="800"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="1000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="2000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="3000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="4000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="5000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="7500"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="10000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="20000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="50000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="100000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="250000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="500000"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="1e+06"} 12
lotus_vm_applyblocks_cron_bucket{network="",le="+Inf"} 12
lotus_vm_applyblocks_cron_sum{network=""} 68.99999999999999
lotus_vm_applyblocks_cron_count{network=""} 12
lotus_vm_applyblocks_early_bucket{network="",le="0.01"} 11
lotus_vm_applyblocks_early_bucket{network="",le="0.05"} 11
lotus_vm_applyblocks_early_bucket{network="",le="0.1"} 11
lotus_vm_applyblocks_early_bucket{network="",le="0.3"} 11
lotus_vm_applyblocks_early_bucket{network="",le="0.6"} 11
lotus_vm_applyblocks_early_bucket{network="",le="0.8"} 11
lotus_vm_applyblocks_early_bucket{network="",le="1"} 11
lotus_vm_applyblocks_early_bucket{network="",le="2"} 11
lotus_vm_applyblocks_early_bucket{network="",le="3"} 11
lotus_vm_applyblocks_early_bucket{network="",le="4"} 11
lotus_vm_applyblocks_early_bucket{network="",le="5"} 11
lotus_vm_applyblocks_early_bucket{network="",le="6"} 11
lotus_vm_applyblocks_early_bucket{network="",le="8"} 12
lotus_vm_applyblocks_early_bucket{network="",le="10"} 12
lotus_vm_applyblocks_early_bucket{network="",le="13"} 12
lotus_vm_applyblocks_early_bucket{network="",le="16"} 12
lotus_vm_applyblocks_early_bucket{network="",le="20"} 12
lotus_vm_applyblocks_early_bucket{network="",le="25"} 12
lotus_vm_applyblocks_early_bucket{network="",le="30"} 12
lotus_vm_applyblocks_early_bucket{network="",le="40"} 12
lotus_vm_applyblocks_early_bucket{network="",le="50"} 12
lotus_vm_applyblocks_early_bucket{network="",le="65"} 12
lotus_vm_applyblocks_early_bucket{network="",le="80"} 12
lotus_vm_applyblocks_early_bucket{network="",le="100"} 12
lotus_vm_applyblocks_early_bucket{network="",le="130"} 12
lotus_vm_applyblocks_early_bucket{network="",le="160"} 12
lotus_vm_applyblocks_early_bucket{network="",le="200"} 12
lotus_vm_applyblocks_early_bucket{network="",le="250"} 12
lotus_vm_applyblocks_early_bucket{network="",le="300"} 12
lotus_vm_applyblocks_early_bucket{network="",le="400"} 12
lotus_vm_applyblocks_early_bucket{network="",le="500"} 12
lotus_vm_applyblocks_early_bucket{network="",le="650"} 12
lotus_vm_applyblocks_early_bucket{network="",le="800"} 12
lotus_vm_applyblocks_early_bucket{network="",le="1000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="2000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="3000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="4000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="5000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="7500"} 12
lotus_vm_applyblocks_early_bucket{network="",le="10000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="20000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="50000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="100000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="250000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="500000"} 12
lotus_vm_applyblocks_early_bucket{network="",le="1e+06"} 12
lotus_vm_applyblocks_early_bucket{network="",le="+Inf"} 12
lotus_vm_applyblocks_early_sum{network=""} 5.999999999999999
lotus_vm_applyblocks_early_count{network=""} 12
lotus_vm_applyblocks_flush_bucket{network="",le="0.01"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="0.05"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="0.1"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="0.3"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="0.6"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="0.8"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="1"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="2"} 11
lotus_vm_applyblocks_flush_bucket{network="",le="3"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="4"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="5"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="6"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="8"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="10"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="13"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="16"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="20"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="25"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="30"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="40"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="50"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="65"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="80"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="100"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="130"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="160"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="200"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="250"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="300"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="400"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="500"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="650"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="800"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="1000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="2000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="3000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="4000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="5000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="7500"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="10000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="20000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="50000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="100000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="250000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="500000"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="1e+06"} 12
lotus_vm_applyblocks_flush_bucket{network="",le="+Inf"} 12
lotus_vm_applyblocks_flush_sum{network=""} 2
lotus_vm_applyblocks_flush_count{network=""} 12
lotus_vm_applyblocks_messages_bucket{network="",le="0.01"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="0.05"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="0.1"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="0.3"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="0.6"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="0.8"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="1"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="2"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="3"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="4"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="5"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="6"} 0
lotus_vm_applyblocks_messages_bucket{network="",le="8"} 3
lotus_vm_applyblocks_messages_bucket{network="",le="10"} 5
lotus_vm_applyblocks_messages_bucket{network="",le="13"} 8
lotus_vm_applyblocks_messages_bucket{network="",le="16"} 10
lotus_vm_applyblocks_messages_bucket{network="",le="20"} 10
lotus_vm_applyblocks_messages_bucket{network="",le="25"} 10
lotus_vm_applyblocks_messages_bucket{network="",le="30"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="40"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="50"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="65"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="80"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="100"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="130"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="160"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="200"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="250"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="300"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="400"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="500"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="650"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="800"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="1000"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="2000"} 11
lotus_vm_applyblocks_messages_bucket{network="",le="3000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="4000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="5000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="7500"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="10000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="20000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="50000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="100000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="250000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="500000"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="1e+06"} 12
lotus_vm_applyblocks_messages_bucket{network="",le="+Inf"} 12
lotus_vm_applyblocks_messages_sum{network=""} 2847
lotus_vm_applyblocks_messages_count{network=""} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.01"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.05"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.1"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.3"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.6"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="0.8"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="1"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="2"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="3"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="4"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="5"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="6"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="8"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="10"} 0
lotus_vm_applyblocks_total_ms_bucket{network="",le="13"} 2
lotus_vm_applyblocks_total_ms_bucket{network="",le="16"} 4
lotus_vm_applyblocks_total_ms_bucket{network="",le="20"} 7
lotus_vm_applyblocks_total_ms_bucket{network="",le="25"} 9
lotus_vm_applyblocks_total_ms_bucket{network="",le="30"} 10
lotus_vm_applyblocks_total_ms_bucket{network="",le="40"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="50"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="65"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="80"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="100"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="130"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="160"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="200"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="250"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="300"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="400"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="500"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="650"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="800"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="1000"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="2000"} 11
lotus_vm_applyblocks_total_ms_bucket{network="",le="3000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="4000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="5000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="7500"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="10000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="20000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="50000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="100000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="250000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="500000"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="1e+06"} 12
lotus_vm_applyblocks_total_ms_bucket{network="",le="+Inf"} 12
lotus_vm_applyblocks_total_ms_sum{network=""} 2937
lotus_vm_applyblocks_total_ms_count{network=""} 12
lotus_vm_execution_running{lane="priority",network=""} 0
lotus_vm_execution_waiting{lane="priority",network=""} 0
lotus_vm_sends{network=""} 0

…er.go, and wdpost_changehandler.go

rvagg · 2024-12-12T05:30:01Z

paychmgr/manager.go

@@ -70,6 +73,10 @@ type Manager struct {

 func NewManager(ctx context.Context, shutdown func(), sm stmgr.StateManagerAPI, pchstore *Store, api PaychAPI) *Manager {
 	impl := &managerAPIImpl{StateManagerAPI: sm, PaychAPI: api}
+
+	// Add network tag to context
+	ctx, _ = tag.New(ctx, tag.Upsert(metrics.Network, buildconstants.NetworkBundle))


but where is this context coming from? do we need to track down the source if we know it's not being set here? I'd rather do this closer to the source if we can work it out

I'll attach the tag to the context directly here

lotus/node/modules/paych.go

Lines 17 to 22 in d463f4c

func NewManager(mctx helpers.MetricsCtx, lc fx.Lifecycle, sm stmgr.StateManagerAPI, pchstore *paychmgr.Store, api paychmgr.PaychAPI) *paychmgr.Manager {

ctx := helpers.LifecycleCtx(mctx, lc)

ctx, shutdown := context.WithCancel(ctx)

return paychmgr.NewManager(ctx, shutdown, sm, pchstore, api)

}

rvagg · 2024-12-12T05:30:31Z

storage/paths/db_index.go

@@ -46,6 +47,12 @@ func NewDBIndex(al *alerting.Alerting, db *harmonydb.DB) *DBIndex {
 	}
 }

+// addNetworkTag adds the network tag to the context for metrics
+func addNetworkTag(ctx context.Context) context.Context {


you could just put this in the metrics package since you're using it so much?

and make it public of course

rvagg · 2024-12-12T05:31:07Z

storage/paths/db_index.go

@@ -120,6 +127,8 @@ func splitString(str string) []string {
 }

 func (dbi *DBIndex) StorageAttach(ctx context.Context, si storiface.StorageInfo, st fsutil.FsStat) error {
+	ctx = addNetworkTag(ctx)


I'm still left with the question of where this context is coming from and whether we can fix it closer to creation

storage/wdpost/wdpost_changehandler.go

…k tagging across multiple modules

virajbhartiya · 2024-12-12T06:13:25Z

@rvagg I have created a single method to add the network tag and also updated the code to attach the tag in the parent method

…ttach method

add netwok tag to metrics

d3b150c

Enhance metrics by adding network tag to daemon and miner commands

747f087

virajbhartiya added 2 commits November 29, 2024 12:29

Update CHANGELOG

e420b87

minor fix

83a2bca

BigLep requested a review from rvagg December 3, 2024 08:41

BigLep assigned virajbhartiya Dec 3, 2024

rvagg reviewed Dec 4, 2024

View reviewed changes

cmd/lotus-miner/run.go Outdated Show resolved Hide resolved

rvagg reviewed Dec 4, 2024

View reviewed changes

cmd/lotus/daemon.go Outdated Show resolved Hide resolved

virajbhartiya and others added 2 commits December 4, 2024 17:08

Update cmd/lotus-miner/run.go

a80b86b

Co-authored-by: Rod Vagg <rod@vagg.org>

Update cmd/lotus/daemon.go

1ab14ea

Co-authored-by: Rod Vagg <rod@vagg.org>

rvagg reviewed Dec 5, 2024

View reviewed changes

cmd/lotus-miner/run.go Outdated Show resolved Hide resolved

BigLep mentioned this pull request Dec 6, 2024

chore(docs): archive old changelog entries #12757

Merged

virajbhartiya added 2 commits December 10, 2024 07:46

chore: remove unused import

3080368

Merge branch 'master' into network-tag

78e0bf5

feat(metrics): add network tagging to context across multiple files

be5d79a

refactor(imports): reorganize and clean up imports in rcmgr.go, manag…

65a02f6

…er.go, and wdpost_changehandler.go

rvagg reviewed Dec 12, 2024

View reviewed changes

storage/wdpost/wdpost_changehandler.go Outdated Show resolved Hide resolved

rvagg reviewed Dec 12, 2024

View reviewed changes

storage/wdpost/wdpost_changehandler.go Outdated Show resolved Hide resolved

feat(metrics): implement AddNetworkTag function for consistent networ…

775cfac

…k tagging across multiple modules

feat(metrics): update context tagging for network metrics in StorageA…

1244845

…ttach method

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add network tag to metrics #12733

feat: add network tag to metrics #12733

virajbhartiya commented Nov 28, 2024

virajbhartiya commented Nov 28, 2024

rvagg commented Nov 28, 2024

virajbhartiya commented Nov 29, 2024

Kubuxu commented Dec 4, 2024

rvagg commented Dec 5, 2024

rvagg commented Dec 5, 2024

virajbhartiya commented Dec 10, 2024 •

edited

Loading

rvagg commented Dec 11, 2024

rvagg commented Dec 11, 2024

rvagg commented Dec 11, 2024

virajbhartiya commented Dec 11, 2024

virajbhartiya commented Dec 11, 2024

rvagg Dec 12, 2024

virajbhartiya Dec 12, 2024

rvagg Dec 12, 2024

rvagg Dec 12, 2024

rvagg Dec 12, 2024

virajbhartiya commented Dec 12, 2024

	func NewManager(mctx helpers.MetricsCtx, lc fx.Lifecycle, sm stmgr.StateManagerAPI, pchstore paychmgr.Store, api paychmgr.PaychAPI) paychmgr.Manager {
	ctx := helpers.LifecycleCtx(mctx, lc)
	ctx, shutdown := context.WithCancel(ctx)

	return paychmgr.NewManager(ctx, shutdown, sm, pchstore, api)
	}

feat: add network tag to metrics #12733

Are you sure you want to change the base?

feat: add network tag to metrics #12733

Conversation

virajbhartiya commented Nov 28, 2024

virajbhartiya commented Nov 28, 2024

rvagg commented Nov 28, 2024

virajbhartiya commented Nov 29, 2024

Kubuxu commented Dec 4, 2024

rvagg commented Dec 5, 2024

rvagg commented Dec 5, 2024

virajbhartiya commented Dec 10, 2024 • edited Loading

rvagg commented Dec 11, 2024

rvagg commented Dec 11, 2024

rvagg commented Dec 11, 2024

virajbhartiya commented Dec 11, 2024

virajbhartiya commented Dec 11, 2024

rvagg Dec 12, 2024

Choose a reason for hiding this comment

virajbhartiya Dec 12, 2024

Choose a reason for hiding this comment

rvagg Dec 12, 2024

Choose a reason for hiding this comment

rvagg Dec 12, 2024

Choose a reason for hiding this comment

rvagg Dec 12, 2024

Choose a reason for hiding this comment

virajbhartiya commented Dec 12, 2024

virajbhartiya commented Dec 10, 2024 •

edited

Loading