chore: add http metrics server in datanode node when greptime start in distributed mode #1256

paomian · 2023-03-27T09:46:25Z

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Unify disorganized metrics into a single file
start http server in datanode instance to export prometheus metrics when start in distributed mode

Checklist

I have written the necessary rustdoc comments.
I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

codecov · 2023-03-27T11:44:08Z

Codecov Report

Merging #1256 (207b3ac) into develop (eb77f9a) will decrease coverage by 0.51%.
The diff coverage is 62.63%.

@@             Coverage Diff             @@
##           develop    #1256      +/-   ##
===========================================
- Coverage    85.91%   85.41%   -0.51%     
===========================================
  Files          499      500       +1     
  Lines        75599    75690      +91     
===========================================
- Hits         64950    64649     -301     
- Misses       10649    11041     +392

src/common/telemetry/src/metrics.rs

src/cmd/src/metasrv.rs

src/servers/src/metrics_server.rs

MichaelScofield

Can we add a grpc interface to expose metrics in datanode, instead of creating a http server?

src/common/telemetry/src/metrics.rs

src/common/telemetry/Cargo.toml

waynexia · 2023-03-28T07:12:01Z

Can we add a grpc interface to expose metrics in datanode, instead of creating a http server?

metrics's expose path usually defaults to http://endpoint/metrics. I think we'd better to do so

MichaelScofield

If some comments are hard to resolve or not related to this PR's scope, feel free to add "todo"s or create issues that tagged with "good first issue".

src/datanode/src/server.rs

src/meta-srv/src/bootstrap.rs

src/meta-srv/src/error.rs

src/servers/src/http.rs

src/servers/src/metrics_handler.rs

src/datanode/src/server.rs

src/common/runtime/src/runtime.rs

src/meta-srv/src/metasrv.rs

src/meta-srv/src/error.rs

src/servers/src/http.rs

paomian · 2023-03-31T06:53:37Z

PTAL @evenyag @MichaelScofield

…n distributed mode

… config

evenyag

LGTM

…n distributed mode (GreptimeTeam#1256) * chore: add http metrics server in datanode node when greptime start in distributed mode * chore: add some docs and license * chore: change metrics_addr to resolve address already in use error * chore add metrics for meta service * chore: replace metrics exporter http server from hyper to axum * chore: format * fix: datanode mode branching error * fix: sqlness test address already in use and start metrics in defualt config * chore: change metrics location * chore: use builder pattern to builder httpserver * chore: remove useless debug_assert macro in httpserver builder * chore: resolve conflicting build error * chore: format code

paomian changed the title ~~chore: add http metrics server in datanode node when greptime start i…~~ chore: add http metrics server in datanode node when greptime start in distributed mode Mar 27, 2023

paomian force-pushed the chore/metrics branch 2 times, most recently from 09c4b1e to 9630d0a Compare March 27, 2023 11:19

paomian force-pushed the chore/metrics branch 2 times, most recently from d571dac to b1beccb Compare March 27, 2023 13:48

sunng87 linked an issue Mar 28, 2023 that may be closed by this pull request

Add prometheus metrics endpoint for datanode and meta #1254

Closed

evenyag reviewed Mar 28, 2023

View reviewed changes

src/common/telemetry/src/metrics.rs Outdated Show resolved Hide resolved

src/cmd/src/metasrv.rs Outdated Show resolved Hide resolved

src/servers/src/metrics_server.rs Outdated Show resolved Hide resolved

MichaelScofield reviewed Mar 28, 2023

View reviewed changes

src/common/telemetry/src/metrics.rs Outdated Show resolved Hide resolved

src/common/telemetry/Cargo.toml Show resolved Hide resolved

paomian force-pushed the chore/metrics branch from 98b2527 to 3ef17c1 Compare March 28, 2023 09:01

paomian requested review from MichaelScofield and evenyag March 28, 2023 12:20

MichaelScofield reviewed Mar 29, 2023

View reviewed changes

evenyag reviewed Mar 29, 2023

View reviewed changes

src/datanode/src/server.rs Outdated Show resolved Hide resolved

src/common/runtime/src/runtime.rs Outdated Show resolved Hide resolved

src/meta-srv/src/metasrv.rs Outdated Show resolved Hide resolved

paomian force-pushed the chore/metrics branch 5 times, most recently from a7b2fde to 9b96ee6 Compare March 30, 2023 06:30

paomian requested review from evenyag and MichaelScofield March 30, 2023 07:19

MichaelScofield approved these changes Mar 30, 2023

View reviewed changes

src/meta-srv/src/error.rs Outdated Show resolved Hide resolved

src/meta-srv/src/error.rs Outdated Show resolved Hide resolved

src/servers/src/http.rs Outdated Show resolved Hide resolved

paomian force-pushed the chore/metrics branch from 9b96ee6 to 00eee8d Compare March 31, 2023 03:41

paomian requested a review from MichaelScofield March 31, 2023 06:31

paomian added 4 commits March 31, 2023 15:44

chore: add http metrics server in datanode node when greptime start i…

7c5105a

…n distributed mode

chore: add some docs and license

d7dff6d

chore: change metrics_addr to resolve address already in use error

2a12605

chore add metrics for meta service

8bb4b36

paomian added 9 commits March 31, 2023 15:46

chore: replace metrics exporter http server from hyper to axum

f4b4549

chore: format

87236ec

fix: datanode mode branching error

a4d4c0b

fix: sqlness test address already in use and start metrics in defualt…

392c28f

… config

chore: change metrics location

f051060

chore: use builder pattern to builder httpserver

205df01

chore: remove useless debug_assert macro in httpserver builder

9cf38cd

chore: resolve conflicting build error

89313a8

chore: format code

207b3ac

paomian force-pushed the chore/metrics branch from 00eee8d to 207b3ac Compare March 31, 2023 07:55

evenyag approved these changes Mar 31, 2023

View reviewed changes

evenyag merged commit a2b262e into GreptimeTeam:develop Mar 31, 2023

MichaelScofield mentioned this pull request Apr 6, 2023

Dashbord service should not startup in metasrv and datanode #1334

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: add http metrics server in datanode node when greptime start in distributed mode #1256

chore: add http metrics server in datanode node when greptime start in distributed mode #1256

paomian commented Mar 27, 2023 •

edited

Loading

codecov bot commented Mar 27, 2023 •

edited

Loading

MichaelScofield left a comment

waynexia commented Mar 28, 2023

MichaelScofield left a comment

paomian commented Mar 31, 2023

evenyag left a comment

chore: add http metrics server in datanode node when greptime start in distributed mode #1256

chore: add http metrics server in datanode node when greptime start in distributed mode #1256

Conversation

paomian commented Mar 27, 2023 • edited Loading

What's changed and what's your intention?

Checklist

Refer to a related PR or issue link (optional)

codecov bot commented Mar 27, 2023 • edited Loading

Codecov Report

MichaelScofield left a comment

Choose a reason for hiding this comment

waynexia commented Mar 28, 2023

MichaelScofield left a comment

Choose a reason for hiding this comment

paomian commented Mar 31, 2023

evenyag left a comment

Choose a reason for hiding this comment

paomian commented Mar 27, 2023 •

edited

Loading

codecov bot commented Mar 27, 2023 •

edited

Loading