Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: switch prom remote write to metric engine #3198

Merged
merged 14 commits into from
Jan 22, 2024

Conversation

waynexia
Copy link
Member

@waynexia waynexia commented Jan 19, 2024

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Run PromStore with metric engine.

All tables created from prom remote write will use metric engine automatically if the config [prom_store].with_metric_engine is set to true. This PR adds a new parameter to prometheus request URL physical-table for prom remote write to specify which physical table to use. And the default physical table name is greptime_physical_table.

An example of this URL is http://172.17.0.1:4000/v1/prometheus/write?physical_table=my_table_name

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@github-actions github-actions bot added docs-not-required This change does not impact docs. Size: M labels Jan 19, 2024
Copy link

codecov bot commented Jan 19, 2024

Codecov Report

Attention: 49 lines in your changes are missing coverage. Please review.

Comparison is base (2bf4b08) 85.82% compared to head (e9ad47f) 85.44%.
Report is 1 commits behind head on main.

❗ Current head e9ad47f differs from pull request most recent head 20bf1bc. Consider uploading reports for the commit 20bf1bc to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3198      +/-   ##
==========================================
- Coverage   85.82%   85.44%   -0.38%     
==========================================
  Files         840      840              
  Lines      137704   137920     +216     
==========================================
- Hits       118183   117849     -334     
- Misses      19521    20071     +550     

@killme2008 killme2008 changed the title feat: switch prom remote write to metric engine feat!: switch prom remote write to metric engine Jan 19, 2024
@github-actions github-actions bot added the breaking-change This pull request contains breaking changes. label Jan 19, 2024
@killme2008
Copy link
Contributor

killme2008 commented Jan 19, 2024

What about using a query parameter instead of an HTTP header?

remote_write:
  - url: "http://localhost:4000/v1/prometheus/write?physical_table=my_physical_table"

It doesn't feel right as an HTTP header; it's actually a request parameter.

src/frontend/src/instance/prom_store.rs Outdated Show resolved Hide resolved
src/common/query/src/prelude.rs Show resolved Hide resolved
src/frontend/src/instance/grpc.rs Outdated Show resolved Hide resolved
src/frontend/src/instance/prom_store.rs Outdated Show resolved Hide resolved
src/operator/src/insert.rs Show resolved Hide resolved
src/operator/src/insert.rs Outdated Show resolved Hide resolved
src/operator/src/insert.rs Outdated Show resolved Hide resolved
src/operator/src/insert.rs Outdated Show resolved Hide resolved
src/servers/src/http/authorize.rs Show resolved Hide resolved
waynexia and others added 4 commits January 21, 2024 16:08
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@waynexia
Copy link
Member Author

BTW this is not a breaking change. It won't affect existing remote write. Only new metrics will be created in metric engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@killme2008
Copy link
Contributor

BTW this is not a breaking change. It won't affect existing remote write. Only new metrics will be created in metric engine

From the user's perspective, the behavior is changed after upgrading. And if we don't provide a default physical table name, the users must change theirs Prometheus setting.

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@waynexia waynexia enabled auto-merge January 22, 2024 06:36
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@killme2008 killme2008 disabled auto-merge January 22, 2024 14:08
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@killme2008 killme2008 enabled auto-merge January 22, 2024 14:11
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@killme2008 killme2008 added this pull request to the merge queue Jan 22, 2024
Merged via the queue into GreptimeTeam:main with commit 31787f4 Jan 22, 2024
20 checks passed
@waynexia waynexia deleted the prom-store-on-phy-table branch January 23, 2024 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change This pull request contains breaking changes. docs-not-required This change does not impact docs.
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

3 participants