Prometheus Remote Read ETL tool #1014

joshmeranda · 2023-02-02T13:50:00Z

OEP: #970

kralicky

Looks good so far. Couple of things to do before we can merge:

There are a lot of todos, you should attempt to resolve as many as possible (all the ones that aren't blocked by other things)
There are a couple of style related comments, do a once over through all the new code and make sure code style stays consistent.

pkg/opni/commands/setup_import.go

pkg/opni/commands/import_progress.go

pkg/opni/util/render.go

pkg/plugins/apis/apiextensions/stream/delegate.go

plugins/metrics/pkg/agent/discovery.go

plugins/metrics/pkg/agent/runner.go

plugins/metrics/pkg/backend/metrics.go

joshmeranda · 2023-02-02T21:07:43Z

Sounds good, adding a few more tests I thought of while writing docs, so I'll add those, and start addressing your comments. Thanks!

plugins/metrics/pkg/agent/http.go

kralicky

Might also need to rebase

pkg/gateway/delegate.go

pkg/opni/commands/import.go

plugins/metrics/pkg/agent/node.go

plugins/metrics/pkg/apis/remoteread/client.go

plugins/metrics/pkg/backend/metrics.go

plugins/metrics/pkg/cortex/api.go

plugins/metrics/pkg/agent/node.go

…gents # Conflicts: # go.mod # go.sum

# Conflicts: # go.mod # go.sum

# Conflicts: # pkg/opni/util/render.go

# Conflicts: # go.sum

joshmeranda requested review from kralicky and alexandreLamarre February 2, 2023 13:50

kralicky requested changes Feb 2, 2023

View reviewed changes

This was referenced Feb 3, 2023

[ENHANCEMENT] Rancher Monitoring ETl Tool #1024

Closed

Delegate Communcation Streams #1025

Open

joshmeranda requested a review from kralicky February 8, 2023 12:35

alexandreLamarre reviewed Feb 8, 2023

View reviewed changes

plugins/metrics/pkg/agent/http.go Outdated Show resolved Hide resolved

kralicky requested changes Feb 8, 2023

View reviewed changes

joshmeranda requested a review from kralicky February 9, 2023 14:27

kralicky reviewed Feb 9, 2023

View reviewed changes

plugins/metrics/pkg/agent/node.go Outdated Show resolved Hide resolved

kralicky previously approved these changes Feb 9, 2023

View reviewed changes

joshmeranda added 18 commits February 9, 2023 14:23

add cli commands

ef49556

basic target path impl

1b70c74

impl agent remote read

e2e32c0

add gateway logic

339cc1c

refractor target

3a97a51

refractor target

38b3d15

add support for target status

7acccaf

ready for demo

3ffebd3

code cleanup after demo

93ac042

better progress typing

dfddd3e

add progress monitoring to import cli

cacec5c

Added new stream delegate api to enable direct requests to specific a…

37c1e65

…gents # Conflicts: # go.mod # go.sum

initial stream delegate integration

88a0309

allow for routing from gateway to agent

665d882

agent target optimizations

00af040

add target discovery

de9518c

# Conflicts: # go.mod # go.sum

move client code onto client side

ab9a699

fix delegate response handling flow

8671ec1

joshmeranda added 20 commits February 9, 2023 15:08

remove redundant import/management clients

d5720b5

add broadcast response aggregator

347b14b

fixes to broadcast aggregation

0967f28

small broadcast fixes

2d542ff

fix target listing issues

55aaf01

add target runner tests

247e6aa

add discovery tests

01263af

final fixes

8b65223

fix typos

094c0e6

fix silly race condition

47c1731

remove unused value

eab3322

add more tests

5b2947f

address PR comments

594cfa1

# Conflicts: # pkg/opni/util/render.go

move discovery logic into external_operator

085fd97

use pkg/task framework in target runner

d7b837e

finalize tests

ac0d285

# Conflicts: # go.sum

remove uneeded commented funcs

6279ec4

code cleanup and command completion

c67107a

refactoring

5bc502e

dependency inject MetricsNodeDriver

ecbc812

joshmeranda dismissed kralicky’s stale review via ecbc812 February 9, 2023 20:29

joshmeranda force-pushed the etl branch from 25edcca to ecbc812 Compare February 9, 2023 20:29

kralicky previously approved these changes Feb 9, 2023

View reviewed changes

satisfy linter

6f1e119

joshmeranda dismissed kralicky’s stale review via 6f1e119 February 10, 2023 15:09

satisfy CI issues

42a0e9f

joshmeranda force-pushed the etl branch from ad50442 to 42a0e9f Compare February 13, 2023 15:18

joshmeranda merged commit 28268dc into main Feb 13, 2023

joshmeranda mentioned this pull request Feb 14, 2023

fix minimal build issue #1066

Merged

kralicky deleted the etl branch March 20, 2023 19:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prometheus Remote Read ETL tool #1014

Prometheus Remote Read ETL tool #1014

joshmeranda commented Feb 2, 2023

kralicky left a comment

joshmeranda commented Feb 2, 2023

kralicky left a comment

Prometheus Remote Read ETL tool #1014

Prometheus Remote Read ETL tool #1014

Conversation

joshmeranda commented Feb 2, 2023

kralicky left a comment

Choose a reason for hiding this comment

joshmeranda commented Feb 2, 2023

kralicky left a comment

Choose a reason for hiding this comment