NETOBSERV-1427: Added config watcher and supporting metrics reloading #607

OlivierCazade · 2024-02-22T15:13:39Z

Description

Added new type to watch a configmap and notify prometheus encoder

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

openshift-ci-robot · 2024-02-22T15:13:43Z

jotak · 2024-02-27T08:43:06Z

pkg/pipeline/encode/encode_prom.go

+	}
+}
+
+func (e *EncodeProm) cleanDeletedCounters(newCfg api.PromEncode) {


after rebasing this code can probably be simplified, with all cleanup functions merged in a single one, as Kalman did use a common struct for all the metric types

jotak · 2024-02-27T08:52:20Z

pkg/pipeline/encode/encode_prom.go

+			switch mCfg.Type {
+			case api.MetricEncodeOperationName("Counter"):
+				if oldMetric, ok := e.counters[fullMetricName]; ok {
+					if !reflect.DeepEqual(mInfo, oldMetric.info) {


I'm not sure if that works as intended, as info contains a predicate (a function) so they'll likely never be equal: https://pkg.go.dev/reflect#DeepEqual
You could test just the api definition oldMetric.info.MetricsItem

pkg/pipeline/pipeline.go

jotak · 2024-02-27T09:04:58Z

pkg/pipeline/pipeline_watcher.go

+func (pcw *pipelineConfigWatcher) updateFromConfigmap(cm *corev1.ConfigMap) {
+	if rawConfig, ok := cm.Data[pcw.configFile]; ok {
+		config := config.ConfigHotReloadStruct{}
+		err := json.Unmarshal([]byte(rawConfig), config)


govet messages below seem relevant, I guess Unmarshalling would fail here

KalmanMeth · 2024-03-05T07:57:33Z

pkg/pipeline/encode/encode_prom.go

+
+}
+
+func (e *EncodeProm) cleandDeletedMetrics(newCfg api.PromEncode) {


Perhaps cleanDeletedMetrics (without the extra d)?

codecov · 2024-03-14T13:41:28Z

Codecov Report

Attention: Patch coverage is 13.96104% with 265 lines in your changes are missing coverage. Please review.

Project coverage is 65.11%. Comparing base (d2b2352) to head (3d16544).
Report is 2 commits behind head on main.

❗ Current head 3d16544 differs from pull request most recent head 2e05954. Consider uploading reports for the commit 2e05954 to get more accurate results

Files	Patch %	Lines
pkg/pipeline/encode/encode_prom.go	6.32%	163 Missing ⚠️
pkg/pipeline/pipeline_watcher.go	8.95%	61 Missing ⚠️
pkg/utils/kubernetes.go	0.00%	22 Missing ⚠️
pkg/pipeline/encode/encode_s3.go	0.00%	4 Missing ⚠️
...ipeline/encode/opentelemetry/encode_otlpmetrics.go	40.00%	3 Missing ⚠️
pkg/pipeline/pipeline.go	66.66%	2 Missing and 1 partial ⚠️
pkg/pipeline/encode/encode.go	0.00%	2 Missing ⚠️
pkg/pipeline/encode/encode_kafka.go	0.00%	2 Missing ⚠️
...g/pipeline/encode/opentelemetry/encode_otlplogs.go	0.00%	2 Missing ⚠️
.../pipeline/encode/opentelemetry/encode_otlptrace.go	0.00%	2 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #607      +/-   ##
==========================================
- Coverage   67.41%   65.11%   -2.31%     
==========================================
  Files         102      112      +10     
  Lines        7589     7940     +351     
==========================================
+ Hits         5116     5170      +54     
- Misses       2162     2456     +294     
- Partials      311      314       +3

Flag	Coverage Δ
unittests	`65.11% <13.96%> (-2.31%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codecov-commenter · 2024-03-25T15:53:51Z

Codecov Report

Attention: Patch coverage is 15.86716% with 228 lines in your changes are missing coverage. Please review.

Project coverage is 66.49%. Comparing base (a5ad247) to head (c306f62).
Report is 5 commits behind head on main.

Files	Patch %	Lines
pkg/pipeline/encode/encode_prom.go	10.00%	90 Missing ⚠️
pkg/pipeline/pipeline_watcher.go	8.47%	54 Missing ⚠️
pkg/pipeline/pipeline_builder.go	22.85%	26 Missing and 1 partial ⚠️
pkg/utils/kubernetes.go	0.00%	17 Missing ⚠️
pkg/config/pipeline_builder.go	0.00%	16 Missing ⚠️
pkg/config/config.go	0.00%	5 Missing and 1 partial ⚠️
pkg/pipeline/encode/encode_s3.go	0.00%	4 Missing ⚠️
...ipeline/encode/opentelemetry/encode_otlpmetrics.go	40.00%	3 Missing ⚠️
pkg/pipeline/encode/encode.go	0.00%	2 Missing ⚠️
pkg/pipeline/encode/encode_kafka.go	0.00%	2 Missing ⚠️
... and 4 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #607      +/-   ##
==========================================
- Coverage   67.43%   66.49%   -0.94%     
==========================================
  Files         102      104       +2     
  Lines        7593     6659     -934     
==========================================
- Hits         5120     4428     -692     
+ Misses       2162     1916     -246     
- Partials      311      315       +4

Flag	Coverage Δ
unittests	`66.49% <15.86%> (-0.94%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

OlivierCazade · 2024-04-16T10:14:21Z

/retest

jotak · 2024-04-19T13:43:24Z

thanks @OlivierCazade !
/lgtm

memodi · 2024-04-24T18:14:50Z

/label qe-approved

openshift-ci-robot · 2024-04-24T18:14:55Z

OlivierCazade · 2024-04-26T11:17:11Z

/approve

openshift-ci · 2024-04-26T11:17:33Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: OlivierCazade

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [OlivierCazade]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the jira/valid-reference label Feb 22, 2024

openshift-ci bot added the do-not-merge/work-in-progress label Feb 22, 2024

OlivierCazade requested review from jotak and jpinsonneau February 22, 2024 15:40

jotak reviewed Feb 27, 2024

View reviewed changes

pkg/pipeline/pipeline.go Show resolved Hide resolved

jotak reviewed Feb 27, 2024

View reviewed changes

openshift-merge-robot added the needs-rebase label Feb 28, 2024

KalmanMeth reviewed Mar 5, 2024

View reviewed changes

OlivierCazade force-pushed the 1427-hotreload branch from 7f972e1 to 60388fb Compare March 14, 2024 12:17

openshift-merge-robot removed the needs-rebase label Mar 14, 2024

OlivierCazade force-pushed the 1427-hotreload branch 3 times, most recently from cc471df to 3d16544 Compare March 14, 2024 13:32

openshift-merge-robot added the needs-rebase label Mar 17, 2024

OlivierCazade force-pushed the 1427-hotreload branch from 3d16544 to 943bc9f Compare March 22, 2024 10:38

openshift-merge-robot removed the needs-rebase label Mar 22, 2024

OlivierCazade force-pushed the 1427-hotreload branch from 943bc9f to 4750c83 Compare March 22, 2024 13:29

openshift-merge-robot added the needs-rebase label Mar 22, 2024

OlivierCazade force-pushed the 1427-hotreload branch from 4750c83 to 2e05954 Compare March 25, 2024 14:43

openshift-merge-robot removed the needs-rebase label Mar 25, 2024

OlivierCazade force-pushed the 1427-hotreload branch from 2e05954 to 387a8ce Compare March 25, 2024 15:53

OlivierCazade mentioned this pull request Apr 4, 2024

NETOBSERV-1427 Added support to FLP dynamic configmap netobserv/network-observability-operator#610

Merged

10 tasks

OlivierCazade force-pushed the 1427-hotreload branch 3 times, most recently from 6564c33 to 69fe1b7 Compare April 4, 2024 11:51

OlivierCazade force-pushed the 1427-hotreload branch 4 times, most recently from ca4a86c to 64d701b Compare April 4, 2024 14:32

OlivierCazade changed the title ~~WIP NETOBSERV-1427: Added config watcher and supporting metrics reloading~~ NETOBSERV-1427: Added config watcher and supporting metrics reloading Apr 19, 2024

openshift-ci bot removed the do-not-merge/work-in-progress label Apr 19, 2024

OlivierCazade requested a review from jotak April 19, 2024 08:33

Added config watcher and supporting metrics reloading

c306f62

OlivierCazade force-pushed the 1427-hotreload branch from 64d701b to c306f62 Compare April 19, 2024 13:31

openshift-ci bot assigned jotak Apr 19, 2024

openshift-ci bot added the lgtm label Apr 19, 2024

openshift-ci bot added the qe-approved QE has approved this pull request label Apr 24, 2024

openshift-ci bot added the approved label Apr 26, 2024

openshift-merge-bot bot merged commit 2ab1f8d into netobserv:main Apr 26, 2024
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-1427: Added config watcher and supporting metrics reloading #607

NETOBSERV-1427: Added config watcher and supporting metrics reloading #607

OlivierCazade commented Feb 22, 2024

openshift-ci-robot commented Feb 22, 2024 •

edited by openshift-ci bot

Loading

Description

Dependencies

Checklist

jotak Feb 27, 2024

jotak Feb 27, 2024

jotak Feb 27, 2024

KalmanMeth Mar 5, 2024

codecov bot commented Mar 14, 2024 •

edited

Loading

codecov-commenter commented Mar 25, 2024 •

edited

Loading

OlivierCazade commented Apr 16, 2024

jotak commented Apr 19, 2024

memodi commented Apr 24, 2024

openshift-ci-robot commented Apr 24, 2024 •

edited by openshift-ci bot

Loading

Description

Dependencies

Checklist

OlivierCazade commented Apr 26, 2024

openshift-ci bot commented Apr 26, 2024


		}

		func (e *EncodeProm) cleandDeletedMetrics(newCfg api.PromEncode) {

NETOBSERV-1427: Added config watcher and supporting metrics reloading #607

NETOBSERV-1427: Added config watcher and supporting metrics reloading #607

Conversation

OlivierCazade commented Feb 22, 2024

Description

Dependencies

Checklist

openshift-ci-robot commented Feb 22, 2024 • edited by openshift-ci bot Loading

Description

Dependencies

Checklist

jotak Feb 27, 2024

Choose a reason for hiding this comment

jotak Feb 27, 2024

Choose a reason for hiding this comment

jotak Feb 27, 2024

Choose a reason for hiding this comment

KalmanMeth Mar 5, 2024

Choose a reason for hiding this comment

codecov bot commented Mar 14, 2024 • edited Loading

Codecov Report

codecov-commenter commented Mar 25, 2024 • edited Loading

Codecov Report

OlivierCazade commented Apr 16, 2024

jotak commented Apr 19, 2024

memodi commented Apr 24, 2024

openshift-ci-robot commented Apr 24, 2024 • edited by openshift-ci bot Loading

Description

Dependencies

Checklist

OlivierCazade commented Apr 26, 2024

openshift-ci bot commented Apr 26, 2024

openshift-ci-robot commented Feb 22, 2024 •

edited by openshift-ci bot

Loading

codecov bot commented Mar 14, 2024 •

edited

Loading

codecov-commenter commented Mar 25, 2024 •

edited

Loading

openshift-ci-robot commented Apr 24, 2024 •

edited by openshift-ci bot

Loading