NETOBSERV-1113 implements RTT option in Console Plugin #365

jpinsonneau · 2023-08-01T13:54:10Z

This PR will need #361 to manage FilterComponent.Number with more than capability.

Max / P90 / P99 followup:
https://issues.redhat.com/browse/NETOBSERV-1227

codecov · 2023-08-01T13:57:29Z

Codecov Report

Patch coverage: 50.40% and project coverage change: -0.86% ⚠️

Comparison is base (d0d7fc5) 57.67% compared to head (c20860f) 56.82%.
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #365      +/-   ##
==========================================
- Coverage   57.67%   56.82%   -0.86%     
==========================================
  Files         166      167       +1     
  Lines        7712     7865     +153     
  Branches      935      955      +20     
==========================================
+ Hits         4448     4469      +21     
- Misses       2993     3115     +122     
- Partials      271      281      +10

Flag	Coverage Δ
uitests	`57.65% <53.53%> (-0.34%)`	⬇️
unittests	`54.51% <38.46%> (-2.26%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
pkg/loki/flow_query.go	`66.10% <0.00%> (-1.73%)`	⬇️
pkg/model/fields/fields.go	`100.00% <ø> (ø)`
web/src/api/ipfix.ts	`100.00% <ø> (ø)`
web/src/components/__tests-data__/metrics.ts	`100.00% <ø> (ø)`
...components/dropdowns/topology-display-dropdown.tsx	`47.36% <ø> (ø)`
web/src/components/filters/filters-toolbar.tsx	`64.17% <ø> (ø)`
...onents/netflow-overview/netflow-overview-panel.tsx	`100.00% <ø> (ø)`
...ponents/netflow-topology/__tests-data__/metrics.ts	`100.00% <ø> (ø)`
web/src/model/config.ts	`100.00% <ø> (ø)`
web/src/model/filters.ts	`72.30% <ø> (ø)`
... and 21 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jotak · 2023-08-03T04:39:26Z

config/sample-frontend-config.yaml

@@ -22,9 +22,13 @@ quickFilters:
  - name: Services network
    filter:
      dst_kind: 'Service'
+  - name: DNS


hmm how does that work if the feature is off? Is it ignored?

That's just the sample. The real config comes from operator
https://github.com/netobserv/network-observability-operator/blob/7e7a9f88f1612354d3fa98cbb4d558f2bddc3b74/controllers/consoleplugin/consoleplugin_objects.go#L313-L318

jotak · 2023-08-03T05:03:28Z

pkg/loki/filter.go

+	intVal, _ := strconv.Atoi(value)
+	for i := 1; i < len(value); i++ {
+		nextMin := int((intVal / int(math.Pow10(i))) + 1)
+		nextMinStr := fmt.Sprintf("%d", nextMin)


Here and below: not a big deal as this wouldn't be frequently called, but strconv.Itoa or strconv.FormatInt are more efficient to convert ints to strings.

(PS: I've seen this commit is from the DNS PR, but since it's already lgtm you might want to change it here instead)

jotak · 2023-08-03T05:28:42Z

pkg/loki/topology_query.go

+	case "flowRtt":
+		f = "avg_over_time"
+		t = "TimeFlowRttNs"
+		factor = 0.000001 // nanoseconds to miliseconds


to avoid any floating point precision issue, it could be better to divide by an int rather than multiply by a decimal
Also, as long as it's only used to inject in a query (as string), that could be written right away as a string (if I'm correct we don't use the numerical value for anything) - that would also avoid that strings.Replace done below to get the right format for loki :-)

sure, a string would be better as it would keep both operator and value at once 👍

jpinsonneau · 2023-08-03T07:02:27Z

Rebased without extra changes

github-actions · 2023-08-03T07:11:43Z

New image:
quay.io/netobserv/network-observability-console-plugin:bb3b18a

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=bb3b18a make set-plugin-image

jotak

Code LGTM, will do some tests before approving

github-actions · 2023-08-24T09:00:58Z

New image:
quay.io/netobserv/network-observability-console-plugin:f6bf6ab

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=f6bf6ab make set-plugin-image

jotak · 2023-08-24T10:37:02Z

@jpinsonneau I get an error when I'm trying it:

This is due to a new input validation check that I added recently: https://github.com/netobserv/network-observability-console-plugin/blob/main/pkg/handler/validation.go#L92-L108
It needs to be updated with the new introduced metric type

jpinsonneau · 2023-08-24T11:35:05Z

@jpinsonneau I get an error when I'm trying it:

This is due to a new input validation check that I added recently: https://github.com/netobserv/network-observability-console-plugin/blob/main/pkg/handler/validation.go#L92-L108 It needs to be updated with the new introduced metric type

Fixed and rebased. Thanks for the check 👍

github-actions · 2023-08-24T12:15:20Z

New image:
quay.io/netobserv/network-observability-console-plugin:5364808

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=5364808 make set-plugin-image

jotak · 2023-08-24T12:42:33Z

Thanks Julien
re-tested, I have some few feedback:

In Overview, Donut: it shows "Ms", which would stand for "Megasecond" ... should be a low-case "ms"
Overview, Timeseries chart: Y axis should show the unit (ms)
Overview, Timeseries chart: it seems there's no tooltip?
Topology, Edge labels: it's also showing Ms instead of ms

On a more general note, I started testing with the default sampling, and this is really impractical as very few RTTs come up. After switching to sampling 1, it started to look good. I guess that's going to be a known issue, at least for some time, but we should probably document that (not only in the downstream doc, but also in the CRD doc). We can create a new task for that in the 1.5 epic.
Also, maybe also in the 1.5 epic, we could add a task to show things like p99 latencies rather than averages, which I think is more relevant to look at for SRE

PS: the issues with the timeseries chart are not blockers I think, if you prefer to keep that for the 1.5 epic; but the Ms/ms should be an easy win, right?

jpinsonneau · 2023-08-24T12:57:05Z

the issues with the timeseries chart are not blockers I think, if you prefer to keep that for the 1.5 epic; but the Ms/ms should be an easy win, right?

Sure let's do this: d321f52

We already have the MAX / P90 / P99 issue under the overview enhancement epic:
https://issues.redhat.com/browse/NETOBSERV-1227

We can put other items under the followup epic:
https://issues.redhat.com/browse/NETOBSERV-1225

jpinsonneau · 2023-08-25T13:35:18Z

We should mention in the UI that these metrics are TCP only. I suggest to add this in each graph title and tooltips.
Would that work for you guys @dushyantbehl & @msherif1234 ?

jotak · 2023-08-29T07:24:48Z

We should mention in the UI that these metrics are TCP only. I suggest to add this in each graph title and tooltips. Would that work for you guys @dushyantbehl & @msherif1234 ?

+1, even say "TCP handshakes" as this is limited to that

github-actions · 2023-08-29T09:15:13Z

New image:
quay.io/netobserv/network-observability-console-plugin:f37677f

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=f37677f make set-plugin-image

jotak

LGTM
Some feedback added to the next epic
@jpinsonneau feel free to implement your suggestion about the chart tooltips mentioning TCP

jotak · 2023-09-05T11:52:15Z

/approve

openshift-ci · 2023-09-05T11:52:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jotak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jotak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the do-not-merge/work-in-progress label Aug 1, 2023

jpinsonneau force-pushed the 1113 branch 2 times, most recently from 70047c4 to c7ba0a4 Compare August 2, 2023 15:08

jpinsonneau marked this pull request as ready for review August 2, 2023 15:09

openshift-ci bot removed the do-not-merge/work-in-progress label Aug 2, 2023

jpinsonneau requested review from jotak, msherif1234, OlivierCazade and dushyantbehl August 2, 2023 15:16

jotak reviewed Aug 3, 2023

View reviewed changes

openshift-merge-robot added the needs-rebase label Aug 3, 2023

jpinsonneau force-pushed the 1113 branch from 82f4591 to 0b17129 Compare August 3, 2023 07:01

openshift-merge-robot removed the needs-rebase label Aug 3, 2023

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 3, 2023

openshift-merge-robot added the needs-rebase label Aug 5, 2023

jpinsonneau force-pushed the 1113 branch from 0b17129 to 8a51781 Compare August 17, 2023 10:47

jpinsonneau requested a review from jotak August 17, 2023 10:47

openshift-merge-robot removed the needs-rebase label Aug 17, 2023

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 17, 2023

openshift-merge-robot added the needs-rebase label Aug 18, 2023

jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 24, 2023

jotak reviewed Aug 24, 2023

View reviewed changes

jotak mentioned this pull request Aug 24, 2023

NETOBSERV-1110: Enable support for Flow RTT netobserv/network-observability-operator#394

Merged

jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 24, 2023

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 24, 2023

openshift-merge-robot added the needs-rebase label Aug 24, 2023

jpinsonneau added 7 commits August 25, 2023 11:59

RTT column & filter

ddfa716

metrics query

0be98b4

overview & topology rtt

484c2d7

Itoa feedback

1084148

factor feedback

48eac9e

add missing rtt to allowed metricTypes

f21dfc5

lowercase ms

c20860f

jpinsonneau force-pushed the 1113 branch from d321f52 to c20860f Compare August 25, 2023 10:03

openshift-merge-robot removed the needs-rebase label Aug 25, 2023

jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Aug 29, 2023

jotak reviewed Aug 29, 2023

View reviewed changes

jotak self-requested a review August 29, 2023 09:37

jotak approved these changes Aug 29, 2023

View reviewed changes

openshift-ci bot assigned jotak Aug 29, 2023

openshift-ci bot added the lgtm label Aug 29, 2023

openshift-ci bot added the approved label Sep 5, 2023

openshift-merge-robot merged commit 967e4c8 into netobserv:main Sep 5, 2023
10 checks passed

jotak mentioned this pull request Sep 7, 2023

NETOBSERV-1113: Implement RTT option in Console Plugin #347

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-1113 implements RTT option in Console Plugin #365

NETOBSERV-1113 implements RTT option in Console Plugin #365

jpinsonneau commented Aug 1, 2023 •

edited

Loading

codecov bot commented Aug 1, 2023 •

edited

Loading

jotak Aug 3, 2023

jpinsonneau Aug 3, 2023

jotak Aug 3, 2023

jpinsonneau Aug 3, 2023

jotak Aug 3, 2023

jpinsonneau Aug 3, 2023

jpinsonneau Aug 3, 2023

jpinsonneau commented Aug 3, 2023

github-actions bot commented Aug 3, 2023

jotak left a comment

github-actions bot commented Aug 24, 2023

jotak commented Aug 24, 2023

jpinsonneau commented Aug 24, 2023

github-actions bot commented Aug 24, 2023

jotak commented Aug 24, 2023 •

edited

Loading

jpinsonneau commented Aug 24, 2023

jpinsonneau commented Aug 25, 2023

jotak commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

jotak left a comment

jotak commented Sep 5, 2023

openshift-ci bot commented Sep 5, 2023

NETOBSERV-1113 implements RTT option in Console Plugin #365

NETOBSERV-1113 implements RTT option in Console Plugin #365

Conversation

jpinsonneau commented Aug 1, 2023 • edited Loading

codecov bot commented Aug 1, 2023 • edited Loading

Codecov Report

jotak Aug 3, 2023

Choose a reason for hiding this comment

jpinsonneau Aug 3, 2023

Choose a reason for hiding this comment

jotak Aug 3, 2023

Choose a reason for hiding this comment

jpinsonneau Aug 3, 2023

Choose a reason for hiding this comment

jotak Aug 3, 2023

Choose a reason for hiding this comment

jpinsonneau Aug 3, 2023

Choose a reason for hiding this comment

jpinsonneau Aug 3, 2023

Choose a reason for hiding this comment

jpinsonneau commented Aug 3, 2023

github-actions bot commented Aug 3, 2023

jotak left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 24, 2023

jotak commented Aug 24, 2023

jpinsonneau commented Aug 24, 2023

github-actions bot commented Aug 24, 2023

jotak commented Aug 24, 2023 • edited Loading

jpinsonneau commented Aug 24, 2023

jpinsonneau commented Aug 25, 2023

jotak commented Aug 29, 2023

github-actions bot commented Aug 29, 2023

jotak left a comment

Choose a reason for hiding this comment

jotak commented Sep 5, 2023

openshift-ci bot commented Sep 5, 2023

jpinsonneau commented Aug 1, 2023 •

edited

Loading

codecov bot commented Aug 1, 2023 •

edited

Loading

jotak commented Aug 24, 2023 •

edited

Loading