NETOBSERV-1190: update dns tracker to calc latency instead of TS #149

msherif1234 · 2023-07-13T18:42:43Z

add new LRU map to correlate DNS Q/R and calculate TS
manual testing

]$ dig google.com

;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30678
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             224     IN      A       172.253.122.139
google.com.             224     IN      A       172.253.122.113
google.com.             224     IN      A       172.253.122.101
google.com.             224     IN      A       172.253.122.102
google.com.             224     IN      A       172.253.122.100
google.com.             224     IN      A       172.253.122.138

;; Query time: 21 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri Jul 14 08:31:27 EDT 2023
;; MSG SIZE  rcvd: 135

Then use flows_logs collector

ipv4: 08:31:27.947192 unknown IP 127.0.0.53:53 > 127.0.0.1:49033: protocol:udp type: 0 code: 0 dir:0 bytes:177 packets:1 flags:0 ends: 08:31:27.947192 dnsId: 30678 dnsFlags: 0x8180 dnsLatency(ms): 21 rtt 0

test on cluster

{
  "AgentIP": "10.0.128.2",
  "Bytes": 137,
  "DnsFlags": 33152,
  "DnsFlagsResponseCode": "NoError",
  "DnsId": 29212,
  "DnsLatencyMs": 4,
  "DstAddr": "10.131.0.8",
  "DstK8S_HostIP": "10.0.128.2",
  "DstK8S_HostName": "ci-ln-mv3kgpt-72292-vcr5x-worker-a-2crf4",
  "DstK8S_Name": "dns-default-wcrv6",
  "DstK8S_Namespace": "openshift-dns",
  "DstK8S_OwnerName": "dns-default",
  "DstK8S_OwnerType": "DaemonSet",
  "DstK8S_Type": "Pod",
  "DstMac": "0A:58:0A:83:00:08",
  "DstPort": 54476,
  "Duplicate": false,
  "Etype": 2048,
  "FlowDirection": "0",
  "IfDirection": 0,
  "Interface": "unknown",
  "Packets": 1,
  "Proto": 17,
  "SrcAddr": "169.254.169.254",
  "SrcMac": "0A:58:0A:83:00:01",
  "SrcPort": 53,
  "TimeFlowEndMs": 1689336618135,
  "TimeFlowStartMs": 1689336618135,
  "TimeReceived": 1689336618,
  "app": "netobserv-flowcollector"
}

openshift-ci-robot · 2023-07-13T18:42:47Z

@msherif1234: This pull request references NETOBSERV-1190 which is a valid jira issue.

In response to this:

add new LRU map to correlate DNS Q/R and calculate TS

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-07-13T18:47:27Z

@msherif1234: This pull request references NETOBSERV-1190 which is a valid jira issue.

In response to this:

add new LRU map to correlate DNS Q/R and calculate TS
manual testing

]$ dig google.com

; <<>> DiG 9.16.33-RH <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15672
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             194     IN      A       172.253.122.113
google.com.             194     IN      A       172.253.122.100
google.com.             194     IN      A       172.253.122.139
google.com.             194     IN      A       172.253.122.138
google.com.             194     IN      A       172.253.122.101
google.com.             194     IN      A       172.253.122.102

;; Query time: 132 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Thu Jul 13 14:34:28 EDT 2023
;; MSG SIZE  rcvd: 135

Then use flows_logs collector

ipv4: 14:34:28.458790 unknown IP 127.0.0.53:53 > 127.0.0.1:43613: protocol:udp type: 0 code: 0 dir:0 bytes:177 packets:1 flags:0 ends: 14:34:28.458790 dnsId: 15672 dnsFlags: 0x8180 dnsLatency: nanos:131602039 rtt 0

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2023-07-13T18:49:38Z

Codecov Report

Merging #149 (e5a3902) into main (1045794) will increase coverage by 0.24%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #149      +/-   ##
==========================================
+ Coverage   39.00%   39.25%   +0.24%     
==========================================
  Files          31       31              
  Lines        2238     2214      -24     
==========================================
- Hits          873      869       -4     
+ Misses       1314     1296      -18     
+ Partials       51       49       -2

Flag	Coverage Δ
unittests	`39.25% <0.00%> (+0.24%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/exporter/proto.go	`95.12% <0.00%> (+12.14%)`	⬆️
pkg/flow/record.go	`63.63% <0.00%> (-0.47%)`	⬇️

msherif1234 · 2023-07-14T11:57:40Z

/ok-to-test

github-actions · 2023-07-14T11:59:01Z

New image: quay.io/netobserv/netobserv-ebpf-agent:0dc1c9d. It will expire after two weeks.

openshift-ci-robot · 2023-07-14T12:13:14Z

@msherif1234: This pull request references NETOBSERV-1190 which is a valid jira issue.

In response to this:

add new LRU map to correlate DNS Q/R and calculate TS
manual testing

]$ dig google.com

; <<>> DiG 9.16.33-RH <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15672
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             194     IN      A       172.253.122.113
google.com.             194     IN      A       172.253.122.100
google.com.             194     IN      A       172.253.122.139
google.com.             194     IN      A       172.253.122.138
google.com.             194     IN      A       172.253.122.101
google.com.             194     IN      A       172.253.122.102

;; Query time: 132 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Thu Jul 13 14:34:28 EDT 2023
;; MSG SIZE  rcvd: 135

Then use flows_logs collector

ipv4: 14:34:28.458790 unknown IP 127.0.0.53:53 > 127.0.0.1:43613: protocol:udp type: 0 code: 0 dir:0 bytes:177 packets:1 flags:0 ends: 14:34:28.458790 dnsId: 15672 dnsFlags: 0x8180 dnsLatency: nanos:131602039 rtt 0

test on cluster

{
 "AgentIP": "10.0.128.2",
 "Bytes": 137,
 "DnsFlags": 33152,
 "DnsFlagsResponseCode": "NoError",
 "DnsId": 29212,
 "DnsLatencyMs": 4,
 "DstAddr": "10.131.0.8",
 "DstK8S_HostIP": "10.0.128.2",
 "DstK8S_HostName": "ci-ln-mv3kgpt-72292-vcr5x-worker-a-2crf4",
 "DstK8S_Name": "dns-default-wcrv6",
 "DstK8S_Namespace": "openshift-dns",
 "DstK8S_OwnerName": "dns-default",
 "DstK8S_OwnerType": "DaemonSet",
 "DstK8S_Type": "Pod",
 "DstMac": "0A:58:0A:83:00:08",
 "DstPort": 54476,
 "Duplicate": false,
 "Etype": 2048,
 "FlowDirection": "0",
 "IfDirection": 0,
 "Interface": "unknown",
 "Packets": 1,
 "Proto": 17,
 "SrcAddr": "169.254.169.254",
 "SrcMac": "0A:58:0A:83:00:01",
 "SrcPort": 53,
 "TimeFlowEndMs": 1689336618135,
 "TimeFlowStartMs": 1689336618135,
 "TimeReceived": 1689336618,
 "app": "netobserv-flowcollector"
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-07-14T12:34:49Z

@msherif1234: This pull request references NETOBSERV-1190 which is a valid jira issue.

In response to this:

add new LRU map to correlate DNS Q/R and calculate TS
manual testing

]$ dig google.com

;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30678
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             224     IN      A       172.253.122.139
google.com.             224     IN      A       172.253.122.113
google.com.             224     IN      A       172.253.122.101
google.com.             224     IN      A       172.253.122.102
google.com.             224     IN      A       172.253.122.100
google.com.             224     IN      A       172.253.122.138

;; Query time: 21 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri Jul 14 08:31:27 EDT 2023
;; MSG SIZE  rcvd: 135

Then use flows_logs collector

ipv4: 08:31:27.947192 unknown IP 127.0.0.53:53 > 127.0.0.1:49033: protocol:udp type: 0 code: 0 dir:0 bytes:177 packets:1 flags:0 ends: 08:31:27.947192 dnsId: 30678 dnsFlags: 0x8180 dnsLatency(ms): 21 rtt 0

test on cluster

{
 "AgentIP": "10.0.128.2",
 "Bytes": 137,
 "DnsFlags": 33152,
 "DnsFlagsResponseCode": "NoError",
 "DnsId": 29212,
 "DnsLatencyMs": 4,
 "DstAddr": "10.131.0.8",
 "DstK8S_HostIP": "10.0.128.2",
 "DstK8S_HostName": "ci-ln-mv3kgpt-72292-vcr5x-worker-a-2crf4",
 "DstK8S_Name": "dns-default-wcrv6",
 "DstK8S_Namespace": "openshift-dns",
 "DstK8S_OwnerName": "dns-default",
 "DstK8S_OwnerType": "DaemonSet",
 "DstK8S_Type": "Pod",
 "DstMac": "0A:58:0A:83:00:08",
 "DstPort": 54476,
 "Duplicate": false,
 "Etype": 2048,
 "FlowDirection": "0",
 "IfDirection": 0,
 "Interface": "unknown",
 "Packets": 1,
 "Proto": 17,
 "SrcAddr": "169.254.169.254",
 "SrcMac": "0A:58:0A:83:00:01",
 "SrcPort": 53,
 "TimeFlowEndMs": 1689336618135,
 "TimeFlowStartMs": 1689336618135,
 "TimeReceived": 1689336618,
 "app": "netobserv-flowcollector"
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

…query/rsp TS Signed-off-by: msherif1234 <mmahmoud@redhat.com>

msherif1234 · 2023-07-14T12:37:57Z

/ok-to-test

github-actions · 2023-07-14T12:39:38Z

New image: quay.io/netobserv/netobserv-ebpf-agent:1404a60. It will expire after two weeks.

jotak · 2023-07-17T10:13:00Z

I am still wondering if we could find ways to not generate any new flow, but just edit the existing ones - I did some tries for that but without success, cf my slack message
Anyway this PR is already a nice improvement, thanks @msherif1234 !
/lgtm

msherif1234 · 2023-07-17T11:21:03Z

I am still wondering if we could find ways to not generate any new flow, but just edit the existing ones - I did some tries for that but without success, cf my slack message Anyway this PR is already a nice improvement, thanks @msherif1234 ! /lgtm

Thanks @jotak lets proceed with that to unlock QE and continue looking for improvemnst

msherif1234 · 2023-07-17T11:25:32Z

/approve

openshift-ci · 2023-07-17T11:25:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: msherif1234

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [msherif1234]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot added the jira/valid-reference label Jul 13, 2023

msherif1234 changed the title ~~NETOBSERV-1190: update dns tracker to provide DNS latency instead of query/rsp TS~~ NETOBSERV-1190: update dns tracker to provide latency instead of TS Jul 13, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 14, 2023

msherif1234 changed the title ~~NETOBSERV-1190: update dns tracker to provide latency instead of TS~~ NETOBSERV-1190: update dns tracker to calc latency instead of TS Jul 14, 2023

NETOBSERV-1190: update dns tracker to provide DNS latency instead of …

e5a3902

…query/rsp TS Signed-off-by: msherif1234 <mmahmoud@redhat.com>

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 14, 2023

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 14, 2023

msherif1234 mentioned this pull request Jul 14, 2023

NETOBSERV-1190: change DNS to use latency instead of TS netobserv/network-observability-operator#395

Merged

openshift-ci bot assigned jotak Jul 17, 2023

openshift-ci bot added the lgtm label Jul 17, 2023

openshift-ci bot added the approved label Jul 17, 2023

openshift-merge-robot merged commit 4a182aa into netobserv:main Jul 17, 2023
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NETOBSERV-1190: update dns tracker to calc latency instead of TS #149

NETOBSERV-1190: update dns tracker to calc latency instead of TS #149

msherif1234 commented Jul 13, 2023 •

edited

Loading

openshift-ci-robot commented Jul 13, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jul 13, 2023 •

edited by openshift-ci bot

Loading

codecov bot commented Jul 13, 2023 •

edited

Loading

msherif1234 commented Jul 14, 2023

github-actions bot commented Jul 14, 2023

openshift-ci-robot commented Jul 14, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jul 14, 2023 •

edited by openshift-ci bot

Loading

msherif1234 commented Jul 14, 2023

github-actions bot commented Jul 14, 2023

jotak commented Jul 17, 2023

msherif1234 commented Jul 17, 2023

msherif1234 commented Jul 17, 2023

openshift-ci bot commented Jul 17, 2023

NETOBSERV-1190: update dns tracker to calc latency instead of TS #149

NETOBSERV-1190: update dns tracker to calc latency instead of TS #149

Conversation

msherif1234 commented Jul 13, 2023 • edited Loading

openshift-ci-robot commented Jul 13, 2023 • edited by openshift-ci bot Loading

openshift-ci-robot commented Jul 13, 2023 • edited by openshift-ci bot Loading

codecov bot commented Jul 13, 2023 • edited Loading

Codecov Report

msherif1234 commented Jul 14, 2023

github-actions bot commented Jul 14, 2023

openshift-ci-robot commented Jul 14, 2023 • edited by openshift-ci bot Loading

openshift-ci-robot commented Jul 14, 2023 • edited by openshift-ci bot Loading

msherif1234 commented Jul 14, 2023

github-actions bot commented Jul 14, 2023

jotak commented Jul 17, 2023

msherif1234 commented Jul 17, 2023

msherif1234 commented Jul 17, 2023

openshift-ci bot commented Jul 17, 2023

msherif1234 commented Jul 13, 2023 •

edited

Loading

openshift-ci-robot commented Jul 13, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jul 13, 2023 •

edited by openshift-ci bot

Loading

codecov bot commented Jul 13, 2023 •

edited

Loading

openshift-ci-robot commented Jul 14, 2023 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Jul 14, 2023 •

edited by openshift-ci bot

Loading