Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-1478: Add eBPF Agent filtering capability #307

Merged
merged 5 commits into from
Apr 12, 2024

Conversation

msherif1234
Copy link
Contributor

@msherif1234 msherif1234 commented Mar 29, 2024

Description

This PR allow provisioning global filter to limit what goes to eBPF flows table
this also fix #266

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

  • Direction: Ingress , use srcIP CIDR
  • Direction: Egress, use dstIP CIDR

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

udo bpftool map dump id 211
[{
       "key": {
           "prefix_len": 24,
           "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       },
       "value": {
           "protocol": 6,
           "dstPortStart": 0,
           "dstPortEnd": 0,
           "srcPortStart": 80,
           "srcPortEnd": 0,
           "icmpType": 0,
           "icmpCode": 0,
           "direction": "INGRESS",
           "ip": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       }
   }
]

irq/177-iwlwifi-1194    [010] ..s2.  6252.626434: bpf_trace_printk: ip_filter:
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 172
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 253
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 122
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 101
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0 
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: 

irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626443: bpf_trace_printk: result: 2

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

  • Direction: Ingress , use srcIP CIDR
  • Direction: Egress, use dstIP CIDR

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 211
[{
       "key": {
           "prefix_len": 24,
           "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       },
       "value": {
           "protocol": 6,
           "dstPortStart": 0,
           "dstPortEnd": 0,
           "srcPortStart": 80,
           "srcPortEnd": 0,
           "icmpType": 0,
           "icmpCode": 0,
           "direction": "INGRESS",
           "ip": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       }
   }
]

irq/177-iwlwifi-1194    [010] ..s2.  6252.626434: bpf_trace_printk: ip_filter:
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 172
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 253
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 122
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 101
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0 
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: 

irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626443: bpf_trace_printk: result: 2

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@codecov-commenter
Copy link

codecov-commenter commented Mar 29, 2024

Codecov Report

Attention: Patch coverage is 31.65829% with 136 lines in your changes are missing coverage. Please review.

Project coverage is 33.91%. Comparing base (cb2cf53) to head (28bce79).
Report is 1 commits behind head on main.

Files Patch % Lines
pkg/ebpf/flow_filter.go 43.66% 71 Missing and 9 partials ⚠️
pkg/ebpf/tracer.go 0.00% 36 Missing ⚠️
pkg/agent/agent.go 0.00% 19 Missing ⚠️
pkg/ebpf/bpf_x86_bpfel.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #307      +/-   ##
==========================================
+ Coverage   33.63%   33.91%   +0.27%     
==========================================
  Files          47       48       +1     
  Lines        3865     4043     +178     
==========================================
+ Hits         1300     1371      +71     
- Misses       2477     2575      +98     
- Partials       88       97       +9     
Flag Coverage Δ
unittests 33.91% <31.65%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 211
[{
       "key": {
           "prefix_len": 24,
           "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       },
       "value": {
           "protocol": 6,
           "dstPortStart": 0,
           "dstPortEnd": 0,
           "srcPortStart": 80,
           "srcPortEnd": 0,
           "icmpType": 0,
           "icmpCode": 0,
           "direction": "INGRESS",
           "ip": [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
           ]
       }
   }
]

irq/177-iwlwifi-1194    [010] ..s2.  6252.626434: bpf_trace_printk: ip_filter:
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 172
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 253
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 122
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 101
irq/177-iwlwifi-1194    [010] ..s2.  6252.626438: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626439: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0 
irq/177-iwlwifi-1194    [010] ..s2.  6252.626440: bpf_trace_printk: 0
irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: 

irq/177-iwlwifi-1194    [010] ..s2.  6252.626441: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626442: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2.  6252.626443: bpf_trace_printk: result: 2

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/170-iwlwifi-1187    [003] ..s2. 14437.549124: bpf_trace_printk: rule found

irq/170-iwlwifi-1187    [003] ..s2. 14437.549125: bpf_trace_printk: srcPortStart matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549127: bpf_trace_printk: ip not zero ip[0]:192

irq/170-iwlwifi-1187    [003] ..s2. 14437.549129: bpf_trace_printk: dstIP matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549130: bpf_trace_printk: direction matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549131: bpf_trace_printk: result: 3

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/170-iwlwifi-1187    [003] ..s2. 14437.549124: bpf_trace_printk: rule found

irq/170-iwlwifi-1187    [003] ..s2. 14437.549125: bpf_trace_printk: srcPortStart matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549127: bpf_trace_printk: ip not zero ip[0]:192

irq/170-iwlwifi-1187    [003] ..s2. 14437.549129: bpf_trace_printk: dstIP matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549130: bpf_trace_printk: direction matched

irq/170-iwlwifi-1187    [003] ..s2. 14437.549131: bpf_trace_printk: result: 3

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 29, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@msherif1234 msherif1234 force-pushed the agent_filtering branch 5 times, most recently from 0babff1 to f447aba Compare March 30, 2024 17:18
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 30, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on optional
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 30, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on optional
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

check filter drop statistics

curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8330    0  8330    0 ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 240
   0  2180k      0 --:--:-- --:--:-- --:--:-- 2711k

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Mar 30, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on optional
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Apr 1, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
    Note: if u wanted to match on any CIDR we can use 0.0.0.0\0 syntax
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on optional
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Apr 1, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF table
based on filter rule direction user has to provision ipCIDR based on this rule

Supported options

  • ENABLE_FLOW_FILTER knob to enable flow filtering feature required
  • FLOW_FILTER_IP_CIDR matching IP CIDR required
    Note: if u wanted to match on any CIDR we can use 0.0.0.0/0 syntax
  • FLOW_FILTER_DIRECTION direction to apply the filtering rule on optional
  • FLOW_FILTER_PROTOCOL tcp/udp/sctp/icmpv4/icmpv4 optional
  • FLOW_FILTER_SOURCE_PORT single int value for sport optional
  • FLOW_FILTER_SOURCE_PORT_RANGE use range format [xxx-yyy] for sport optional
  • FLOW_FILTER_DESTINATION_PORT single int value for dport optional
  • FLOW_FILTER_DESTINATION_PORT_RANGE use range format [xxx-yyy] for dport optional
  • FLOW_FILTER_ICMP_TYPE for icmp and icmp6 type value optional
  • FLOW_FILTER_ICMP_CODE for icmp and icmp6 code value optional
  • FLOW_FILTER_IP for matching against specific ip address optional

The following shows example of provision this feature

sudo TARGET_HOST=127.0.0.1 TARGET_PORT=9999 ENABLE_FLOW_FILTER=true FLOW_FILTER_IP_CIDR="172.253.122.1/24" FLOW_FILTER_DIRECTION="ingress" FLOW_FILTER_PROTOCOL="tcp"  FLOW_FILTER_SOURCE_PORT=80 FLOW_FILTER_IP="192.168.1.253" LOG_LEVEL="trace" METRICS_ENABLE=true  ./bin/netobserv-ebpf-agent

The above rule provision ingress filter rule so the source CIDR is 172.253.122.1/24 for protocol TCP with srcPort 80 which is the configuration to match my test with
curl google.com

I also enabled metrics because we counter to show flows been filtered

UT

  • run the above standalone agent command
  • from another xterm run curl -4 google.com -v
  • from another xterm check metrics and ebpf map content and logs
curl 127.0.0.1:9090/metrics | grep Filter
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100  8318    0  8318    0     0  3062k      0 --:--:-- --:--ebpf_agent_dropped_flows_total{reason="FlowFilterDropCounter",source="flow-fetcher"} 1076
:-- --:--:-- 4061k

sudo bpftool map dump id 273 -p
[{
       "key": ["0x18","0x00","0x00","0x00","0xac","0xfd","0x7a","0x01","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "value": ["0x06","0x00","0x00","0x00","0x00","0x50","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0xc0","0xa8","0x01","0xfd","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00","0x00"
       ],
       "formatted": {
           "key": {
               "prefix_len": 24,
               "ip_data": [172,253,122,1,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           },
           "value": {
               "protocol": 6,
               "dstPortStart": 0,
               "dstPortEnd": 0,
               "srcPortStart": 80,
               "srcPortEnd": 0,
               "icmpType": 0,
               "icmpCode": 0,
               "direction": "INGRESS",
               "ip": [192,168,1,253,0,0,0,0,0,0,0,0,0,0,0,0
               ]
           }
       }
   }
]

irq/170-iwlwifi-1187    [003] ..s2. 14437.549100: bpf_trace_printk: ip_filter:
irq/170-iwlwifi-1187    [003] ..s2. 14437.549111: bpf_trace_printk: 172
irq/170-iwlwifi-1187    [003] ..s2. 14437.549112: bpf_trace_printk: 253
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 122
irq/170-iwlwifi-1187    [003] ..s2. 14437.549113: bpf_trace_printk: 101
irq/170-iwlwifi-1187    [003] ..s2. 14437.549114: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549115: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549116: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549117: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549118: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549119: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549120: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0 
irq/170-iwlwifi-1187    [003] ..s2. 14437.549121: bpf_trace_printk: 0
irq/170-iwlwifi-1187    [003] ..s2. 14437.549122: bpf_trace_printk:

irq/177-iwlwifi-1194    [010] ..s2. 16041.762556: bpf_trace_printk: rule found

irq/177-iwlwifi-1194    [010] ..s2. 16041.762557: bpf_trace_printk: protocol matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762560: bpf_trace_printk: srcPortStart matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762563: bpf_trace_printk: ip not zero ip[0]:192

irq/177-iwlwifi-1194    [010] ..s2. 16041.762564: bpf_trace_printk: dstIP matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762576: bpf_trace_printk: direction matched

irq/177-iwlwifi-1194    [010] ..s2. 16041.762577: bpf_trace_printk: result: 4

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@msherif1234
Copy link
Contributor Author

/ok-to-test

@openshift-ci openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 1, 2024
Copy link

github-actions bot commented Apr 1, 2024

New image:
quay.io/netobserv/netobserv-ebpf-agent:e235a56

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=e235a56 make set-agent-image

@github-actions github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 1, 2024
@openshift-ci openshift-ci bot added the lgtm label Apr 5, 2024
@openshift-ci openshift-ci bot removed the lgtm label Apr 5, 2024
@github-actions github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 5, 2024
Comment on lines +88 to +94
filterFlows = defineMetric(
"filtered_flows_total",
"Number of filtered flows",
TypeCounter,
"source",
"reason",
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to update the dashboards ?

Maybe we can manage that in netobserv/network-observability-operator#607 or create a followup in Jira

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we need to update the dashboard or we just add it to the query list not sure @jotak but that is correct we can either added to 607 PR on a following PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's do a follow-up JIRA then; I would say the goal for NETOBSERV-1478 is reached already

For users who enable filtering, I guess it can be interesting to know the ratio of filtered in/filtered out flows indeed, so yeah why not adding that to the health dashboard

Comment on lines 128 to 136
if (rule->icmpCode != 0) {
if (rule->icmpCode == id->icmp_code) {
BPF_PRINTK("icmpCode matched\n");
result++;
} else {
result = 0;
goto end;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it makes sense to have icmpCode independent of icmpType so we should move that check inside of if (rule->icmpType != 0) condition above and document that icmpCode requires icmpType to be set.

WDYT ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they are two independent filters so u set one or the other or both or none

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, but each type have its own related code implementation:
https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml

so it doesn't make any sense to have icmpCode filter only

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see so either type + code or type only I will fix it

@jotak
Copy link
Member

jotak commented Apr 8, 2024

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Apr 8, 2024
Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>
Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>
Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>
Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>
… metric

Signed-off-by: Mohamed Mahmoud <mmahmoud@redhat.com>
@Amoghrd
Copy link

Amoghrd commented Apr 8, 2024

/ok-to-test

@openshift-ci openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 8, 2024
Copy link

github-actions bot commented Apr 8, 2024

New image:
quay.io/netobserv/netobserv-ebpf-agent:bd5c6e0

It will expire after two weeks.

To deploy this build, run from the operator repo, assuming the operator is running:

USER=netobserv VERSION=bd5c6e0 make set-agent-image

Copy link
Contributor

@jpinsonneau jpinsonneau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @msherif1234

@Amoghrd
Copy link

Amoghrd commented Apr 12, 2024

/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved QE has approved this pull request label Apr 12, 2024
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Apr 12, 2024

@msherif1234: This pull request references NETOBSERV-1478 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

Description

This PR allow provisioning global filter to limit what goes to eBPF flows table
this also fix #266

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
  • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
  • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
  • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
  • Standard QE validation, with pre-merge tests unless stated otherwise.
  • Regression tests only (e.g. refactoring with no user-facing change).
  • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@msherif1234
Copy link
Contributor Author

/approve

Copy link

openshift-ci bot commented Apr 12, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: msherif1234

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit 60194d7 into netobserv:main Apr 12, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved jira/valid-reference lgtm ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. qe-approved QE has approved this pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add feature to exclude ip from flow
8 participants