Added TCP Flags to Flow Record metrics #68

shach33 · 2022-11-21T07:45:06Z

I added a 16 bit flag to every record. The TCP header is read and 8 flags (fin, syn, rst, psh, ack, urg, ece, cwr) are added to bits 0-7 in flags member of metrics struct.
During accumulation, the flags member is bitwise OR'ed to get all flags in bits 0-7.

mariomac · 2022-11-21T08:59:17Z

@shach33 Thank you for your contribution! In order to facilitate the C code review, could you keep the original indent configuration? It's 4 spaces, no tab.

proto/flow.proto

ronensc · 2022-12-05T15:03:27Z

@shach33 Thank you very much for this PR! :)
I am interested in the TCP flags to allow the connection tracking module in FLP to identify which of the endpoints started the connection (i.e. who sent the first SYN).

IIUC accumulating all the flags of the flowlogs with bitwise OR won't allow me to find which endpoint started the connection.

As known, in TCP the 3-step handshake goes like this:

client -> server: SYN
server -> client: SYN+ACK
client -> server: ACK

I think that it is most likely that the packets of step 1 and 3 will end up in the same flowlog. In this case, that flowlog will have both SYN and ACK flags making it indistinguishable from the flowlog of step 2.
Please correct me if I'm wrong.

cc @eranra @jotak @mariomac

praveingk · 2022-12-06T05:09:56Z

@shach33 Thank you very much for this PR! :) I am interested in the TCP flags to allow the connection tracking module in FLP to identify which of the endpoints started the connection (i.e. who sent the first SYN).

IIUC accumulating all the flags of the flowlogs with bitwise OR won't allow me to find which endpoint started the connection.

As known, in TCP the 3-step handshake goes like this:

client -> server: SYN

server -> client: SYN+ACK

client -> server: ACK

I think that it is most likely that the packets of step 1 and 3 will end up in the same flowlog. In this case, that flowlog will have both SYN and ACK flags making it indistinguishable from the flowlog of step 2. Please correct me if I'm wrong.

cc @eranra @jotak @mariomac

Hi @ronensc , Thanks for highlighting that. Yes, you are right, we might need to capture only Syn/Fin/Rst in the MSB of the u16 flag, which might help in getting that info.

For eg. The logic in the ebpf code could be :

#define TCP_SYN_BIT 0x80
#define TCP_FIN_BIT 0x40
#define TCP_RST_BIT 0x20

if tcp->syn==1 && tcp->ack==0 {
    flags_t = flags_t | TCP_SYN_BIT;
}
if tcp->fin==1 && tcp->ack==0 {
    flags_t = flags_t | TCP_FIN_BIT;
}
if tcp->rst==1 && tcp->ack==0 {
    flags_t = flags_t | TCP_RST_BIT;
}

// Rest of the logic to aggregate flags_t from the TCP header

Does that work? Additionally, We would like to know more such use-cases where aggregation of flags_t is not very useful., so that we can address them here.
Also, would like to know if there are more information from the TCP header you would like us to capture in the flags.

ronensc · 2022-12-06T10:01:43Z

Thanks @praveingk
I had a discussion with @eranra about your suggestion.
We think that it might work for the use case of identifying the first SYN. But, if we set the SYN flag in a flowlog only when the ACK flag is not set in the packet, then the flowlog of the SYN+ACK packet won't have the SYN flag set. So using the term "TCP flags" in the context of flowlogs feels inaccurate. We can use a different term for these flags such as "start" to avoid confusion.

We also talked about that in latency measurements, we need to spot both the SYN and SYN+ACK flowlogs. And in the suggested approach this is not possible.

We suggest other alternatives:

Aggregate packets into flowlogs based on their TCP flags. In other words, packets with different TCP flags will end up in different flowlogs.
Instead of having a bitwise-or aggregation of the TCP flags, having a list of the unique TCP flags combination. For example, if a flowlog consists of 3 packets with the flags [SYN, ACK, ACK] then the flowlog will have the 2-item list [SYN, ACK]. And if a flowlog consists of 3 pakcets with the flags [SYN+ACK, ACK, ACK] then the flowlog will have the 2-item list [SYN+ACK, ACK].

praveingk · 2022-12-07T04:40:19Z

Thanks @praveingk I had a discussion with @eranra about your suggestion. We think that I might work for the use case of identifying the first SYN. But, if we set the SYN flag in a flowlog only when the ACK flag is not set in the packet, then the flowlog of the SYN+ACK packet won't have the SYN flag set. So using the term "TCP flags" in the context of flowlogs feels inaccurate. We can use a different term for these flags such as "start" to avoid confusion.

We also talked about that in latency measurements, we need to spot both the SYN and SYN+ACK flowlogs. And in the suggested approach this is not possible.

We suggest other alternatives:

Aggregate packets into flowlogs based on their TCP flags. In other words, packets with different TCP flags will end up in different flowlogs.

Instead of having a bitwise-or aggregation of the TCP flags, having a list of the unique TCP flags combination. For example, if a flowlog consists of 3 packets with the flags [SYN, ACK, ACK] then the flowlog will have the 2-item list [SYN, ACK]. And if a flowlog consists of 3 pakcets with the flags [SYN+ACK, ACK, ACK] then the flowlog will have the 2-item list [SYN+ACK, ACK].

@ronensc : Yes, I understand the point, and I agree. Can you confirm if the list of combinations that can be incorporated in the eBPF code are below:

SYN
FIN
RST
SYN+ACK
FIN+ACK
RST+ACK
URG
PSH
ECE
CWR

ronensc · 2022-12-07T12:26:29Z

I think this list is sufficient.
@eranra WDYT?

eranra · 2022-12-07T13:18:00Z

I think this list is sufficient. @eranra WDYT?

This is precisely what we need !!!

@ronensc @praveingk @shach33 ^^^

mariomac · 2022-12-09T09:27:15Z

@ronensc with respect to:

Instead of having a bitwise-or aggregation of the TCP flags, having a list of the unique TCP flags combination. For example, if a flowlog consists of 3 packets with the flags [SYN, ACK, ACK] then the flowlog will have the 2-item list [SYN, ACK]. And if a flowlog consists of 3 pakcets with the flags [SYN+ACK, ACK, ACK] then the flowlog will have the 2-item list [SYN+ACK, ACK].

I have concerns with respect to the implementation on that. AFAIK, eBPF doesn't allow dynamic memory to be created. That would mean that, for each record, you'd need to create a big-enough, fixed-size array to store the flag sequences. I'm afraid this could impact in the overall memory and performance.

bpf/flows.c

proto/flow.proto

shach33 · 2022-12-12T08:53:27Z

I am working on it. Will push the changes soon.

praveingk · 2022-12-12T13:23:51Z

@ronensc with respect to:

Instead of having a bitwise-or aggregation of the TCP flags, having a list of the unique TCP flags combination. For example, if a flowlog consists of 3 packets with the flags [SYN, ACK, ACK] then the flowlog will have the 2-item list [SYN, ACK]. And if a flowlog consists of 3 pakcets with the flags [SYN+ACK, ACK, ACK] then the flowlog will have the 2-item list [SYN+ACK, ACK].

I have concerns with respect to the implementation on that. AFAIK, eBPF doesn't allow dynamic memory to be created. That would mean that, for each record, you'd need to create a big-enough, fixed-size array to store the flag sequences. I'm afraid this could impact in the overall memory and performance.

Hi @mariomac , including the combinations, we have only 10 possibilities, which can be stored in u16 with a bit mask. That should be ok right?

mariomac · 2022-12-12T13:58:03Z

@praveingk yes, a 16-bit number it's fine. Maybe I misunderstood @ronensc message and I thought some kind of list/array would be implemented to store that information.

ronensc · 2022-12-14T14:36:36Z

@praveingk yes, a 16-bit number it's fine. Maybe I misunderstood @ronensc message and I thought some kind of list/array would be implemented to store that information.

@mariomac you got me right. I wan't aware of eBPF limitations...

bpf/flows.c

pkg/exporter/kafka_proto_test.go

bpf/flows.c

pkg/flow/record.go

praveingk · 2023-01-06T05:32:35Z

@shach33 Can you rebase this PR to main branch since there are merge conflicts. After the rebase, I would like to add flags to ipfix export as well.

fixing indents Indent fixes for flows.c Indent fixes for flows.c-II Still fixing indents

Updated changes according to comments on PR

Rebase merges

mariomac

Much better, thanks! Few minor comments.

bpf/flows.c

pkg/exporter/proto.go

codecov-commenter · 2023-01-11T04:55:44Z

Codecov Report

Merging #68 (509b40d) into main (6a351f2) will increase coverage by 0.10%.
The diff coverage is 6.66%.

❗ Current head 509b40d differs from pull request most recent head d5a2ef2. Consider uploading reports for the commit d5a2ef2 to get more accurate results

@@            Coverage Diff             @@
##             main      #68      +/-   ##
==========================================
+ Coverage   41.50%   41.60%   +0.10%     
==========================================
  Files          29       29              
  Lines        1983     1978       -5     
==========================================
  Hits          823      823              
+ Misses       1122     1116       -6     
- Partials       38       39       +1

Flag	Coverage Δ
unittests	`41.60% <6.66%> (+0.10%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/agent/ip.go	`70.40% <ø> (ø)`
pkg/exporter/ipfix.go	`0.00% <0.00%> (ø)`
pkg/exporter/proto.go	`100.00% <100.00%> (ø)`
pkg/flow/record.go	`83.33% <100.00%> (+0.47%)`	⬆️
pkg/exporter/grpc_proto.go	`57.14% <0.00%> (-14.29%)`	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

jotak · 2023-01-12T10:37:21Z

bpf/flows.c

@@ -97,7 +138,7 @@ static inline int fill_iphdr(struct iphdr *ip, void *data_end, flow_id *id) {
 }

 // sets flow fields from IPv6 header information
-static inline int fill_ip6hdr(struct ipv6hdr *ip, void *data_end, flow_id *id) {
+static inline int fill_ip6hdr(struct ipv6hdr *ip, void *data_end, flow_id *id, u16 *flags) {


shouldn't we call set_flags here too, in case IPPROTO_TCP block?

Good catch! We always forget to test IPv6 :-(

…into shach33/main

jotak

lgtm!

jotak · 2023-01-18T07:37:19Z

Merging!
Kudos @shach33 @praveingk

* Test commit * Adding TCP flags * Adding TCP flags to record metrics.TODO: Remove log stmts * Updated tcp flags u32-> u16 * Updated indentation in C file * Update flow.h * Indent fixed. Tab -> 4 space fixing indents Indent fixes for flows.c Indent fixes for flows.c-II Still fixing indents * Removed extra file * Added clang-format config to bpf/. To use clang-format -i <file.c> * Fixed message record member sequences * Added TCP Flags combinations Read from flw Updated changes according to comments on PR * Removed log statements. Rebase merges * Fixing merge conflits * Revert the renaming of grpc_test file * Simplify setting flags for v4,v6 * Add flags to ipfix exporter * Update protobuf * Remove type conversions due to conflicts * Remove .gitignore change * Fix indentation * Remove commented line * Fixed typecast errors since move to go 1.18 * Reverting commit d1e9c44. Added changes as separate PR * set flags for v6 Co-authored-by: Pravein Govindan Kannan <pravein.govindan.kannan@ibm.com>

mariomac reviewed Nov 22, 2022

View reviewed changes

proto/flow.proto Outdated Show resolved Hide resolved

mariomac suggested changes Dec 9, 2022

View reviewed changes

bpf/flows.c Outdated Show resolved Hide resolved

proto/flow.proto Outdated Show resolved Hide resolved

mariomac reviewed Jan 5, 2023

View reviewed changes

shach33 and others added 13 commits January 9, 2023 12:37

Test commit

859c84d

Adding TCP flags

03352d4

Adding TCP flags to record metrics.TODO: Remove log stmts

0061895

Updated tcp flags u32-> u16

5e076e6

Updated indentation in C file

be3d92d

Update flow.h

6cbfef9

Indent fixed. Tab -> 4 space

62c049b

fixing indents Indent fixes for flows.c Indent fixes for flows.c-II Still fixing indents

Removed extra file

88d2e01

Added clang-format config to bpf/. To use clang-format -i <file.c>

941c1f3

Fixed message record member sequences

4ff16f9

Added TCP Flags combinations Read from flw

9ab2594

Updated changes according to comments on PR

Removed log statements.

6ac73e2

Rebase merges

Fixing merge conflits

f7df3f8

shach33 force-pushed the main branch from 5bf87f2 to f7df3f8 Compare January 10, 2023 07:09

mariomac suggested changes Jan 10, 2023

View reviewed changes

bpf/flows.c Outdated Show resolved Hide resolved

pkg/exporter/proto.go Outdated Show resolved Hide resolved

praveingk added 6 commits January 10, 2023 16:13

Revert the renaming of grpc_test file

9f98068

Simplify setting flags for v4,v6

7f0ef4a

Add flags to ipfix exporter

4fb16f5

Update protobuf

561aeae

Remove type conversions due to conflicts

3f75bcb

Remove .gitignore change

5b9badc

praveingk and others added 4 commits January 11, 2023 10:28

Fix indentation

509b40d

Remove commented line

d5a2ef2

Fixed typecast errors since move to go 1.18

d1e9c44

Reverting commit d1e9c44. Added changes as separate PR

f5f7df0

mariomac self-requested a review January 12, 2023 10:04

jotak reviewed Jan 12, 2023

View reviewed changes

praveingk added 2 commits January 12, 2023 16:37

set flags for v6

940f5cc

Merge branch 'main' of https://github.com/shach33/netobserv-ebpf-agent …

82a96ae

…into shach33/main

jotak approved these changes Jan 12, 2023

View reviewed changes

mariomac approved these changes Jan 17, 2023

View reviewed changes

jotak merged commit 45b4757 into netobserv:main Jan 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added TCP Flags to Flow Record metrics #68

Added TCP Flags to Flow Record metrics #68

shach33 commented Nov 21, 2022

mariomac commented Nov 21, 2022 •

edited

Loading

ronensc commented Dec 5, 2022

praveingk commented Dec 6, 2022 •

edited

Loading

ronensc commented Dec 6, 2022 •

edited

Loading

praveingk commented Dec 7, 2022

ronensc commented Dec 7, 2022

eranra commented Dec 7, 2022

mariomac commented Dec 9, 2022

shach33 commented Dec 12, 2022

praveingk commented Dec 12, 2022

mariomac commented Dec 12, 2022

ronensc commented Dec 14, 2022

praveingk commented Jan 6, 2023

mariomac left a comment

codecov-commenter commented Jan 11, 2023 •

edited

Loading

jotak Jan 12, 2023

mariomac Jan 12, 2023

jotak left a comment

jotak commented Jan 18, 2023

Added TCP Flags to Flow Record metrics #68

Added TCP Flags to Flow Record metrics #68

Conversation

shach33 commented Nov 21, 2022

mariomac commented Nov 21, 2022 • edited Loading

ronensc commented Dec 5, 2022

praveingk commented Dec 6, 2022 • edited Loading

ronensc commented Dec 6, 2022 • edited Loading

praveingk commented Dec 7, 2022

ronensc commented Dec 7, 2022

eranra commented Dec 7, 2022

mariomac commented Dec 9, 2022

shach33 commented Dec 12, 2022

praveingk commented Dec 12, 2022

mariomac commented Dec 12, 2022

ronensc commented Dec 14, 2022

praveingk commented Jan 6, 2023

mariomac left a comment

Choose a reason for hiding this comment

codecov-commenter commented Jan 11, 2023 • edited Loading

Codecov Report

jotak Jan 12, 2023

Choose a reason for hiding this comment

mariomac Jan 12, 2023

Choose a reason for hiding this comment

jotak left a comment

Choose a reason for hiding this comment

jotak commented Jan 18, 2023

mariomac commented Nov 21, 2022 •

edited

Loading

praveingk commented Dec 6, 2022 •

edited

Loading

ronensc commented Dec 6, 2022 •

edited

Loading

codecov-commenter commented Jan 11, 2023 •

edited

Loading