Community-Id spec support

[nathandau]

Hi,

Wanted to ask the question of whether n2disk would consider supporting the community-id spec as seen here - https://github.com/corelight/community-id-spec

Multiple network flow or analysis sensors support this whcih improves analysis workflow. having the same seed value configured in n2disk to generate the same hash across tools is beneficial to analysis workflow. Is this possible at all?

Also may assist with npcapextract search and extract functionality instead of defining 5 tuple bpf syntax it could be npcapextract community-id to pin point the same stream as other tools observe.

Thanks,
Nathan

[cardigliano]

In order to implement this, it is required to compute the community-id and add it to the index. The nBPF syntax should also support the community-id keyword to run extractions with npcapextract based on that.

[nathandau]

Thats the exact functionality i would like to use. If it is implemented i have the dev repo of ntop configured in order to test and provide feedback.

[nathandau]

Is there any info you need from my end or want for me to update to a dev version to test this functionality?

[cardigliano]

@nathandau we are still discussing the implementation of this internally, this takes time as it requires changes to the core engine and data structures. I will update this issue as soon as we have news. Thank you.

[nathandau]

Not a problem thanks for the update

[nathandau]

Hi @cardigliano , How is the conversation going internally around the implementation of this functionality? Just thought id check in if it was still being considered?

Regards,
Nathan

[cardigliano]

Hi @nathandau, this is still an interesting feature for us, however there is still an open discussion about the implementation for a few reason (and also because it is low priority at the moment):

• the community id takes a lot of space in the index (we are considering adding an hash of the hash)
• the current index needs to be extended to make room for it (a new index version need to be handled in essence)

[readcoil]

Hi @cardigliano, was there any progress on community-id support? I have some use cases I'm looking at currently that would benefit from it greatly.

[cardigliano]

@readcoil this has an impact on performance, in addition to reworking the index, however there should be no problem in addint it in the exported flow metadata for instance. What is your use case?

[readcoil]

Thanks for the quick reply @cardigliano. I'm mostly wanting to be able to retrieve packets associated with NIDS alerts (suricata/zeek etc). I'm guessing querying on the underlying community-id components (proto, srcip, srcprt, dstip, dstport) will have the same result, but the queries may be slower and pivoting may become problematic. Do you know of any better solutions or should I just work with the underlying flow data?

[readcoil]

I think I've probably answered my own question. Querying the 5-tuple with BPF should meet my current objectives.