probe: conntrack: fix output parsing

[alban]

With net.netfilter.nf_conntrack_acct = 1, conntrack adds the following
fields in the output: packets=3 bytes=164

And with SELinux (e.g. Fedora), conntrack adds: secctx=...

The parsing with fmt.Sscanf introduced in #2095 was unfortunately
rejecting lines with those fields. This patch fixes that by adding more
complicated parsing in decodeTwoTuples() with FieldsFunc and SplitN.

Fixes #2117
Regression from #2095

---

• test under different configuration (SELinux, nf_conntrack_acct)
• check performances compared to sscanf and xml

---

/cc @iaguis @2opremio

[2opremio]

Thanks @alban . Regexps tend to perform pretty badly, let's confirm with a test but I suspect a simple "key=value" parser would be a better option.

[alban]

tl;dr: decodeStreamedFlow() took

• 0.56s in the scanf version
• 1.73s in th regexp version

So regexp is significantly slower. I will try with a simple "key=value" parser. Note that there is several keys with the same string, so we have to read them in order.

Long version:

pprof couldn't work for me on go 1.7, so I had to revert to go 1.6 with:

git revert d1cf9f60fced05ccb9283d281c9405209bb4b0b0
git revert 002770d3949b52bcc0c0dba444e74090b97b53e8
git revert 28213a00a463ca68bd32cf972ebe74280f427885

Then, the perf check is done with:

sudo ./scope launch --probe.http.listen :4041
go tool pprof http://localhost:4041/debug/pprof/profile
web

The test I did was:

sudo docker run -ti --rm busybox
while date ; do echo GET | nc $IP 80 > /dev/null ; done

This was not a good idea to pick an external $IP for the test because that server latency changed when I tried the xml version. DoS mitigation? I'll use an internal server for the next test.

[alban]

Patch updated with a manual parser instead of regexp. It seems to work at a first glance but I still need to check the performances.

[alban]

I checked the performance again, but this time:

• without using an external server, but using nginx in Docker instead
• running 4 clients in parallel with the following commands:

docker run -ti --rm busybox
while date ; do (sleep 0.05 ; echo GET / ) | nc 172.17.0.3 80 > /dev/null ; done

Then I compare the perfs from pprof:

1. xml parsing: DecodeElement: 4.9s
2. sscanf parsing (as it is on master, but with a local workaround for parsing the SELinux attributes): decodeStreamedFlow: 2.87s
3. manual parsing in this PR: decodeStreamedFlow: 1.34s

I am not sure why my code is faster than the scanf, but that seems good.

[2opremio]

I am not sure why my code is faster than the scanf, but that seems good.

Awesome, great job!

[alban]

Updated. I moved the 60 lines in the helper function, and updated the description of the PR. It worked for me on Fedora (with the SELinux field). I tried with and without "net.netfilter.nf_conntrack_acct". PTAL.

[2opremio]

@alban Can you please a few examples of the lines which use to fail in conntrack_test.go?

[2opremio]

Also, please extend the commented format examples in conntrack.go or remove them altogether.

[alban]

Updated and rebased. Only the dumped flows have the extra fields, so I updated the examples and the tests for the dumped flows only.