Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster conntrack parser #2488

Closed
wants to merge 13 commits into from
Closed

Faster conntrack parser #2488

wants to merge 13 commits into from

Conversation

bboreham
Copy link
Collaborator

@bboreham bboreham commented May 1, 2017

Profiling indicated that conntrack parsing was around 20% of the CPU time of the probe in Weave Cloud dev, so I wrote a scanner/parser instead of relying on sscanf and split.

Test run decoding a conntrack -L dump with 27,000 lines:

Before:

ok  	github.com/weaveworks/scope/probe/endpoint	19.174s
ok  	github.com/weaveworks/scope/probe/endpoint	17.206s
ok  	github.com/weaveworks/scope/probe/endpoint	17.377s
ok  	github.com/weaveworks/scope/probe/endpoint	18.486s

After

ok  	github.com/weaveworks/scope/probe/endpoint	3.817s
ok  	github.com/weaveworks/scope/probe/endpoint	4.411s
ok  	github.com/weaveworks/scope/probe/endpoint	3.702s
ok  	github.com/weaveworks/scope/probe/endpoint	4.022s

3.7/17.2=22%

Individual commits left for now in case I need to remember how I got here, but I would squash before merging. I don't recommend reading the individual commits for review.

@2opremio
Copy link
Contributor

2opremio commented May 2, 2017

Thanks @bboreham ! I shall take a look tomorrow

It seems you have put quite a bit of work into this. Wouldn't it have been better spent ditching the text parser and finishing vishvananda/netlink#171 ? After vishvananda/netlink#213 I think it wouldn't had taken too long.

@bboreham
Copy link
Collaborator Author

bboreham commented May 2, 2017

@2opremio never occurred to me. Maybe raise that as an issue in this repo?

@2opremio
Copy link
Contributor

2opremio commented May 3, 2017

Done: #2495

@bboreham Mind fixing the linter problems?

@rade
Copy link
Member

rade commented Jul 19, 2017

@bboreham what's happening with this?

@bboreham
Copy link
Collaborator Author

I guess we should close this since we prefer to use eBPF.

@rade rade closed this Feb 26, 2018
@bboreham
Copy link
Collaborator Author

We moved to using netlink in #3298.

@bboreham bboreham deleted the faster-conntrack-parser branch January 13, 2020 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants