VPN performancev2 #210
Closed
VPN performancev2 #210
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The creation of instance objects should be avoided in packet processing loops to reduce garbage collection overhead. class.lua now supports a simple mechanism where an app can declare an object to no longer be in use. This object is then put on a list in the corresponding class and will be re-used by the constructor method instead of allocating a new object. The class API has been changed slightly to no longer require the declaration of constructor methods. Any method can act as a constructor or extend the standard constructor by calling the new() method of the super class.
Also includes some minor bug fixes here and there.
The conversion from numeric to printable for ipv6 addresses was seriously broken. The fix includes a redefinition of the source and destination address in the ipv6 header as uint8_t.
lib/pcap/filter.lua provides a facility to compile an arbitrary libpcap filter expression and apply it to a data packet for efficient classification.
The new method parse_match() allows the parsing and matching of a single protocol header without having to allocate a table, which can help to avoid garbage. The new method parse_n() parses exactly n protocol headers without applying any additional checks. Also new is the pop_raw() method that pops a protocol header without parsing it. This requires the application to already know the type and size of the next header.
Both apps use the efficient filter module to avoid garbage during packet classification.
The vpws and ns_responder apps now handle checksums correctly in all cases. The GRE key field is now supported correctly as well.
The origin.type member of a newly allocated buffer needs to be set explicitely after receiving a memory block with undefined contents from malloc(3).
|
[make test] at 5ab8d34 |
lib/protocol/datagram.lua: pre-allocate the _packet variable upon instantiation as ctype "struct packet *[1]" to avoid boxing. This appears to eliminate the bulk of garbage in the datagram class. lib/protocol/icmp/: pre-define pointer ctypes in various classes to avoid NYIs due to unsupported FastFunc ffi.typeof. lib/protocol/ipv6.lua: pre-allocate a pseudo-header for every ipv6 object to avoid another NYI.
|
I just discovered a bug in the ICMPv6 neighbor discovery code. I'm closing this request rather than posting a fix afterwards. |
eugeneia
added a commit
to eugeneia/snabb
that referenced
this pull request
Nov 6, 2019
e587f8c55 Merge pull request snabbco#225 from vavrusa/master 5ea3a881e bpf: add missing constants for linux 4.10 - 4.15 2c691e5a7 Merge pull request snabbco#224 from wingo/lseek-syscall-tweak ae38bdbd7 Make "offset" arg to lseek a signed integer 5cb3b6950 Merge pull request snabbco#221 from wingo/util-ls-fix 8e0874609 Promptly close util.ls() dir fd; fix bug with deleted entries 277517436 Merge pull request snabbco#220 from sileht/master 8e48fd094 linux/nl.lua: Use ndmsg struct instead of rtmsg for neighbors 3e482bc4e Merge pull request snabbco#215 from qsn/gettid 57520cce3 expose gettid db1a88e94 Merge pull request snabbco#214 from jsitnicki/sof-flags-linux 270a6e611 Add missing type for struct scm_timestamping for Linux e49232047 Add missing SCM_* constants for Linux timestamping API 60fcc6b48 Add constants for SOF_* flags for Linux 26ac34851 Merge pull request snabbco#210 from alexandergall/linux-if-ioctls 3e6d3e27c Merge pull request snabbco#211 from fperrad/deb d425a22b2 dummy changelog b27eca538 update .gitignore 92292aa4b debian files be257a7e1 debian files generated by lua-create-gitbuildpackage-layout 50a02b94b Add some SIOC{G,S}IF* ioctls for Linux ee90324 Merge pull request snabbco#209 from justincormack/osx_clock ee17863 Add CLOCK_ constants for OSX 178d244 Merge pull request snabbco#208 from justincormack/holes 61450f5 Add SEEK_DATA and SEEK_HOLE constants 9a7b584 Add memfd fnctl sealing support 99beaf5 more test fixes for memfd cc221e4 fix ctest for new fcntl changes, typowq 56c4c76 fix ctest for new fcntl changes 96073cc fix typo 7a73e8a Add more constants for fcntl, memfd 8d3034c Fix ppc64le syscall numbers for newer calls 62828f6 Merge pull request snabbco#206 from johnae/master c4002b6 The spook project is using ljsyscall. 0b266e8 Merge pull request snabbco#205 from lukego/close-fd-safely ad91aa9 Add more protection against fd double-close c8baf9e Merge pull request snabbco#202 from justincormack/dockerignore 66843c5 Use dockerignore to simplify Dockerfile 7b7211d Merge pull request snabbco#200 from justincormack/redo-dockerfile a779caa Docker Cloud does not start processes at priority 0, remove from test b85382d Rework Dockerfile and tests 24f7789 Merge pull request snabbco#199 from vavrusa/master 2ecf486 linux/constants: added new BPF map and prog types 00c1949 use Alpine 3.4 for docker build 0bcafc6 Merge pull request snabbco#196 from kbara/removecunused a4217a8 Merge pull request snabbco#195 from kbara/fixgetcpu ee67430 Remove unused variables from c.lua 4b2e0b2 Fix getcpu: the cpu argument was incorrectly given the node variable previously 214550a update changelog and rockspec for 0.12 release 21f3fd8 Merge pull request snabbco#192 from vavrusa/master 0da437f linux/bpf_prog_load: support custom kernel builds 9981190 fix missing vhangup f245114 Merge pull request snabbco#191 from vavrusa/linux-perf-open 93558c1 linux: added support for tracing/performance counters 0511fb8 Merge pull request snabbco#190 from vavrusa/master 1f141ca linux: added new constants (e.g. attach BPF to socket) 36274f3 linux/bpf: added strflag support 4fd3bd6 Merge pull request snabbco#189 from vavrusa/master aaa89cb linux: added support for eBPF e095295 linux: added new syscall numbers (up to __nr_bpf) 1e079c4 test calls container so just needs file to run d92625e define a docker compose test 5f14711 Merge pull request snabbco#188 from aperezdc/fix-if-nameindex 8915a83 Close socket immediately after error in if_nametoindex() 96c0286 use addons for travis, as learned at fosdem 71241e0 update changelog for unreleased changes 559b499 ignore audit arch constant 5d867d3 new architectures do not have open, will use openat 49d9ff9 test issues with new constants 7cd460e aarch64 audit constants for seccomp 6249e99 more docker examples dd00af9 rockspec fix d20033b rockspec for new release fb17244 update copyright years 9d87597 more Changelog tweaks before release 53856b0 typo 1881526 sometimes winsz is 0, eg if terminal not set up ab0c08d add error message a1c207e update Changelog for forthcoming 0.11 release ce12fb2 addDocker hub to README 602a2b3 fix osx fstatat to use 32 bit stat type, as cannot find how to call to get 64 bit one 8a0a6ad Now have arm machines with working seccomp d14bd38 Appears that setting maac address on lo often works 6e878a1 remove debug print from test 30e9b5b allow skip on EPERM for adjtimex bff3e90 Add strace in Docker image for convenient debugging e67fa31 Use alpine 3.3 for Docker 5148bc3 fix ipv6 tests 185c1a6 more failures with no ipv6 98cc9a2 more fixes for ipv4 only environments for netlink tests 248a935 fix bind errors in environments that do not support ipv6 0eacd64 clean up travis file 5fb71b6 switch to newer Ubuntu in travis 8235724 fix more constant checks not in headers 8dff4ef constants missing on travis db51b08 update Changelog 7540b04 add new rtnetlink values, so tests work under docker 22604b8 remove test that fails in some environments d431693 fix waitid test under docker 1595b7d fix swap test under docker 321fdd2 Now an alpine package available 984b533 Add Dockerfile, fix some of the tests that made unreasonable assumptions 9aeff88 recent osx has *at functions 18cd829 better handling for xattr errors b6bb892 freebsd 11 now has utimensat 7065b0d on freebsd/zfs chflags will fail, skip git-subtree-dir: lib/ljsyscall git-subtree-split: e587f8c55aad3955dddab3a4fa6c1968037b5c6e
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This set of changes implements the performance improvements of the lib/protocol framework and the VPWS app that I've outlined on the mailing list some time ago. The most important improvements are
-- a "recycling" mechanism for object instances through the "class.lua" interface
-- a binding to libpcap for efficient packet matching
The run time behaviour of apps/vpn/vpws.lua still exhibits various different states regarding the percentage of compiled, interpreted and GC code when run repeatedly, which I find hard to understand. The code changes in other parts of the Snabb Switch influence this as well.