Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intel1g supporting multiple processes and RSS #897

Merged
merged 40 commits into from
Mar 6, 2017

Conversation

petebristow
Copy link
Contributor

Lots to be done, thoughts would be great, the tests are currently the best guide to usage, particularly test6.sh. It's fairly opinionated.

  1. We shouldn't rely on loopback mode for testing. i350 support has never worked in non loopback mode AFAICT
  2. selftest() and selftest.sh could do with being extended. The contained selftest.sh seems adequate for this.
  3. A separate process can be in charge of exceptional error counters rather doing them in the fastpath.
  4. Better commentary and datasheet references are required.

@plajjan
Copy link
Contributor

plajjan commented Apr 27, 2016

Big thumbs up for RSS :)
I'm not qualified to give any real feedback on this although I've skimmed through it - mostly out of curiosity. Did note 2 space indent which is different from the de facto 3 space indent.

@petebristow petebristow mentioned this pull request Apr 27, 2016
@petebristow
Copy link
Contributor Author

From a driver users perspective does it look like the kind of interface you would want, particularly if it was replicated into the 82599 driver?

@lukego
Copy link
Member

lukego commented Apr 28, 2016

Cool! Pioneering a new design for multiprocess drivers :-)

Personally I find the plumbing rather stylish with using flock to become the master, the NIC software-software semaphore to protect global register updates, and automatically provisioning queues that are used. This keeps the interface simple and the code short.

cc @hb9cwp who is surely affected by this driver rewrite and likely has feedback too?

@petebristow petebristow changed the title [wip] Intel1g supporting multiple processes and RSS Intel1g supporting multiple processes and RSS Jun 1, 2016
@petebristow
Copy link
Contributor Author

This driver is now in reasonably good shape. An assignee and review would be great.

@eugeneia eugeneia self-assigned this Jun 15, 2016
@eugeneia
Copy link
Member

@petebristow The top-level comment in intel1g.lua says this supports both Intel I350 and 82599 (awesome?), but it does not implement VMDq as does does the existing Intel10G driver, am I right? If my understanding is correct, maybe rename the module to intel_rss to avoid confusion?

A section about this driver in README.md would be nice. E.g., describe configuration, modes of operation etc.

} __attribute__((packed))
]])
assert(ffi.sizeof(rxdesc_t),
"sizeof(rxdesc_t)= ".. ffi.sizeof(rxdesc_t) .. ", but must be 16 Byte")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be assert(ffi.sizeof(rxdesc_t) == 16, ...)? Not sure how much sense the assertion makes though, in which cases would it not hold true?

@petebristow
Copy link
Contributor Author

This is the first driver that can support a herd of snabb processes. Do you envisage a need for a herd of VMDq processes as well? intel_rss probably doesn't make sense if this driver is going to get VMDq support. Obviously intel1g is also a poor choice as it supports 10g. intel_herd intel_mp ?

I've started to manage the herd with something a little like
https://gist.github.com/petebristow/1231338296c2dfe8c9a28d4939411aee

README.md better comments and some tests that exercise the 82599 portion are in the works.

@lukego Is the Mellanox driver going to take a similar approach to multiple processes?

@eugeneia
Copy link
Member

@petebristow I am fine with intel_herd or intel_mp. :-) I just skimmed the code and it looks fantastic (very straight forward). I think with the coming documentation and tests for 82599 this is ready to be merged!

@eugeneia
Copy link
Member

@petebristow If you merge #942 SnabbBot will run selftest.sh.

assert((self.ndesc %128) ==0,
"ndesc must be a multiple of 128 (for Rx only)") -- see 7.1.4.5

self.rxpackets = ffi.new("struct packet *[?]", self.ndesc)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the packet ctype is now public (actually depending on #996), the pattern ffi.new("struct packet *[?]",... could be written as

desq_t = ffi.typeof("$ *[?]", packet.packet_t)
self.rxpackets = ffi.new(desq_t, self.ndesq)

Cc @kbara

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. What do you see as the main advantage of doing it that way?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems cleaner to me to not invoke the C name directly, in case the ever definition changes. Also, if desq_t was defined module-wide, then the JIT would see one ctype instead of different ctypes for each app instance.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok; the latter is a good enough reason for me. :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind my advice above, as we have decided against exposing packet_t, see #996 (comment).

petebristow and others added 22 commits August 27, 2016 21:11
…ometimes it runs slow in absolute terms which needs timeline for further investigation, hence the 400000pps test. The test was written on a very fast consumer CPU rather than a slower per core server CPU, 200000 should pass on E5 chips ok
The timer() function now sets its new deadline based on the current time
rather than the previous deadline. This is more consistent with the
documentation and prevents timeouts from being "saved up" in the style
of a token bucket when the timer is not called for a period of time.
These are convenience functions for the most common timer use cases.

timeout(seconds) returns true after the time has elapsed.
throttle(seconds) returns true at regular intervals.
Fix a problem where engine.now() could return a stale value if the
engine is not running i.e. timestamp of the last breath.

Now the engine only uses its cached time value while the breathe loop is
active and otherwise it fetches the current time from the time source.

(Could also consider always returning the latest time without caching
but this would potentially make engine.now() too expensive...?)
This function is unused due to the previous commits.
…xported, mac address is export via a counter
@eugeneia eugeneia merged commit 8b664a2 into snabbco:master Mar 6, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants