Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add net benchmark harness #21

Merged
merged 3 commits into from
May 10, 2020
Merged

Add net benchmark harness #21

merged 3 commits into from
May 10, 2020

Conversation

willscott
Copy link
Contributor

Factor some common testing infrastructure for evaluating yamux and mplex.

Example usage of FindNetworkLimit is to wrap it in a test, e.g.

func TestSmallPackets(t *testing.T) {
	slowdown, err := test.FindNetworkLimit(testSmallPackets)
	if err != nil {
		t.Fatal(err)
	}
	if slowdown > 0.01 {
		t.Fatalf("Slowdown from was >1%%: %f", slowdown)
	}
}

Tests will be added in go-libp2p-yamux and go-libp2p-mplex wrapping this. example output looks something like:

$ go test -run TestSmallPackets . -cpu 4 -v -benchtime 10s
go test -run TestSmallPackets . -cpu 4 -v -benchtime 10s
=== RUN   TestSmallPackets
CPU Bound Limit:   756516	     15630 ns/op	14714113.83 MB/s
Latency Bound Limit: 1ms
Bandwidth Bound Limit: 102400Kbps
Network Bound Limit:   667627	     18475 ns/op	10986033.14 MB/s
At MaxProc 1 10240Kbps / 5ms latency:    51513	    234283 ns/op	66800.22 MB/s
At MaxProc 2 10240Kbps / 5ms latency:    51474	    233296 ns/op	67018.94 MB/s
At MaxProc 4 10240Kbps / 5ms latency:    51253	    467991 ns/op	33285.00 MB/s
Slowdown is 0.372082%
--- PASS: TestSmallPackets (68.04s)
PASS
ok  	github.com/libp2p/go-mplex	68.664s

The basic mechanism this harness is implementing is first identifying how much the boundary between CPU-bound and Network-bound limitations of a given testing operation. Then, on the network-bound side of that boundary, increasing parallelism and watching the ratio of how utilized the network link is.

Notes: the output from the wrapped testing.Benchmark calls don't account for threads/processes properly - the reported ns/op and MB/s does a racy division on number of spawned threads, as far as i can tell. it ends up not particularly useful, but the overall bytes pushed over the network is bench-marked correctly across parallelism.

@aarshkshah1992
Copy link

aarshkshah1992 commented May 1, 2020

@willscott Will review this early-mid net week. Does that work for you ?

@willscott
Copy link
Contributor Author

@willscott Will review this early-mid net week. Does that work for you ?

👍

@willscott
Copy link
Contributor Author

@aarshkshah1992 - are you wanting to review this?

@willscott
Copy link
Contributor Author

merging before this gets too stale. the additional test helpers seem fairly low risk.

@willscott willscott merged commit 99f31e5 into master May 10, 2020
@willscott willscott deleted the feat/netbench branch May 10, 2020 05:13
@aschmahmann aschmahmann mentioned this pull request Sep 22, 2020
72 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants