Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement simple benchmarks for instantiation. #2900

Merged

Conversation

peterhuene
Copy link
Member

This adds benchmarks around module instantiation using criterion.

Both the default (i.e. on-demand) and pooling allocators are tested
sequentially and in parallel using a thread pool.

Instantiation is tested with an empty module, a module with a single page
linear memory, a larger linear memory with a data initializer, and a "hello
world" Rust WASI program.

@peterhuene
Copy link
Member Author

Note: as the pooling/uffd allocator requires multiple instantiations from the same engine instance, these benchmarks were not implemented with sightglass yet.

@abrown
Copy link
Contributor

abrown commented May 12, 2021

(I'm just waiting to see if cargo-deny blows up like it did for me over in #2888...)

@peterhuene
Copy link
Member Author

It did indeed :) There's no rush on merging this, so I'll do so after your PR goes in.

@alexcrichton
Copy link
Member

FWIW from CI:

sequential/default/empty.wat
                        time:   [3.0138 us 3.0327 us 3.0524 us]
sequential/default/small_memory.wat
                        time:   [9.7558 us 9.9036 us 10.053 us]
sequential/default/data_segments.wat
                        time:   [20.643 us 21.009 us 21.425 us]
sequential/default/wasi.wasm
                        time:   [29.953 us 30.328 us 30.756 us]
sequential/pooling/empty.wat
                        time:   [3.1582 us 3.1839 us 3.2118 us]
sequential/pooling/small_memory.wat
                        time:   [5.2716 us 5.3406 us 5.4295 us]
sequential/pooling/data_segments.wat
                        time:   [14.172 us 14.299 us 14.444 us]
sequential/pooling/wasi.wasm
                        time:   [26.612 us 26.868 us 27.165 us]
parallel/default/1000 instances with 1 thread
                        time:   [33.318 ms 33.611 ms 33.904 ms]
parallel/default/1000 instances with 2 threads
                        time:   [21.400 ms 21.664 ms 21.949 ms]
parallel/pooling/1000 instances with 1 thread
                        time:   [29.697 ms 30.111 ms 30.531 ms]
parallel/pooling/1000 instances with 2 threads
                        time:   [23.463 ms 23.723 ms 24.012 ms]
sequential/uffd/empty.wat
                        time:   [2.8096 us 2.8404 us 2.8799 us]
sequential/uffd/small_memory.wat
                        time:   [3.5588 us 3.5888 us 3.6222 us]
sequential/uffd/data_segments.wat
                        time:   [3.7279 us 3.7799 us 3.8324 us]
sequential/uffd/wasi.wasm
                        time:   [11.984 us 12.179 us 12.396 us]
parallel/uffd/1000 instances with 1 thread
                        time:   [14.558 ms 14.702 ms 14.848 ms]
parallel/uffd/1000 instances with 2 threads
                        time:   [11.417 ms 11.549 ms 11.694 ms]

@abrown
Copy link
Contributor

abrown commented May 13, 2021

#2888 is merged so perhaps a rebase would make the CI go green (I kicked off the CI in case that works...).

This adds benchmarks around module instantiation using criterion.

Both the default (i.e. on-demand) and pooling allocators are tested
sequentially and in parallel using a thread pool.

Instantiation is tested with an empty module, a module with a single page
linear memory, a larger linear memory with a data initializer, and a "hello
world" Rust WASI program.
@peterhuene peterhuene force-pushed the benchmark-instantiation branch from 65b71e5 to 1b8efa7 Compare May 14, 2021 02:27
@peterhuene peterhuene merged commit 18c61cd into bytecodealliance:main May 17, 2021
@peterhuene peterhuene deleted the benchmark-instantiation branch May 17, 2021 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants