-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fuzz] Configure the differential
target
#4773
Conversation
This change is a follow-on from bytecodealliance#4515 to add the ability to configure the `differential` fuzz target by limiting which engines and modules are used for fuzzing. This is incredibly useful when troubleshooting, e.g., when an engine is more prone to failure, we can target that engine exclusively. The effect of this configuration is visible in the statistics now printed out from bytecodealliance#4739. Engines are configured using the `ALLOWED_ENGINES` environment variable. We can either subtract from the set of allowed engines (e.g., `ALLOWED_ENGINES=-v8`) or build up a set of allowed engines (e.g., `ALLOWED_ENGINES=wasmi,spec`), but not both at the same time. `ALLOWED_ENGINES` only configures the left-hand side engine; the right-hand side is always Wasmtime. When omitted, `ALLOWED_ENGINES` defaults to [`wasmtime`, `wasmi`, `spec`, `v8`]. The generated WebAssembly modules are configured using `ALLOWED_MODULES`. This environment variables works the same as above but the available options are: [`wasm-smith`, `single-inst`].
Here is an example from running the target locally: $ ALLOWED_MODULES=-wasm-smith cargo +nightly fuzz run differentia
...
=== Execution rate (384 successes / 2000 attempted modules): 19.20% ===
wasmi: 0.96%, spec: 8.85%, wasmtime: 83.97%, v8: 6.22%
wasm-smith: 0.00%, single-inst: 100.00% |
@jameysharp, you've had some opinions on this in the past so I tagged you for review (?): I really would have preferred to use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's true: I have, at times, had opinions! 😆
I don't feel very strongly about any of these review comments. This PR is fine as-is, so I'm marking this approved. But if you want to take a little more time on it I think these suggestions are improvements.
Subscribe to Label Actioncc @fitzgen
This issue or pull request has been labeled: "fuzzing"
Thus the following users have been cc'd because of the following labels:
To subscribe or unsubscribe from this label, edit the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you've addressed all my suggestions. Thanks! I hope you found these suggestions helpful.
Yeah, they were great! |
This change is a follow-on from #4515 to add the ability to configure
the
differential
fuzz target by limiting which engines and modules areused for fuzzing. This is incredibly useful when troubleshooting, e.g.,
when an engine is more prone to failure, we can target that engine
exclusively. The effect of this configuration is visible in the
statistics now printed out from #4739.
Engines are configured using the
ALLOWED_ENGINES
environment variable.We can either subtract from the set of allowed engines (e.g.,
ALLOWED_ENGINES=-v8
) or build up a set of allowed engines (e.g.,ALLOWED_ENGINES=wasmi,spec
), but not both at the same time.ALLOWED_ENGINES
only configures the left-hand side engine; theright-hand side is always Wasmtime. When omitted,
ALLOWED_ENGINES
defaults to [
wasmtime
,wasmi
,spec
,v8
].The generated WebAssembly modules are configured using
ALLOWED_MODULES
. This environment variables works the same as abovebut the available options are: [
wasm-smith
,single-inst
].