Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving Handling of Custom Inputs #2422

Merged
merged 37 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
762461e
introducing MappingMutator
riesentoaster Jul 13, 2024
9571619
extending mapping mutators
riesentoaster Jul 17, 2024
fda42e7
adding example fuzzer
riesentoaster Jul 17, 2024
1f2507d
making crossover mutators more flexible.
riesentoaster Jul 23, 2024
529e399
Merge remote-tracking branch 'upstream/main' into mapping-mutator
riesentoaster Jul 23, 2024
e4ab632
moving example fuzzer
riesentoaster Jul 23, 2024
1769ff1
fixing dependency paths
riesentoaster Jul 23, 2024
11801f5
formatting
riesentoaster Jul 23, 2024
84d704e
fixing no std error
riesentoaster Jul 23, 2024
5f6c689
fixing broken docs link
riesentoaster Jul 23, 2024
ac7dfb9
fixing import paths
riesentoaster Jul 23, 2024
4df0d65
fixing imports
riesentoaster Jul 23, 2024
dbbe477
more format fixing
riesentoaster Jul 23, 2024
8a50e0a
adding new example fuzzer to CI
riesentoaster Jul 23, 2024
2c19717
fixing further imports
riesentoaster Jul 23, 2024
a672041
fixing formatting
riesentoaster Jul 23, 2024
4d3b8a1
formatting fixes
riesentoaster Jul 23, 2024
66a33d8
Merge remote-tracking branch 'upstream/main' into mapping-mutator
riesentoaster Aug 7, 2024
abc2458
improving docs for the example fuzzer
riesentoaster Aug 7, 2024
86e072b
adding documentation and tests to mapping mutators
riesentoaster Aug 7, 2024
d871728
make extraction function for mapped crossover mutators more general
riesentoaster Sep 5, 2024
5f1c5bc
adding MutVecFunctionMappingMutator
riesentoaster Sep 7, 2024
1171cc0
Introducing WrapsReference
riesentoaster Sep 9, 2024
3a40f76
code cleanup for mapping mutators
riesentoaster Sep 10, 2024
afc5259
adding tests and docs to mapping mutators
riesentoaster Sep 11, 2024
d9cb391
Merge remote-tracking branch 'upstream/main' into mapping-mutator
riesentoaster Sep 11, 2024
fc1943f
reformatting comments
riesentoaster Sep 11, 2024
2d257b9
fixing merging of mutators in example fuzzer
riesentoaster Sep 11, 2024
147e88b
formatting
riesentoaster Sep 11, 2024
4ea8b72
formatting v2
riesentoaster Sep 11, 2024
b6e2af1
cleanup according to PR comments
riesentoaster Sep 11, 2024
4c8fe8f
Merge branch 'main' into mapping-mutator
domenukk Sep 11, 2024
61f3822
adding type constraint to MappedInput helper functions to remove the …
riesentoaster Sep 18, 2024
4411d6b
matching functions passed to mapped_havoc_mutations
riesentoaster Sep 18, 2024
745945e
removing unnecessary constraints
riesentoaster Sep 18, 2024
41de711
mapping mutators now contain the name of their inner mutator
riesentoaster Sep 18, 2024
3afb25a
Merge branch 'main' into mapping-mutator
riesentoaster Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ jobs:
- ./fuzzers/baby/baby_fuzzer_grimoire
- ./fuzzers/baby/baby_fuzzer_gramatron
- ./fuzzers/baby/baby_fuzzer
- ./fuzzers/baby/baby_fuzzer_custom_input
- ./fuzzers/baby/baby_fuzzer_nautilus
# - ./fuzzers/baby/backtrace_baby_fuzzers
- ./fuzzers/baby/baby_fuzzer_unicode
Expand Down
2 changes: 1 addition & 1 deletion fuzzers/baby/baby_fuzzer/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use libafl::{
fuzzer::{Fuzzer, StdFuzzer},
generators::RandPrintablesGenerator,
inputs::{BytesInput, HasTargetBytes},
mutators::scheduled::{havoc_mutations, StdScheduledMutator},
mutators::{havoc_mutations::havoc_mutations, scheduled::StdScheduledMutator},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should re-export the og havoc_mutations to not break all code out there, what do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. That's a fair point. For code we control, I'd much rather have the import from the source (mutators::scheduled has nothing to do with havoc_mutations anymore).

Unfortunately, deprecating re-exports is not supported yet (see issue). So I guess we're stuck with just re-exporting them and add a comment telling people to do the other thing and explaining why the re-export is there. Maybe link the issue as well.

Opinions?

observers::StdMapObserver,
schedulers::QueueScheduler,
stages::mutational::StdMutationalStage,
Expand Down
24 changes: 24 additions & 0 deletions fuzzers/baby/baby_fuzzer_custom_input/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[package]
name = "baby_fuzzer_custom_input"
version = "0.1.0"
authors = ["Valentin Huber <contact@valentinhuber.me>"]
edition = "2021"

[features]
default = ["simple_interface"]
simple_interface = []

[profile.dev]
panic = "abort"

[profile.release]
panic = "abort"
lto = true
codegen-units = 1
opt-level = 3
debug = true

[dependencies]
libafl = { path = "../../../libafl/" }
libafl_bolts = { path = "../../../libafl_bolts/" }
serde = "*"
7 changes: 7 additions & 0 deletions fuzzers/baby/baby_fuzzer_custom_input/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Baby fuzzer

This is a minimalistic fuzzer demonstrating how to employ mapping mutators to use default mutators on custom inputs. Custom inputs are necessary when the input to your program is a combination of parts, especially when those parts have different data types. Check multipart inputs if you have an input consisting of multiple parts of the same datatype and you don't need your mutation scheduler to be able to select which mutation is performed on which part.

The fuzzer runs on a single core until a crash occurs and then exits. The tested program is a simple Rust function without any instrumentation. For real fuzzing, you will want to add some sort to add coverage or other feedback.

You can run this example using `cargo run`.
147 changes: 147 additions & 0 deletions fuzzers/baby/baby_fuzzer_custom_input/src/input.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
use std::{
borrow::Cow,
hash::{DefaultHasher, Hash, Hasher},
};

use libafl::{
corpus::CorpusId,
generators::Generator,
inputs::{BytesInput, HasTargetBytes, Input, MutVecInput},
mutators::{MutationResult, Mutator},
prelude::RandBytesGenerator,
state::HasRand,
Error, SerdeAny,
};
use libafl_bolts::{rands::Rand, Named};
use serde::{Deserialize, Serialize};

/// The custom [`Input`] type used in this example, consisting of a byte array part, a byte array that is not always present, and a boolean
///
/// Imagine these could be used to model command line arguments for a bash command, where
/// - `byte_array` is binary data that is always needed like what is passed to stdin,
/// - `optional_byte_array` is binary data passed as a command line arg, and it is only passed if it is not `None` in the input,
/// - `boolean` models the presence or absence of a command line flag that does not require additional data
#[derive(Serialize, Deserialize, Clone, Debug, Hash, SerdeAny)]
pub struct CustomInput {
pub byte_array: Vec<u8>,
pub optional_byte_array: Option<Vec<u8>>,
pub boolean: bool,
}

/// Hash-based implementation
impl Input for CustomInput {
fn generate_name(&self, _id: Option<CorpusId>) -> String {
let mut hasher = DefaultHasher::new();
self.hash(&mut hasher);
format!("{:016x}", hasher.finish())
}
}

impl CustomInput {
/// Returns a mutable reference to the byte array
pub fn byte_array_mut(&mut self) -> MutVecInput<'_> {
(&mut self.byte_array).into()
}

/// Returns an immutable reference to the byte array wrapped in [`Some`]
pub fn byte_array_optional<'a>(&'a self) -> &'a [u8] {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name is no longer correct

&self.byte_array
}

/// Returns a mutable reference to the optional byte array
pub fn optional_byte_array_mut(&mut self) -> Option<MutVecInput<'_>> {
self.optional_byte_array.as_mut().map(|e| e.into())
}

/// Returns an immutable reference to the optional byte array
pub fn optional_byte_array_optional<'a>(&'a self) -> Option<&'a [u8]> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably also drop this extra optional dude here

self.optional_byte_array.as_deref()
}
}

/// A generator for [`CustomInput`] used in this example
pub struct CustomInputGenerator {
pub max_len: usize,
}

impl CustomInputGenerator {
/// Creates a new [`CustomInputGenerator`]
pub fn new(max_len: usize) -> Self {
Self { max_len }
}
}

impl<S> Generator<CustomInput, S> for CustomInputGenerator
where
S: HasRand,
{
fn generate(&mut self, state: &mut S) -> Result<CustomInput, Error> {
let mut generator = RandBytesGenerator::new(self.max_len);

let byte_array = generator.generate(state).unwrap().target_bytes().into();
let optional_byte_array = state
.rand_mut()
.coinflip(0.5)
.then(|| generator.generate(state).unwrap().target_bytes().into());
let boolean = state.rand_mut().coinflip(0.5);

Ok(CustomInput {
byte_array,
optional_byte_array,
boolean,
})
}
}

/// [`Mutator`] that toggles the optional byte array of a [`CustomInput`], i.e. sets it to [`None`] if it is not, and to a random byte array if it is [`None`]
pub struct ToggleOptionalByteArrayMutator<G> {
generator: G,
}

impl<S> ToggleOptionalByteArrayMutator<RandBytesGenerator<S>>
where
S: HasRand,
{
/// Creates a new [`ToggleOptionalByteArrayMutator`]
pub fn new(length: usize) -> Self {
Self {
generator: RandBytesGenerator::new(length),
}
}
}

impl<G, S> Mutator<CustomInput, S> for ToggleOptionalByteArrayMutator<G>
where
S: HasRand,
G: Generator<BytesInput, S>,
{
fn mutate(&mut self, state: &mut S, input: &mut CustomInput) -> Result<MutationResult, Error> {
input.optional_byte_array = match input.optional_byte_array {
None => Some(self.generator.generate(state)?.target_bytes().into()),
Some(_) => None,
};
Ok(MutationResult::Mutated)
}
}

impl<G> Named for ToggleOptionalByteArrayMutator<G> {
fn name(&self) -> &Cow<'static, str> {
&Cow::Borrowed("ToggleOptionalByteArrayMutator")
}
}

/// [`Mutator`] that toggles the boolean field in a [`CustomInput`]
pub struct ToggleBooleanMutator;

impl<S> Mutator<CustomInput, S> for ToggleBooleanMutator {
domenukk marked this conversation as resolved.
Show resolved Hide resolved
fn mutate(&mut self, _state: &mut S, input: &mut CustomInput) -> Result<MutationResult, Error> {
input.boolean = !input.boolean;
Ok(MutationResult::Mutated)
}
}

impl Named for ToggleBooleanMutator {
fn name(&self) -> &Cow<'static, str> {
&Cow::Borrowed("ToggleBooleanMutator")
}
}
Loading