Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apply some form of fuzz testing to the package functions #439

Open
mcol opened this issue Nov 19, 2024 · 3 comments
Open

apply some form of fuzz testing to the package functions #439

mcol opened this issue Nov 19, 2024 · 3 comments

Comments

@mcol
Copy link
Contributor

mcol commented Nov 19, 2024

A way to harden the package against misspecified inputs is by calling all functions with a variety of given inputs and record the failures. Rather than concentrating on a large number of random inputs, for our needs at this stage it's sufficient to just test our large number of functions on the same inputs to reveal lacks in our input validation or elsewhere.

I've written this simple package, to implement this: https://mcol.github.io/caught-by-the-fuzz/

This takes advantage of the fact that when we have handled an error, we report the function name in the error message. So by ignoring those errors, we are left with the unhandled errors. These are effectively already unit tests that could be added to our test suite.

This is meant as a tracking bug for this work.

@mcol mcol added this to the Release version 1.0.0 milestone Nov 19, 2024
@mcol mcol added the fuzzing label Nov 19, 2024
@RLumSK
Copy link
Member

RLumSK commented Nov 19, 2024

👏 You didn't tell me that you wrote a package, you just said a few lines ... :D

@mcol
Copy link
Contributor Author

mcol commented Nov 19, 2024

It's literally a few lines (~50 at the moment). The package is just for convenience of installation! :)

@RLumSK
Copy link
Member

RLumSK commented Nov 19, 2024

No, but great, this fits actually perfect into what we discussed earlier!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants