Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tyche logging support #258

Merged
merged 4 commits into from
Sep 9, 2024
Merged

Tyche logging support #258

merged 4 commits into from
Sep 9, 2024

Conversation

vasumv
Copy link
Collaborator

@vasumv vasumv commented Aug 30, 2024

Resolves #257

Logging support for Tyche (feature flag enabled by -DjqfObservability). The output file will be target/fuzz-results/TEST_CLASS/TEST_METHOD/observations.jsonl. The output file can then be uploaded to
https://tyche-pbt.github.io/tyche-extension/

Modifies FuzzStatement to add logging at the end of the guidance loop if the flag is enabled. Adds the fuzz.util.Observability.event method that can be invoked in the fuzz target to log specific input features. Per-input coverage still not yet implemented. Since the coverage data in Tyche is per-input line coverage for each file, this should be done as a post processing step.

Example fuzz target (for ChocoPy):

@Fuzz
public void fuzzSemanticAnalysis(@From(ChocoPySemanticGenerator.class) String code) {
    Program program = RefParser.process(code, false);
    assumeTrue(!program.hasErrors());
    event("numStatements", program.statements.size());
    event("numDeclarations", program.declarations.size());
    Program typedProgram = RefAnalysis.process(program);
    event("numErrors", program.getErrorList().size());
    assumeTrue(!typedProgram.hasErrors());
}

Example visualization:
image

@vasumv vasumv self-assigned this Aug 30, 2024
@vasumv vasumv marked this pull request as ready for review September 5, 2024 14:54
@vasumv vasumv requested a review from aoli-al September 5, 2024 18:33
Copy link
Owner

@rohanpadhye rohanpadhye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@vasumv vasumv merged commit cb44a7b into master Sep 9, 2024
1 check passed
@vasumv vasumv deleted the vasu/tyche_support branch September 9, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add logging support for Tyche
3 participants