Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate test checker executable #106

Merged
merged 15 commits into from
Nov 20, 2023
Merged

Generate test checker executable #106

merged 15 commits into from
Nov 20, 2023

Conversation

tahina-pro
Copy link
Member

This PR introduces 3D option --test_checker, which, when followed by a parser name of the form Modulename.parsername, generates a test executable, test.exe, so that ./test.exe filename.dat arg1 arg2... opens the file filename.dat and runs the validator for Modulename.parsername, with arguments arg1, arg2..., on the binary data contained in that file, and returns 0 if the data in the file passes validation, 1 if it fails validation, 2 for any other error (wrong number of arguments, file is missing, cannot mmap, etc.)

The file passed to test.exe is assumed to contain all of the input data to be passed to the validator, in binary form. If the user wants to start from some filename.hex that contains hex data instead (e.g. .pcap), it is the responsibility of the user to first run something like xxd -r -p filename.hex filename.dat to convert that hex file to a binary file before passing the latter to test.exe

This should fix #105

@tahina-pro tahina-pro added enhancement New feature or request 3d The 3d dependent data description language and frontend labels Oct 25, 2023
@tahina-pro tahina-pro requested a review from nikswamy October 25, 2023 21:44
@nikswamy
Copy link
Contributor

Thanks for adding the error handler!

@tahina-pro
Copy link
Member Author

The generated test checker now rejects test files that contain data beyond a valid object: in other words, if the validator succeeds but does not read all input, then the test checker now considers the test failing, and returns status code 1. Validator failure returns status code 2, and 3 for any other kind of errors.

@tahina-pro
Copy link
Member Author

The --z3_test and --z3_diff_test modes now output each test case generated by Z3 into a raw binary file, whose name is of the form witness.nnn.POS.validatorname.arg1.arg2.dat and witness.nnn.NEG.validatorname.arg1.arg2.dat for positive and negative test cases generated by --z3_test , and witness.nnn.POS.validatorname1.NEG.validatorname2.arg1.arg2.arg3.dat for differential test cases generated by --z3_diff_test. Those are raw binary files, so it is the responsibility of the user to run xxd -p on those files to convert them to hexadecimal format.

@tahina-pro tahina-pro merged commit 1daf54a into master Nov 20, 2023
3 checks passed
@lemmy
Copy link
Contributor

lemmy commented Nov 23, 2023

There seems to be a off-by-one of some sorts s.t. the first negative witness is actually a positive one.

@lemmy ➜ /workspaces/RFC-3D/foo2 (mku-tshark) $ for f in witness*; do echo $f && hexdump $f; done
witness.0.POS.VxlanValidateVxlanHeader.dat
0000000 0808 0808 0808                         
0000006
witness.1.POS.VxlanValidateVxlanHeader.dat
0000000 0f0f 0f00 0f0f                         
0000006
witness.2.POS.VxlanValidateVxlanHeader.dat
0000000 0018 0000 0000                         
0000006
witness.3.NEG.VxlanValidateVxlanHeader.dat <- payload same as 0.Pos
0000000 0808 0808 0808                         
0000006
witness.4.NEG.VxlanValidateVxlanHeader.dat
0000000 0000                                   
0000001
witness.5.NEG.VxlanValidateVxlanHeader.dat
0000000 0001                                   
0000001

@tahina-pro tahina-pro deleted the _taramana_3d_test_exe branch October 25, 2024 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3d The 3d dependent data description language and frontend enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend 3d test infrastructure to accept a module and a witness
3 participants