-
Notifications
You must be signed in to change notification settings - Fork 375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add gnodev test -update-golden-tests
flag
#449
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great 💯
Thank you for the contribution as always 🙏
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you very much for this contrib. However, I think sync
is too ambiguous and risks conflicting with a potential upcoming feature.
what do you think about renaming --sync
to --update-golden-tests
?
Sure, that makes sense. I will update it. |
@moul can you approve this? |
gnodev test -update-golden-tests
flag
Signed-off-by: Manfred Touron <94029+moul@users.noreply.github.com>
Description
#440 3 of 3
This commit is the last step to fixing the false testing result during Github CI
We keep the feature to sync the actual output to test comments in files. It allowed us to see wanted outcomes overwritten by actual results during local testing. "Git status" shows modified testing files.
Added a syncWanted parameter to the RunFileTest() in tests/file.go and runFileTest() in tests/file_test.go. It makes the function contract clear without relying upon the side effect of the syncWanted variable while we still maintain the custom testing flag available in file_test.go for the Makefile to execute in CI.
Set package variable syncWanted as a golang custom testing flag --sync in tests/file_test.go. The default value is false. It allows the CI to stop at failure cases when executing file tests in the tests/files and tests/files2 folder
Added --sync flog in cmd/gnodev/test.go. The default value is false. It allows CI to stop at failure cases when executing file tests in ./examples
Added three entries in the Makefile. It allows us to turn on the sync flag during the local testing
test.files1.sync
test.files2.sync
test.examples.sync
To run tests/files and tests/files2 file test with sync on
To run file test in ./examples with sync on
How has this been tested?
Modify the realm hash in
tests/files/zrealm_example.gno
tests/files2/zrealm_example.gno
make test.files1 - failed
make test.files2 - failed
make test.files1.sync - passed
make test.files2.sync - passed
The correct hash value was written back to the file test comments.
Replaced print with println in examples/gno.land/r/demo/releases-example/release0_filetest.gno
make test.examples - failed
make test.examples.sync - passed
The correct rendering output was written back to the file test comments.