Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: show extra information on failure #363

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

andrewliebenow
Copy link
Contributor

No description provided.

@andrewliebenow
Copy link
Contributor Author

Old:

---- tr::tr_octal_above_one_byte_value stdout ----
thread 'tr::tr_octal_above_one_byte_value' panicked at plib/src/testing.rs:85:5:
assertion `left == right` failed
  left: "(Ł)"
 right: "Ł)"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

New:

---- tr::tr_octal_above_one_byte_value stdout ----
thread 'tr::tr_octal_above_one_byte_value' panicked at plib/src/testing.rs:229:9:
------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Test failed with 2 total failure types
--------------------------------------------------------------------------------

Command executed
--------------------------------------------------------------------------------
tr -d \501
--------------------------------------------------------------------------------

Failure 1 of 2: stdout differs from what was expected
--------------------------------------------------------------------------------

Actual
--------------------------------------------------------------------------------
(Ł)
--------------------------------------------------------------------------------

Expected
--------------------------------------------------------------------------------
Ł)
--------------------------------------------------------------------------------

Failure 2 of 2: stderr differs from what was expected
--------------------------------------------------------------------------------

Actual
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

Expected
--------------------------------------------------------------------------------
tr: warning: the ambiguous octal escape \501 is being interpreted as the 2-byte sequence \050, 1

--------------------------------------------------------------------------------


stderr, for diagnosing failure
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

@andrewliebenow
Copy link
Contributor Author

The new format might look a bit verbose, but I've repeatedly run up against:

  1. Not remembering what left/right correspond to (actual/expected)
  2. Not knowing if the comparison is in stdout or stderr
  3. Not having access to any additional information beyond that which is provided by the first failed assertion

@andrewliebenow andrewliebenow force-pushed the testing-add-failure-information branch 2 times, most recently from f81b4ac to 7bde6c9 Compare October 27, 2024 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant