Skip to content

Commit

Permalink
[lldb][test] Mark gtest cases as XFAIL if the test suite is XFAIL (ll…
Browse files Browse the repository at this point in the history
…vm#102986)

When a test case inside of a gtest suite fails, we report a failure
which causes the entire `ninja check-lldb` invocation to fail, even if
the outer test case is marked as XFAIL - each test case result is
reported as its own lit test run. This PR updates lit so it checks
whether each test case's parent test suite is XFAIL before setting the
status to FAIL.

This is especially problematic because the failing tests can't manually
be marked as XFAIL, due to
llvm#102264.

Fixes llvm#102265

### Repro instructions

1. Modify any gtest test case to generate a failure.
2. Mark the outer lit test with XFAIL using either `--xfail-tests` flag
or `LIT_XFAIL` env var.
3. Run the tests
4. Observe the lit test is XFAIL as expected, but the failed child test
cases show up as separate failures.

Co-authored-by: kendal <kendal@thebrowser.company>
(cherry picked from commit 9f89d31)
  • Loading branch information
kendalharland authored and kendal committed Sep 12, 2024
1 parent b02ca46 commit 4abee04
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion llvm/utils/lit/lit/formats/googletest.py
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,11 @@ def remove_gtest(tests):
returnCode = lit.Test.SKIPPED
elif "failures" in testinfo:
has_failure_in_shard = True
returnCode = lit.Test.FAIL
returnCode = (
lit.Test.XFAIL
if test.isExpectedToFail()
else lit.Test.FAIL
)
output = header
for fail in testinfo["failures"]:
output += fail["failure"] + "\n"
Expand Down

0 comments on commit 4abee04

Please sign in to comment.