forge coverage doesn't fail on failing tests

[0xGuybrush]

Component

Forge

Have you ensured that all of these are up to date?

• Foundry
• Foundryup

What version of Foundry are you on?

forge 0.2.0 (28b2ae6 2023-03-06T00:04:30.253422Z)

What command(s) is the bug in?

forge coverage

Operating System

macOS (Apple Silicon)

Describe the bug

Description

Running forge coverage exits with code 0 even on test failures. This may be just a mismatch in my expections, but just used to running similar modes e.g. in JS testing tools where coverage would fail on any test failures.

If it's deliberate that coverage doesn't fail on failing tests (as in fairness the act of producing metric didn't fail), it could be nice to call it out in the Foundry book that it's a command to run coverage only, not to run tests with coverage.

Steps to reproduce:

• Create a failing test:

function testThatFalseIsTrue() public {
  assertEq(true, false, "This will fail");
}

• Run forge test — test fails as expected
• Run forge coverage

Current outcome

• forge coverage passes and prints coverage metrics. Returns 0 exit code

Expected outcome

• Highlights same test failures as forge test (or exits non-zero).

[llllvvuu]

Might this be related to #3900 ?

[0xGuybrush]

Might this be related to #3900 ?

sorry, you're right — I think it's a duplicate. I did try to search for an open issue, I'm not sure why I didn't see it. WIll close this one. Thanks!