Throwing in beforeAll is ignored

[pzcfg]

Describe the bug

I'm building out some tests for my project, and am trying to debug both the tests and the helper functions I'm building to setup the test states, etc.

While trying to debug why my tests weren't working, I discovered that if I throw an error inside of beforeAll, or return a promise that I reject, vitest seems to carry on as if everything completed successfully. This is making it extremely hard to debug tests, as well as disambiguate whether the problem is with my project, the test, or the test setup.

Reproduction

beforeAll(async () => {
   throw new Error("STOP")
})

or

beforeAll(async () => {
   return new Promise((resolve, reject) => { reject(); })
})

System Info

System:
    OS: macOS 12.0
    CPU: (10) arm64 Apple M1 Pro
    Memory: 97.80 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.1 - ~/.nvm/versions/node/v16.13.1/bin/node
    Yarn: 1.22.17 - ~/.nvm/versions/node/v16.13.1/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.1/bin/npm
  Browsers:
    Chrome: 100.0.4896.127
    Safari: 15.0
  npmPackages:
    vite: ^2.9.6 => 2.9.6 
    vitest: ^0.10.0 => 0.10.0

Used Package Manager

yarn

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

Everything works with your reproduction: meaning, tests fail.

https://stackblitz.com/edit/vitest-dev-vitest-xqtujg?file=test/suite.test.ts

[pzcfg]

Aah, I see what is happening. I have multiple describe() blocks, and even if the beforeAll() in the first block throws immediately, I don't see that error until all of the other tests have completed or timed out.

https://stackblitz.com/edit/vitest-dev-vitest-oycgdw?file=test%2Fsuite.test.ts

In our current environment, our tests or dependent on the effects of their previous ones (not ideal, I know, but it's how it is for now) so if any block fails I would like to shown the error immediately and probably exit out. Is there a configuration we can use for that?

[sheremet-va]

This is not possible. describe blocks exist to prevent that :)

Try moving your tests to the same describe block, or move beforeAll to the top level.

[pzcfg]

Would it at least be possible to show the thrown errors immediately? I can see console logs/errors/etc. as soon as they run instead of after all tests have ended/timed out, and I did notice that I could see the errors if I essentially wrapped all of the functions in

() => {
  try {
    ...
  } catch (e) { console.error(e); }
}

but that seems excessive to have to do for the purposes of faster results/debugging

https://stackblitz.com/edit/vitest-dev-vitest-oycgdw?file=test%2Fsuite.test.ts

[sheremet-va]

Would it at least be possible to show the thrown errors immediately?

Depends on what reporter do you use? The base reporter dynamically updates the tree and only the relevant part is visible until every test is processed.

We also have verbose reporter that shows the error immediately.

[sheremet-va]

I guess it would be possible to achieve with "bail" option from #1459. Closing this issue in favor of it.