fix: improve error handling in global fixtures #5208

[TG199]

PR Checklist

• Addresses an existing open issue: fixes 🐛 Bug: Error in mochaGlobalTeardown is swallowed and exits with code 0 #5208
• That issue was marked as status: accepting prs
• Steps in CONTRIBUTING.md were taken

Overview

Description
This PR addresses an issue with the handling of errors in global teardown functions. Previously, if a global teardown function threw an error, the error was not logged clearly, and the process exit code might not reflect the failure.

Changes Made
Updated the _runGlobalFixtures method in lib/mocha.js to:
Log errors with a clear and descriptive message.
Set the process exit code to 1 when a global teardown function fails.
Improved debugging output to aid in identifying which function caused the error.
New Behavior
When a global teardown function fails:
A descriptive error message is logged.
The process exits with a non-zero code, signaling the failure.

[TG199]

All tests pass locally using npm run test-node:unit on both the main branch and my working branch. However, they fail in CI. This suggests an environment-specific issue. I'm investigating further and would appreciate any guidance on differences between the CI and local setups.

[JoshuaKGoldberg]

All tests pass locally using npm run test-node:unit on both the main branch and my working branch. However, they fail in CI. This suggests an environment-specific issue. I'm investigating further and would appreciate any guidance on differences between the CI and local setups.

Ah, this looks like #5278. Sorry for the confusing errors here - you can ignore them.

[JoshuaKGoldberg]

A good start, thanks for sending this! I think there's some work to be done to flesh the PR out.

[JoshuaKGoldberg]

[Functionality] This is a good start, but I think we'll want to do more than just log the error to the console. I think we'll want to mark the test as failed too. It's easy to miss console logs that don't change the exit code.

[JoshuaKGoldberg]

[Testing] This change is missing test coverage. It's great that existing tests pass, but there should be a test added that shows Mocha acting appropriately when an a global fixture does reject.

[JoshuaKGoldberg]

@TG199 this is still an open piece of feedback, can you add tests please?

[TG199]

Thanks for the review 🙏🏽. I'll make the necessary changes asap.

[JoshuaKGoldberg]

👋 Just checking in @TG199, did you mean to re-request review?

[TG199]

Oh, actually yes, had no idea I could request for a review, thanks!

[JoshuaKGoldberg]

Whenever tests are added, feel free to re-request review and I can take a look :)

[JoshuaKGoldberg]

@TG199 this is still an open piece of feedback, can you add tests please?

[TG199]

hi @JoshuaKGoldberg, need a little help here, I've been running into challenges trying to mark failed test. these are the different approaches I've tried:
First attempt - Trying to use runner.failures:

Mocha.prototype._runGlobalFixtures = async function _runGlobalFixtures(
fixtureFns = [],
context = {}
) {
try {
await fixtureFn.call(context);
} catch (err) {
console.error(err.stack);
if (this.runner) {
this.runner.failures++;
}
throw err;
}
}
However, this.runner isn't available in this context

Second attempt - Using a fakeHook this.runner.failHook:

// In _runGlobalFixtures
if (this.runner) {
const falseHook = new this._Hook("globalTeardown", "global teardown");
this.runner.failHook(falseHook, err);
}
this is when I noticed this.runner is not available here. !!this.runner (logged: false)

Current attempt - Using a flag and run override:

Mocha.prototype._runGlobalFixtures = async function _runGlobalFixtures(
fixtureFns = [],
context = {},
) {
for (const fixtureFn of fixtureFns) {
try {
await fixtureFn.call(context);
} catch (err) {
console.error('Global fixture error:', err.stack);
this.globalTeardownErrored = true;
}
}
return context;
};

const originalRun = Mocha.prototype.run;
Mocha.prototype.run = function (fn) {
const self = this;
return originalRun.call(this, (exitCode) => {
const finalExitCode = exitCode > 0 || self.globalTeardownErrored ? 1 : 0;
if (fn) fn(finalExitCode);
});
};
This works for getting the exit code right, but I'm not sure if this is the best approach for integrating with Mocha's test reporting.
Questions:

What's the recommended way to access the runner or mark tests as failed from within _runGlobalFixtures?
Is there a better way to track these failures that would integrate better with Mocha's reporting system?
Should I be handling this at a different point in Mocha

Any guidance would be greatly appreciated! Thank you!

[JoshuaKGoldberg]

Glad to see you're making progress on this! A couple quick tips to help me help you...

First, it's easier to read code snippets if you format code blocks as code. See https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code. A lot easier to read 🙂

Second, it's hard to help from just some standalone code. Can you try pushing your attempt(s) up as Git branches or commits?

What's the recommended way to access the runner or mark tests as failed from within _runGlobalFixtures?

Generally, the context parameter that it takes in. Although the docs on https://mochajs.org/api/mocha#globalTeardown -> https://mochajs.org/api/global#MochaGlobalFixture don't mention a parameter, you can see in code that it's given one:

mocha/lib/mocha.js

Lines 1228 to 1231 in deb8679

	Mocha.prototype._runGlobalFixtures = async function _runGlobalFixtures(
	fixtureFns = [],
	context = {}
	) {

I also found this this by:

1. Putting a console.log(new Error().stack) in the global-fixture example's mochaGlobalTeardown: https://github.com/mochajs/mocha-examples/blob/48f723f4a6f28870ae1c49cffb74d228786b11e6/packages/global-fixture/fixture.js#L27: to see that it's called by mocha/lib/mocha.js:1234
2. Putting a console.log(context) in `mocha/lib/mocha.js:1234: to see what's in the context

Is there a better way to track these failures that would integrate better with Mocha's reporting system?
Should I be handling this at a different point in Mocha

I'm not sure but it seems reasonable to me to look there. runGlobalTeardown -> _runGlobalFixtures is the main pair of functions that calls the teardown fixtures. Either they or something that calls them would be a good place to try adding the logic.

Something to consider: runGlobalTeardown is specific to teardown; _runGlobalFixtures is for both setup and teardown. Maybe that's a hint we should also handle setup functions too? What if they error?

[TG199]

Glad to see you're making progress on this! A couple quick tips to help me help you...

First, it's easier to read code snippets if you format code blocks as code. See https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code. A lot easier to read 🙂

I think I need to checkout Github docs more often

Second, it's hard to help from just some standalone code. Can you try pushing your attempt(s) up as Git branches or commits?

What's the recommended way to access the runner or mark tests as failed from within _runGlobalFixtures?

Generally, the context parameter that it takes in. Although the docs on https://mochajs.org/api/mocha#globalTeardown -> https://mochajs.org/api/global#MochaGlobalFixture don't mention a parameter, you can see in code that it's given one:

mocha/lib/mocha.js

Lines 1228 to 1231 in deb8679

	Mocha.prototype._runGlobalFixtures = async function _runGlobalFixtures(
	fixtureFns = [],
	context = {}
	) {

I also found this this by:

1. Putting a console.log(new Error().stack) in the global-fixture example's mochaGlobalTeardown: https://github.com/mochajs/mocha-examples/blob/48f723f4a6f28870ae1c49cffb74d228786b11e6/packages/global-fixture/fixture.js#L27: to see that it's called by mocha/lib/mocha.js:1234
2. Putting a console.log(context) in `mocha/lib/mocha.js:1234: to see what's in the context

Oh, okay, I try this out.

Is there a better way to track these failures that would integrate better with Mocha's reporting system?
Should I be handling this at a different point in Mocha

I'm not sure but it seems reasonable to me to look there. runGlobalTeardown -> _runGlobalFixtures is the main pair of functions that calls the teardown fixtures. Either they or something that calls them would be a good place to try adding the logic.

Something to consider: runGlobalTeardown is specific to teardown; _runGlobalFixtures is for both setup and teardown. Maybe that's a hint we should also handle setup functions too? What if they error?

True, I'll keep that in mind, infinite thanks!

[TG199]

Hi @JoshuaKGoldberg will this do? 🙂