Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testsNotRan to keep count of testsNotRan to ensure moduleDone is called #1417

Closed
wants to merge 1 commit into from

Conversation

step2yeung
Copy link
Contributor

Fixes #1416

This PR adds testsNotRan to a module. This is used to keep track of any tests where isValid() returns false.
This attribute will be used in this condition:

if ( ( module.testsRun + module.testsNotRan ) === numberOfTests( module ) ) {

checking for whether the module has completed running its tests.

src/test.js Outdated Show resolved Hide resolved
@step2yeung step2yeung force-pushed the moduleDoneFix branch 2 times, most recently from 4360c16 to 047da76 Compare November 21, 2019 22:50
src/test.js Outdated Show resolved Hide resolved
src/test.js Outdated Show resolved Hide resolved
@step2yeung step2yeung force-pushed the moduleDoneFix branch 2 times, most recently from 7afb00a to 21571b7 Compare November 23, 2019 04:51
@davecombs
Copy link

could 'testsNotRan' be changed to 'testsNotRun' to match 'testsRun'? It's inconsistent as it is now.

@step2yeung
Copy link
Contributor Author

@davecombs updated.
@scalvert @rwjblue any additional comments?

src/test.js Outdated Show resolved Hide resolved
src/core/config.js Outdated Show resolved Hide resolved
Copy link
Member

@Krinkle Krinkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per @rwjblue.

@step2yeung
Copy link
Contributor Author

step2yeung commented Jul 10, 2020

Finally got a chance to get back to this and got a simple fix to some of the problem I ran into previously.
TLDR: Renamed this testsNotRun to testsIgnored and removed unskippedTestsRun, updated the references to the variables.

One important note:
By removing unskippedTestsRun, the one slight behavior change is the after hook will be called after skipped tests. Prior to this change, the after hook is called on the last unskipped test.
Since I removed the concept of unskipped tests, I also removed the dependency on this. I believe this makes the code a bit easier to grok. I can change it back to the previous behavior.

@rwjblue care to take another look?
fyi @Krinkle

@Krinkle Krinkle self-assigned this Jul 10, 2020
src/test.js Show resolved Hide resolved
src/test.js Show resolved Hide resolved
Krinkle pushed a commit that referenced this pull request Aug 26, 2020
…filters

Previously, the `suiteEnd` event and the `moduleDone` callback
were fired when we've run all the tests, based on counting the
number of registered tests in a module and the number of tests
actually run.

This logic does not hold up when there are active module/test filters
used in the runner, because then some of the registered tests will
never run.

The added tests demonstrate the issue, as they will fail without
the src patch. Several of the `suiteEnd` events would not fire.

The impact of this bug was that the `suiteEnd` event and `moduleDone`
callback don't fire if there were active filters such as `--filters`
in the CLI or via the "Filter" field in the HTML Reporter.

This in turn meant that reporters such as TAP would sometimes
reflect the structure of the test suite in an odd way due to
it sometimes missing an "end" event for a group of tests.

Fixes #1416.
Closes #1417.
@@ -81,14 +81,10 @@ QUnit.module( "after (skip)", {
}
} );

QUnit.test( "does not run after initial tests", function( assert ) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that this is correct. Looking at it locally, this means the after hook never runs, which would be a regression. During the first/last actually running test, it previously checked if we're currently executing the last non-ignored test. Which meant that if the last registered test is skipped, the after hook will run correctly after the last actually run test.

However, that logic was removed as part of this patch, and thus we now get a similar bug but affecting the after hook rather than the moduleDone callback.

I think for this to work we need to continue to "look ahead" like it did before, so that it knows during this test that it will be the last.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, good catch @Krinkle! Let me see what I can do

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@step2yeung Friendly ping to check if you're able to find time for this. If not, I might give it a go next week for the next patch release. Thanks again!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder Krinkle! Will get back to it this week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Mismatched "suiteEnd" event when a filter is applied
6 participants