Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull latest commits from node core #43

Merged
merged 24 commits into from
Feb 2, 2023
Merged
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
012acb0
feat: verbous error when entire test tree is canceled
MoLow Aug 2, 2022
d885ee2
feat: support programmatically running `--test`
MoLow Aug 15, 2022
27241c3
fix: fix `duration_ms` to be milliseconds
MoLow Sep 4, 2022
6755536
feat: support using `--inspect` with `--test`
MoLow Sep 10, 2022
c50f844
fix: include stack of uncaught exceptions
MoLow Sep 14, 2022
1950b38
test: fix test-runner-inspect
MoLow Sep 14, 2022
c5fd64c
feat: add --test-name-pattern CLI flag
cjihrig Sep 1, 2022
46dce07
feat: add extra fields in AssertionError YAML
bengl Oct 26, 2022
0bfdb77
fix: call {before,after}Each() on suites
cjihrig Oct 27, 2022
08269c5
fix: report tap subtest in order
MoLow Oct 28, 2022
f2815af
fix: fix afterEach not running on test failures
MrJithil Nov 7, 2022
cff397a
fix: avoid swallowing of asynchronously thrown errors
fossamagna Nov 7, 2022
2e499ee
feat: support function mocking
cjihrig Apr 4, 2022
5f8ce61
feat: add initial TAP parser
manekinekko Nov 21, 2022
5ba2500
fix: remove stdout and stderr from error
cjihrig Nov 25, 2022
b942f93
feat: add getter and setter to MockTracker
fossamagna Nov 18, 2022
b3b384e
fix: don't use a symbol for runHook()
cjihrig Dec 6, 2022
71b659e
feat: add t.after() hook
cjihrig Dec 8, 2022
c0854ac
test: fix invalid output TAP if there newline in test name
pulkit-30 Dec 11, 2022
9b49978
chore: refactor `tap_parser` to use more primordials
aduh95 Dec 11, 2022
4e778bc
chore: refactor `tap_lexer` to use more primordials
aduh95 Dec 12, 2022
d1343a7
feat: parse yaml
MoLow Dec 13, 2022
c80e426
fix: run t.after() if test body throws
cjihrig Dec 17, 2022
9fa496b
test: fix mock.method to support class instances
ErickWendel Dec 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: add --test-name-pattern CLI flag
This commit adds support for running tests that match a
regular expression.

Fixes: nodejs/node#42984
(cherry picked from commit 87170c3f9271da947a7b33d0696ec4cf8aab6eb6)
  • Loading branch information
cjihrig authored and MoLow committed Feb 2, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit c5fd64cc2e2e22b35fd6c94d30dac34f281067ab
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -228,6 +228,41 @@ test('this test is not run', () => {
})
```

## Filtering tests by name

The [`--test-name-pattern`][] command-line option can be used to only run tests
whose name matches the provided pattern. Test name patterns are interpreted as
JavaScript regular expressions. The `--test-name-pattern` option can be
specified multiple times in order to run nested tests. For each test that is
executed, any corresponding test hooks, such as `beforeEach()`, are also
run.

Given the following test file, starting Node.js with the
`--test-name-pattern="test [1-3]"` option would cause the test runner to execute
`test 1`, `test 2`, and `test 3`. If `test 1` did not match the test name
pattern, then its subtests would not execute, despite matching the pattern. The
same set of tests could also be executed by passing `--test-name-pattern`
multiple times (e.g. `--test-name-pattern="test 1"`,
`--test-name-pattern="test 2"`, etc.).

```js
test('test 1', async (t) => {
await t.test('test 2');
await t.test('test 3');
});
test('Test 4', async (t) => {
await t.test('Test 5');
await t.test('test 6');
});
```

Test name patterns can also be specified using regular expression literals. This
allows regular expression flags to be used. In the previous example, starting
Node.js with `--test-name-pattern="/test [4-5]/i"` would match `Test 4` and
`Test 5` because the pattern is case-insensitive.

Test name patterns do not change the set of files that the test runner executes.

## Extraneous asynchronous activity

Once a test function finishes executing, the TAP results are output as quickly
7 changes: 7 additions & 0 deletions bin/node--test-name-pattern.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env node

const { argv } = require('#internal/options')

argv['test-name-pattern'] = true

require('./node-core-test.js')
5 changes: 5 additions & 0 deletions bin/node-core-test.js
Original file line number Diff line number Diff line change
@@ -10,9 +10,14 @@ const { argv } = require('#internal/options')

Object.assign(argv, minimist(process.argv.slice(2), {
boolean: ['test', 'test-only'],
string: ['test-name-pattern'],
default: Object.prototype.hasOwnProperty.call(argv, 'test') ? { test: argv.test } : undefined
}))

if (typeof argv['test-name-pattern'] === 'string') {
argv['test-name-pattern'] = [argv['test-name-pattern']]
}

process.argv.splice(1, Infinity, ...argv._)
if (argv.test) {
require('#internal/main/test_runner')
33 changes: 31 additions & 2 deletions lib/internal/test_runner/test.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
// https://github.com/nodejs/node/blob/cb7e0c59df10a42cd6930ca7f99d3acee1ce7627/lib/internal/test_runner/test.js
// https://github.com/nodejs/node/blob/87170c3f9271da947a7b33d0696ec4cf8aab6eb6/lib/internal/test_runner/test.js

'use strict'

const {
ArrayPrototypeMap,
ArrayPrototypePush,
ArrayPrototypeReduce,
ArrayPrototypeShift,
ArrayPrototypeSlice,
ArrayPrototypeSome,
ArrayPrototypeUnshift,
FunctionPrototype,
MathMax,
@@ -15,6 +17,7 @@ const {
PromisePrototypeThen,
PromiseResolve,
ReflectApply,
RegExpPrototypeExec,
SafeMap,
SafeSet,
SafePromiseAll,
@@ -33,7 +36,11 @@ const {
} = require('#internal/errors')
const { getOptionValue } = require('#internal/options')
const { TapStream } = require('#internal/test_runner/tap_stream')
const { createDeferredCallback, isTestFailureError } = require('#internal/test_runner/utils')
const {
convertStringToRegExp,
createDeferredCallback,
isTestFailureError
} = require('#internal/test_runner/utils')
const {
createDeferredPromise,
kEmptyObject
@@ -61,6 +68,15 @@ const kDefaultTimeout = null
const noop = FunctionPrototype
const isTestRunner = getOptionValue('--test')
const testOnlyFlag = !isTestRunner && getOptionValue('--test-only')
const testNamePatternFlag = isTestRunner
? null
: getOptionValue('--test-name-pattern')
const testNamePatterns = testNamePatternFlag?.length > 0
? ArrayPrototypeMap(
testNamePatternFlag,
(re) => convertStringToRegExp(re, '--test-name-pattern')
)
: null
const kShouldAbort = Symbol('kShouldAbort')
const kRunHook = Symbol('kRunHook')
const kHookNames = ObjectSeal(['before', 'after', 'beforeEach', 'afterEach'])
@@ -196,6 +212,18 @@ class Test extends AsyncResource {
this.timeout = timeout
}

if (testNamePatterns !== null) {
// eslint-disable-next-line no-use-before-define
const match = this instanceof TestHook || ArrayPrototypeSome(
testNamePatterns,
(re) => RegExpPrototypeExec(re, name) !== null
)

if (!match) {
skip = 'test name does not match pattern'
}
}

if (testOnlyFlag && !this.only) {
skip = '\'only\' option not set'
}
@@ -673,6 +701,7 @@ class ItTest extends Test {
return { ctx: { signal: this.signal, name: this.name }, args: [] }
}
}

class Suite extends Test {
constructor (options) {
super(options)
23 changes: 22 additions & 1 deletion lib/internal/test_runner/utils.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
// https://github.com/nodejs/node/blob/659dc126932f986fc33c7f1c878cb2b57a1e2fac/lib/internal/test_runner/utils.js
// https://github.com/nodejs/node/blob/87170c3f9271da947a7b33d0696ec4cf8aab6eb6/lib/internal/test_runner/utils.js
'use strict'
const { RegExpPrototypeExec } = require('#internal/per_context/primordials')
const { basename } = require('path')
const { createDeferredPromise } = require('#internal/util')
const {
codes: {
ERR_INVALID_ARG_VALUE,
ERR_TEST_FAILURE
},
kIsNodeError
} = require('#internal/errors')

const kMultipleCallbackInvocations = 'multipleCallbackInvocations'
const kRegExpPattern = /^\/(.*)\/([a-z]*)$/
const kSupportedFileExtensions = /\.[cm]?js$/
const kTestFilePattern = /((^test(-.+)?)|(.+[.\-_]test))\.[cm]?js$/

@@ -55,7 +57,26 @@ function isTestFailureError (err) {
return err?.code === 'ERR_TEST_FAILURE' && kIsNodeError in err
}

function convertStringToRegExp (str, name) {
const match = RegExpPrototypeExec(kRegExpPattern, str)
const pattern = match?.[1] ?? str
const flags = match?.[2] || ''

try {
return new RegExp(pattern, flags)
} catch (err) {
const msg = err?.message

throw new ERR_INVALID_ARG_VALUE(
name,
str,
`is an invalid regular expression.${msg ? ` ${msg}` : ''}`
)
}
}

module.exports = {
convertStringToRegExp,
createDeferredCallback,
doesPathMatchFilter,
isSupportedFileType,
1 change: 1 addition & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@
"bin": {
"node--test": "./bin/node--test.js",
"node--test-only": "./bin/node--test-only.js",
"node--test-name-pattern": "./bin/node--test-name-pattern.js",
"test": "./bin/node-core-test.js"
},
"imports": {
6 changes: 3 additions & 3 deletions test/message.js
Original file line number Diff line number Diff line change
@@ -13,7 +13,7 @@ const binPath = resolve(__dirname, '..', bin.test)
const MESSAGE_FOLDER = join(__dirname, './message/')
const WAIT_FOR_ELLIPSIS = Symbol('wait for ellispis')

const TEST_RUNNER_FLAGS = ['--test', '--test-only']
const TEST_RUNNER_FLAGS = ['--test', '--test-only', '--test-name-pattern']

function readLines (file) {
return createInterface({
@@ -109,8 +109,8 @@ const main = async () => {
)
.toString().split(' ')

const nodeFlags = flags.filter(flag => !TEST_RUNNER_FLAGS.includes(flag)).join(' ')
const testRunnerFlags = flags.filter(flag => TEST_RUNNER_FLAGS.includes(flag)).join(' ')
const nodeFlags = flags.filter(flag => !TEST_RUNNER_FLAGS.find(f => flag.startsWith(f))).join(' ')
const testRunnerFlags = flags.filter(flag => TEST_RUNNER_FLAGS.find(f => flag.startsWith(f))).join(' ')

const command = testRunnerFlags.length
? `${process.execPath} ${nodeFlags} ${binPath} ${testRunnerFlags} ${filePath}`
48 changes: 48 additions & 0 deletions test/message/test_runner_test_name_pattern.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
// https://github.com/nodejs/node/blob/87170c3f9271da947a7b33d0696ec4cf8aab6eb6/test/message/test_runner_test_name_pattern.js
// Flags: --no-warnings --test-name-pattern=enabled --test-name-pattern=/pattern/i
'use strict'
const common = require('../common')
const {
after,
afterEach,
before,
beforeEach,
describe,
it,
test
} = require('#node:test')

test('top level test disabled', common.mustNotCall())
test('top level skipped test disabled', { skip: true }, common.mustNotCall())
test('top level skipped test enabled', { skip: true }, common.mustNotCall())
it('top level it enabled', common.mustCall())
it('top level it disabled', common.mustNotCall())
it.skip('top level skipped it disabled', common.mustNotCall())
it.skip('top level skipped it enabled', common.mustNotCall())
describe('top level describe disabled', common.mustNotCall())
describe.skip('top level skipped describe disabled', common.mustNotCall())
describe.skip('top level skipped describe enabled', common.mustNotCall())
test('top level runs because name includes PaTtErN', common.mustCall())

test('top level test enabled', common.mustCall(async (t) => {
t.beforeEach(common.mustCall())
t.afterEach(common.mustCall())
await t.test(
'nested test runs because name includes PATTERN',
common.mustCall()
)
}))

describe('top level describe enabled', () => {
before(common.mustCall())
beforeEach(common.mustCall(2))
afterEach(common.mustCall(2))
after(common.mustCall())

it('nested it disabled', common.mustNotCall())
it('nested it enabled', common.mustCall())
describe('nested describe disabled', common.mustNotCall())
describe('nested describe enabled', common.mustCall(() => {
it('is enabled', common.mustCall())
}))
})
107 changes: 107 additions & 0 deletions test/message/test_runner_test_name_pattern.out
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
TAP version 13
# Subtest: top level test disabled
ok 1 - top level test disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped test disabled
ok 2 - top level skipped test disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped test enabled
ok 3 - top level skipped test enabled # SKIP
---
duration_ms: *
...
# Subtest: top level it enabled
ok 4 - top level it enabled
---
duration_ms: *
...
# Subtest: top level it disabled
ok 5 - top level it disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped it disabled
ok 6 - top level skipped it disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped it enabled
ok 7 - top level skipped it enabled # SKIP
---
duration_ms: *
...
# Subtest: top level describe disabled
ok 8 - top level describe disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped describe disabled
ok 9 - top level skipped describe disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: top level skipped describe enabled
ok 10 - top level skipped describe enabled # SKIP
---
duration_ms: *
...
# Subtest: top level runs because name includes PaTtErN
ok 11 - top level runs because name includes PaTtErN
---
duration_ms: *
...
# Subtest: top level test enabled
# Subtest: nested test runs because name includes PATTERN
ok 1 - nested test runs because name includes PATTERN
---
duration_ms: *
...
1..1
ok 12 - top level test enabled
---
duration_ms: *
...
# Subtest: top level describe enabled
# Subtest: nested it disabled
ok 1 - nested it disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: nested it enabled
ok 2 - nested it enabled
---
duration_ms: *
...
# Subtest: nested describe disabled
ok 3 - nested describe disabled # SKIP test name does not match pattern
---
duration_ms: *
...
# Subtest: nested describe enabled
# Subtest: is enabled
ok 1 - is enabled
---
duration_ms: *
...
1..1
ok 4 - nested describe enabled
---
duration_ms: *
...
1..4
ok 13 - top level describe enabled
---
duration_ms: *
...
1..13
# tests 13
# pass 4
# fail 0
# cancelled 0
# skipped 9
# todo 0
# duration_ms *
14 changes: 14 additions & 0 deletions test/message/test_runner_test_name_pattern_with_only.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// https://github.com/nodejs/node/blob/87170c3f9271da947a7b33d0696ec4cf8aab6eb6/test/message/test_runner_test_name_pattern_with_only.js
// Flags: --no-warnings --test-only --test-name-pattern=enabled
'use strict'
const common = require('../common')
const { test } = require('#node:test')

test('enabled and only', { only: true }, common.mustCall(async (t) => {
await t.test('enabled', common.mustCall())
await t.test('disabled', common.mustNotCall())
}))

test('enabled but not only', common.mustNotCall())
test('only does not match pattern', { only: true }, common.mustNotCall())
test('not only and does not match pattern', common.mustNotCall())
Loading