Skip to content

Commit

Permalink
feat(reporters): summary option for verbose and default reporte…
Browse files Browse the repository at this point in the history 
…rs (#6893)
  • Loading branch information
@AriPerkkio
AriPerkkio authored Nov 18, 2024
1 parent 171041a commit 511b73c
Show file tree
Hide file tree
Showing 22 changed files with 1,124 additions and 523 deletions.
67 changes: 54 additions & 13 deletions docs/guide/reporters.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,33 +96,54 @@ This example will write separate JSON and XML reports as well as printing a verb

### Default Reporter

By default (i.e. if no reporter is specified), Vitest will display results for each test suite hierarchically as they run, and then collapse after a suite passes. When all tests have finished running, the final terminal output will display a summary of results and details of any failed tests.
By default (i.e. if no reporter is specified), Vitest will display summary of running tests and their status at the bottom. Once a suite passes, its status will be reported on top of the summary.

You can disable the summary by configuring the reporter:

:::code-group
```ts [vitest.config.ts]
export default defineConfig({
test: {
reporters: [
['default', { summary: false }]
]
},
})
```
:::

Example output for tests in progress:

```bash
✓ __tests__/file1.test.ts (2) 725ms
✓ __tests__/file2.test.ts (5) 746ms
✓ second test file (2) 746ms
✓ 1 + 1 should equal 2
✓ 2 - 1 should equal 1
✓ test/example-1.test.ts (5 tests | 1 skipped) 306ms
✓ test/example-2.test.ts (5 tests | 1 skipped) 307ms

❯ test/example-3.test.ts 3/5
❯ test/example-4.test.ts 1/5

Test Files 2 passed (4)
Tests 10 passed | 3 skipped (65)
Start at 11:01:36
Duration 2.00s
```

Final output after tests have finished:

```bash
✓ __tests__/file1.test.ts (2) 725ms
✓ __tests__/file2.test.ts (2) 746ms
✓ test/example-1.test.ts (5 tests | 1 skipped) 306ms
✓ test/example-2.test.ts (5 tests | 1 skipped) 307ms
✓ test/example-3.test.ts (5 tests | 1 skipped) 307ms
✓ test/example-4.test.ts (5 tests | 1 skipped) 307ms

Test Files 2 passed (2)
Tests 4 passed (4)
Test Files 4 passed (4)
Tests 16 passed | 4 skipped (20)
Start at 12:34:32
Duration 1.26s (transform 35ms, setup 1ms, collect 90ms, tests 1.47s, environment 0ms, prepare 267ms)
```

### Basic Reporter

The `basic` reporter displays the test files that have run and a summary of results after the entire suite has finished running. Individual tests are not included in the report unless they fail.
The `basic` reporter is equivalent to `default` reporter without `summary`.

:::code-group
```bash [CLI]
Expand Down Expand Up @@ -151,7 +172,7 @@ Example output using basic reporter:

### Verbose Reporter

Follows the same hierarchical structure as the `default` reporter, but does not collapse sub-trees for passed test suites. The final terminal output displays all tests that have run, including those that have passed.
Verbose reporter is same as `default` reporter, but it also displays each individual test after the suite has finished. It also displays currently running tests that are taking longer than [`slowTestThreshold`](/config/#slowtestthreshold). Similar to `default` reporter, you can disable the summary by configuring the reporter.

:::code-group
```bash [CLI]
Expand All @@ -161,12 +182,32 @@ npx vitest --reporter=verbose
```ts [vitest.config.ts]
export default defineConfig({
test: {
reporters: ['verbose']
reporters: [
['verbose', { summary: false }]
]
},
})
```
:::

Example output for tests in progress with default `slowTestThreshold: 300`:

```bash
✓ __tests__/example-1.test.ts (2) 725ms
✓ first test file (2) 725ms
✓ 2 + 2 should equal 4
✓ 4 - 2 should equal 2

❯ test/example-2.test.ts 3/5
↳ should run longer than three seconds 1.57s
❯ test/example-3.test.ts 1/5

Test Files 2 passed (4)
Tests 10 passed | 3 skipped (65)
Start at 11:01:36
Duration 2.00s
```

Example of final terminal output for a passing test suite:

```bash
Expand Down
3 changes: 2 additions & 1 deletion packages/vitest/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,11 @@
"expect-type": "^1.1.0",
"magic-string": "^0.30.12",
"pathe": "^1.1.2",
"restore-cursor": "^5.1.0",
"std-env": "^3.8.0",
"tinybench": "^2.9.0",
"tinyexec": "^0.3.1",
"tinypool": "^1.0.1",
"tinypool": "^1.0.2",
"tinyrainbow": "^1.2.0",
"vite": "^5.0.0",
"vite-node": "workspace:*",
Expand Down
107 changes: 33 additions & 74 deletions packages/vitest/src/node/reporters/base.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ import c from 'tinyrainbow'
import { isCI, isDeno, isNode } from '../../utils/env'
import { hasFailedSnapshot } from '../../utils/tasks'
import { F_CHECK, F_POINTER, F_RIGHT } from './renderers/figures'
import { countTestErrors, divider, formatProjectName, formatTimeString, getStateString, getStateSymbol, renderSnapshotSummary, taskFail, withLabel } from './renderers/utils'
import { countTestErrors, divider, formatProjectName, formatTime, formatTimeString, getStateString, getStateSymbol, padSummaryTitle, renderSnapshotSummary, taskFail, withLabel } from './renderers/utils'

const BADGE_PADDING = ' '
const LAST_RUN_LOG_TIMEOUT = 1_500

export interface BaseOptions {
isTTY?: boolean
Expand All @@ -27,14 +26,12 @@ export abstract class BaseReporter implements Reporter {
failedUnwatchedFiles: Task[] = []
isTTY: boolean
ctx: Vitest = undefined!
renderSucceed = false

protected verbose = false

private _filesInWatchMode = new Map<string, number>()
private _timeStart = formatTimeString(new Date())
private _lastRunTimeout = 0
private _lastRunTimer: NodeJS.Timeout | undefined
private _lastRunCount = 0

constructor(options: BaseOptions = {}) {
this.isTTY = options.isTTY ?? ((isNode || isDeno) && process.stdout?.isTTY && !isCI)
Expand Down Expand Up @@ -65,9 +62,6 @@ export abstract class BaseReporter implements Reporter {
}

onTaskUpdate(packs: TaskResultPack[]) {
if (this.isTTY) {
return
}
for (const pack of packs) {
const task = this.ctx.state.idMap.get(pack[0])

Expand Down Expand Up @@ -117,6 +111,8 @@ export abstract class BaseReporter implements Reporter {

this.log(` ${title} ${task.name} ${suffix}`)

const anyFailed = tests.some(test => test.result?.state === 'fail')

for (const test of tests) {
const duration = test.result?.duration

Expand All @@ -137,6 +133,15 @@ export abstract class BaseReporter implements Reporter {
+ ` ${c.yellow(Math.round(duration) + c.dim('ms'))}`,
)
}

// also print skipped tests that have notes
else if (test.result?.state === 'skip' && test.result.note) {
this.log(` ${getStateSymbol(test)} ${getTestName(test)}${c.dim(c.gray(` [${test.result.note}]`))}`)
}

else if (this.renderSucceed || anyFailed) {
this.log(` ${c.green(c.dim(F_CHECK))} ${getTestName(test, c.dim(' > '))}`)
}
}
}

Expand All @@ -153,8 +158,6 @@ export abstract class BaseReporter implements Reporter {
}

onWatcherStart(files = this.ctx.state.getFiles(), errors = this.ctx.state.getUnhandledErrors()) {
this.resetLastRunLog()

const failed = errors.length > 0 || hasFailed(files)

if (failed) {
Expand All @@ -177,38 +180,9 @@ export abstract class BaseReporter implements Reporter {
}

this.log(BADGE_PADDING + hints.join(c.dim(', ')))

if (this._lastRunCount) {
const LAST_RUN_TEXT = `rerun x${this._lastRunCount}`
const LAST_RUN_TEXTS = [
c.blue(LAST_RUN_TEXT),
c.gray(LAST_RUN_TEXT),
c.dim(c.gray(LAST_RUN_TEXT)),
]
this.ctx.logger.logUpdate(BADGE_PADDING + LAST_RUN_TEXTS[0])
this._lastRunTimeout = 0
this._lastRunTimer = setInterval(() => {
this._lastRunTimeout += 1
if (this._lastRunTimeout >= LAST_RUN_TEXTS.length) {
this.resetLastRunLog()
}
else {
this.ctx.logger.logUpdate(
BADGE_PADDING + LAST_RUN_TEXTS[this._lastRunTimeout],
)
}
}, LAST_RUN_LOG_TIMEOUT / LAST_RUN_TEXTS.length)
}
}

private resetLastRunLog() {
clearInterval(this._lastRunTimer)
this._lastRunTimer = undefined
this.ctx.logger.logUpdate.clear()
}

onWatcherRerun(files: string[], trigger?: string) {
this.resetLastRunLog()
this.watchFilters = files
this.failedUnwatchedFiles = this.ctx.state.getFiles().filter(file =>
!files.includes(file.filepath) && hasFailed(file),
Expand All @@ -222,11 +196,7 @@ export abstract class BaseReporter implements Reporter {

let banner = trigger ? c.dim(`${this.relative(trigger)} `) : ''

if (files.length > 1 || !files.length) {
// we need to figure out how to handle rerun all from stdin
this._lastRunCount = 0
}
else if (files.length === 1) {
if (files.length === 1) {
const rerun = this._filesInWatchMode.get(files[0]) ?? 1
banner += c.blue(`x${rerun} `)
}
Expand All @@ -248,10 +218,8 @@ export abstract class BaseReporter implements Reporter {

this.log('')

if (!this.isTTY) {
for (const task of this.failedUnwatchedFiles) {
this.printTask(task)
}
for (const task of this.failedUnwatchedFiles) {
this.printTask(task)
}

this._timeStart = formatTimeString(new Date())
Expand Down Expand Up @@ -351,6 +319,8 @@ export abstract class BaseReporter implements Reporter {
}

reportTestSummary(files: File[], errors: unknown[]) {
this.log()

const affectedFiles = [
...this.failedUnwatchedFiles,
...files,
Expand All @@ -364,21 +334,21 @@ export abstract class BaseReporter implements Reporter {

for (const [index, snapshot] of snapshotOutput.entries()) {
const title = index === 0 ? 'Snapshots' : ''
this.log(`${padTitle(title)} ${snapshot}`)
this.log(`${padSummaryTitle(title)} ${snapshot}`)
}

if (snapshotOutput.length > 1) {
this.log()
}

this.log(padTitle('Test Files'), getStateString(affectedFiles))
this.log(padTitle('Tests'), getStateString(tests))
this.log(padSummaryTitle('Test Files'), getStateString(affectedFiles))
this.log(padSummaryTitle('Tests'), getStateString(tests))

if (this.ctx.projects.some(c => c.config.typecheck.enabled)) {
const failed = tests.filter(t => t.meta?.typecheck && t.result?.errors?.length)

this.log(
padTitle('Type Errors'),
padSummaryTitle('Type Errors'),
failed.length
? c.bold(c.red(`${failed.length} failed`))
: c.dim('no errors'),
Expand All @@ -387,19 +357,19 @@ export abstract class BaseReporter implements Reporter {

if (errors.length) {
this.log(
padTitle('Errors'),
padSummaryTitle('Errors'),
c.bold(c.red(`${errors.length} error${errors.length > 1 ? 's' : ''}`)),
)
}

this.log(padTitle('Start at'), this._timeStart)
this.log(padSummaryTitle('Start at'), this._timeStart)

const collectTime = sum(files, file => file.collectDuration)
const testsTime = sum(files, file => file.result?.duration)
const setupTime = sum(files, file => file.setupDuration)

if (this.watchFilters) {
this.log(padTitle('Duration'), time(collectTime + testsTime + setupTime))
this.log(padSummaryTitle('Duration'), formatTime(collectTime + testsTime + setupTime))
}
else {
const executionTime = this.end - this.start
Expand All @@ -409,16 +379,16 @@ export abstract class BaseReporter implements Reporter {
const typecheck = sum(this.ctx.projects, project => project.typechecker?.getResult().time)

const timers = [
`transform ${time(transformTime)}`,
`setup ${time(setupTime)}`,
`collect ${time(collectTime)}`,
`tests ${time(testsTime)}`,
`environment ${time(environmentTime)}`,
`prepare ${time(prepareTime)}`,
typecheck && `typecheck ${time(typecheck)}`,
`transform ${formatTime(transformTime)}`,
`setup ${formatTime(setupTime)}`,
`collect ${formatTime(collectTime)}`,
`tests ${formatTime(testsTime)}`,
`environment ${formatTime(environmentTime)}`,
`prepare ${formatTime(prepareTime)}`,
typecheck && `typecheck ${formatTime(typecheck)}`,
].filter(Boolean).join(', ')

this.log(padTitle('Duration'), time(executionTime) + c.dim(` (${timers})`))
this.log(padSummaryTitle('Duration'), formatTime(executionTime) + c.dim(` (${timers})`))
}

this.log()
Expand Down Expand Up @@ -544,17 +514,6 @@ function errorBanner(message: string) {
return c.red(divider(c.bold(c.inverse(` ${message} `))))
}

function padTitle(str: string) {
return c.dim(`${str.padStart(11)} `)
}

function time(time: number) {
if (time > 1000) {
return `${(time / 1000).toFixed(2)}s`
}
return `${Math.round(time)}ms`
}

function sum<T>(items: T[], cb: (_next: T) => number | undefined) {
return items.reduce((total, next) => {
return total + Math.max(cb(next) || 0, 0)
Expand Down
12 changes: 12 additions & 0 deletions packages/vitest/src/node/reporters/basic.ts
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
import type { File } from '@vitest/runner'
import type { Vitest } from '../core'
import c from 'tinyrainbow'
import { BaseReporter } from './base'

export class BasicReporter extends BaseReporter {
Expand All @@ -7,6 +9,16 @@ export class BasicReporter extends BaseReporter {
this.isTTY = false
}

onInit(ctx: Vitest) {
super.onInit(ctx)

ctx.logger.log(c.inverse(c.bold(c.yellow(' DEPRECATED '))), c.yellow(
`'basic' reporter is deprecated and will be removed in Vitest v3.\n`
+ `Remove 'basic' from 'reporters' option. To match 'basic' reporter 100%, use configuration:\n${
JSON.stringify({ test: { reporters: [['default', { summary: false }]] } }, null, 2)}`,
))
}

reportSummary(files: File[], errors: unknown[]) {
// non-tty mode doesn't add a new line
this.ctx.logger.log()
Expand Down
Loading

0 comments on commit 511b73c

Please sign in to comment.