Vitest Typecheck is not working

[yamcodes]

Description

Hey team, I've noticed that Vitest's typecheck feature doesn't work even though we have *.test-d.ts files in the repo.

There are 2 apparent issues:

• We don't check types in pnpm test due to not configuring typecheck in vitest.config.ts. It's disabled by default.
• Enabling typecheck in vitest.config.ts causes typecheck tests to pass blindly, even where there are failing tests.

How to reproduce

1. Run pnpm test. You will notice that *.test-d.ts files are not reported.
2. In an attempt to enable typecheck, add the following entry to vitest.config:

import { defineConfig } from 'vitest/config'

const resolve = (specifier: string) =>
  new URL(import.meta.resolve(specifier)).pathname

export default defineConfig(env => ({
  test: {
    globals: true,
    include: ['tests/**/*.test.ts'],
    benchmark: {
      include: ['benchmarks/**/*.bench.ts'],
    },
    coverage: {
      thresholds: { 100: true },
      include: ['src/**'],
    },
    setupFiles: env.mode === 'benchmark' ? ['benchmarks/globals.ts'] : [],

+    typecheck: {
+      include: ['tests/**/*.test-d.ts'],
+      enabled: true,
+    },
  },
  resolve: {
    alias: {
      radashi: resolve('./src/mod.js'),
    },
  },
}))

3. Run pnpm test again. It looks like all *.test-d.ts tests are now reported, and pass!
4. Add a bogus test, purposefully failing type test to one of the *.test-d.ts files, say select.test-d.ts:

import * as _ from 'radashi'

describe('select types', () => {
+  test('bogus test to test vitest', () => {
+    expectTypeOf<number>().toEqualTypeOf<string>()
+  })
  test('select with condition', () => {
    const array = [
      { id: 1, name: 'John' },
      { id: 2, name: 'Jane' },
      { id: 3, name: null },
    ]
    const result = _.select(
      array,
      item => item.name,
      item => item.name !== null,
    )
    // No way for TypeScript to understand that the result is a string[]
    expectTypeOf(result).toEqualTypeOf<(string | null)[]>()
  })
  test('select without condition', () => {
    const array = [
      { id: 1, name: 'John' },
      { id: 2, name: 'Jane' },
      { id: 3, name: null },
    ]
    const result = _.select(array, item => item.name)
    expectTypeOf(result).toEqualTypeOf<string[]>()
  })
})

5. Run pnpm test again, all *.test-d.ts tests still pass.

Questions

1. Is this not the correct way to test types, is there something I missed?

Attempted fixes

1. Update Vitest to latest - that alone did not solve the issue for me.
2. After narrowing down against a working minimal example, I've identified that the issue is in tsconfig.json. The following change fixes the issue:

{
-  "include": ["src/**/*.ts"],
+  "include": ["src/**/*.ts", "tests/**/*.test-d.ts"],
  "compilerOptions": {
    "allowImportingTsExtensions": true,
    "moduleResolution": "node",
    "outDir": "./dist/tmp",
    "noEmitOnError": true,
    "declaration": true,
    "emitDeclarationOnly": true,
    "isolatedDeclarations": true,
    "target": "es2017",
    "lib": ["es2017"],
    "esModuleInterop": true,
    "skipLibCheck": true,
    "strict": true,
    "paths": {
      "radashi": ["./src/mod.ts"]
    },
-    "types": []
+    "types": ["vitest/globals"]
  }
}

However, this doesn't seem ideal, as the tsconfig.json in tests implies we should configure tests there and not the root tsconfig.json.

Note that solution (2) requires solution (1). It only works with an updated vitest.

Edit: I think I found a good, minimal solution in #245