vitest typecheck does not work when TypeScript references are used with noEmit

[scottwillmoore]

Describe the bug

I use TypeScript references in my Vite projects as recommended in their official templates. For example, the React with TypeScript template creates both a tsconfig.json and a tsconfig.node.json, which are combined with the use of TypeScript references. This allows the web files and the configuration files to receive different compilerOptions. In my projects I tend to create a minimal tsconfig.json which references tsconfig.node.json and tsconfig.web.json.

The problem occurs when references are used with noEmit such as when vitest executes tsc --noEmit ... to test the types of the project. This doesn't check any files declared in the references. In addition, you appear to get this error: TS6305: Output file '.../index.d.ts' has not been built from source file '.../index.ts'.

This is a documented, and known issue with the TypeScript compiler. It also doesn't look like it is going to be resolved...

To fix this, at the moment there are two solutions:

1. Don't use TypeScript references for important types that need to be checked. For example, instead of using a minimal tsconfig.json with tsconfig.node.json and tsconfig.web.json, I decided to merge my tsconfig.web.json back into my tsconfig.json. Although not ideal, I think this should cover most instances of this problem.

2. Use tsc --build without noEmit. You should setup an outDir so it doesn't pollute your source files. In addition, it is wasteful to build the project when you already use Vite, or another bundler... This would require an update to Vitest in order to be able to use vitest typecheck again.

Reproduction

I don't have time to create a perfect reproduction of the issue, but will work on it when I have some more free time. I thought I would create this issue to help others discover and understand this problem.

For now, you could use this project at 70387a6, and run npm run test:types in the packages/app directory.

I have now created a better reproduction: https://github.com/scottwillmoore/vitest-with-typescript-references!

System Info

System:
    OS: Linux 5.15 NixOS 23.05 (Stoat) 23.05 (Stoat)
    CPU: (6) x64 Intel(R) Core(TM) i5-8600K CPU @ 3.60GHz
    Memory: 4.63 GB / 7.73 GB
    Container: Yes
    Shell: 5.2.15 - /run/current-system/sw/bin/bash
  Binaries:
    Node: 18.16.1 - /nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/bin/node
    npm: 9.5.1 - /nix/store/l43bdmxxlkrzwic3q24ik366qyqg3s1g-nodejs-18.16.1/bin/npm

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[scottwillmoore]

I have now created a better reproduction: https://github.com/scottwillmoore/vitest-with-typescript-references!

[mrazauskas]

@scottwillmoore Thanks for this issue. Recently I released TSTyche, a type testing tool for TypeScript, and was looking around for challenging cases. TSTyche handles references properly. If you are looking for a solution, it is here –

Repo: https://github.com/tstyche/tstyche
Documentation: https://tstyche.org

---

TSTyche has more to offer.

Testing on specific version of TypeScript

Simply like this: tstyche --target 4.8,latest. It works with expect-type assertions or the ones imported from Vitest as well.

Readable assertion errors

TSTyche's own assertions makes life easier. For example, if a matcher like .toBeString() fails, you will see this:

Performance

In an empty repo with just one type test, TSTyche is slightly faster than Vitest. In a large monorepo TSTyche is 2x faster. (EDIT: TSTyche is 3-4x faster in another medium size repo, which is not a monorepo, but a single package and it has no references or so.) Keep in mind that TSTyche takes into account references fields of a tsconfig.json. I got 2x faster when references were removed. I think this is fair, because current Vitest ignores references. Otherwise, TSTyche was faster by some 20-30% depending on the amount of references.

How it works

TSTyche uses TypeScript's project service API, which allows retrieving type information per file. Instead of building the whole program, only minimal information needed to analyse particular file is collected. That is why it is faster, and can provide custom error messages, and can implement .toRaiseError() matcher, and more.

---

@sheremet-va We talked about type testing and // @ts-expect-error comments (here). To come back to that conversation, here is a challenge (actual mistake I have made):

interface Matchers<R, T = unknown> {
  [key: string]: (expected: T) => R;
}
 
// @ts-expect-error: requires a type argument
type M = Matcher<>;

Yep, those are two syntax errors. Compiling and executing the file does not catch them. Also the Matchers interface can be removed and this still will be a "passing" test. Not with TSTyche:

import { expect } from "tstyche";
 
interface Matchers<R, T = unknown> {
  [key: string]: (expected: T) => R;
}
 
expect<Matchers>().type.toRaiseError("requires between 1 and 2 type arguments");