Supporting `vitest` tracking issue #23882

marvinhagemeister · 2024-05-18T18:21:31Z

This is an issue to track the progress on getting the vitest test runner to work. I'm currently using the test suite from urql to check our progress.

~~Web MessagePort not converted to worker_thread MessagePort Bug: port.addListener is not a function #23828~~
Node globals missing in vm.runInThisConext Bug: Node vm doesn't have Node's global global #23852
TTY test result reporter not working because of this check https://github.com/vitest-dev/vitest/blob/main/packages/vitest/src/utils/env.ts#L1
- ~~signal-exit is used under the hood. It's likely that this issue causes CTRL+C to not work: Listeners for SIGINT (and other) signals are not registered in EventEmitter #22892~~
Test run doesn't seem to complete

The text was updated successfully, but these errors were encountered:

birkskyum · 2024-06-30T22:29:01Z

With the standard example it appear to still (deno 1.44.4+bc8a0e6) hang at:

➜ deno task test              
Task test vitest

 DEV  v1.6.0

Both w/o DENO_FUTURE=1

Fixes #24756. Fixes #24796. This also gets vitest working when using [`--pool=forks`](https://vitest.dev/guide/improving-performance#pool) (which is the default as of vitest 2.0). Ref #23882. --- This PR resolves a handful of issues with child_process IPC. In particular: - We didn't support sending typed array views over IPC - Opening an IPC channel resulted in the event loop never exiting - Sending a `null` over IPC would terminate the channel - There was some UB in the read implementation (transmuting an `&[u8]` to `&mut [u8]`) - The `send` method wasn't returning anything, so there was no way to signal backpressure (this also resulted in the benchmark `child_process_ipc.mjs` being misleading, as it tried to respect backpressure. That gave node much worse results at larger message sizes, and gave us much worse results at smaller message sizes). - We weren't setting up the `channel` property on the `process` global (or on the `ChildProcess` object), and also didn't have a way to ref/unref the channel - Calling `kill` multiple times (or disconnecting the channel, then calling kill) would throw an error - Node couldn't spawn a deno subprocess and communicate with it over IPC

Fixes #24756. Fixes #24796. This also gets vitest working when using [`--pool=forks`](https://vitest.dev/guide/improving-performance#pool) (which is the default as of vitest 2.0). Ref #23882. --- This PR resolves a handful of issues with child_process IPC. In particular: - We didn't support sending typed array views over IPC - Opening an IPC channel resulted in the event loop never exiting - Sending a `null` over IPC would terminate the channel - There was some UB in the read implementation (transmuting an `&[u8]` to `&mut [u8]`) - The `send` method wasn't returning anything, so there was no way to signal backpressure (this also resulted in the benchmark `child_process_ipc.mjs` being misleading, as it tried to respect backpressure. That gave node much worse results at larger message sizes, and gave us much worse results at smaller message sizes). - We weren't setting up the `channel` property on the `process` global (or on the `ChildProcess` object), and also didn't have a way to ref/unref the channel - Calling `kill` multiple times (or disconnecting the channel, then calling kill) would throw an error - Node couldn't spawn a deno subprocess and communicate with it over IPC (cherry picked from commit cd59fc5)

birkskyum · 2024-07-31T22:13:29Z

After this PR ( #24763 ) I tried this again with deno 1.45.5, and these scenarios does indeed appear to work now:

using "--pool=forks" with Vitest 1.x w/ DENO_FUTURE=1
using Vitest 2.x w/ DENO_FUTURE=1

Scenarios that doesn't work out-of-the-box, but which will resolve over time / has workarounds:

using Deno 1.x without the DENO_FUTURE=1 flag
using vitest 1.x without the --pool=forks flag

birkskyum · 2024-09-10T13:27:05Z

Alright, so I'm on Vitest 2 (it defaults to pool: forks) and Deno 2 (deno 2.0.0-rc.1+7bfcb4d) now.

Repro

deno.json

{
  "nodeModulesDir": "auto",
  "tasks": {
    "test": "vitest"
  },
  "imports": { "jsdom": "npm:jsdom@^25.0.0", "vitest": "npm:vitest@^2.0.5" }
}

sum.test.ts

import { expect, test } from "vitest";

export function sum(a, b) {
  return a + b
}

test('adds 1 + 2 to equal 3', () => {
  expect(sum(1, 2)).toBe(3)
})

vitest.config.ts

import {defineConfig} from "vitest/config";

export default defineConfig({
  test: {
    environment: 'jsdom'
  }
})

Run

deno install
deno task test

Error:

 DEV  v2.0.5 /Users/admin/repos/deno-kitchensink/vitest-demo


⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
TypeError: Cannot redefine property: location
 ❯ populateGlobal node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/chunks/index.lVXYBqEP.js:374:12
 ❯ Object.setup node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/chunks/index.lVXYBqEP.js:526:33
 ❯ withEnv node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/chunks/runBaseTests.CyvqmuC9.js:92:15
 ❯ run node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/chunks/runBaseTests.CyvqmuC9.js:116:3
 ❯ runBaseTests node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/chunks/base.CC5R_kgU.js:31:3
 ❯ ForksBaseWorker.executeTests node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/workers/forks.js:25:7
 ❯ execute node_modules/.deno/vitest@2.0.5/node_modules/vitest/dist/worker.js:115:5
 ❯ onMessage node_modules/.deno/tinypool@1.0.1/node_modules/tinypool/dist/entry/process.js:55:20

⎯⎯⎯⎯⎯⎯⎯⎯

If I remove environment: "jsdom" it works:

 DEV  v2.0.5 /Users/admin/repos/deno-kitchensink/vitest-demo

 ✓ sum.test.ts (1)
   ✓ adds 1 + 2 to equal 3

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  23:47:08
   Duration  170ms (transform 15ms, setup 0ms, collect 15ms, tests 1ms, environment 0ms, prepare 46ms)

birkskyum · 2024-09-22T22:39:25Z

Commenting out the "location" key from jsdom-keys here will actually make the test pass, so it's pretty close - seems to just be the location that's left. The location error is that vitest can't re-declare or delete it on the global object. Does Deno set a location globally that i.e. Node doesn't do?

➜ deno task test
Task test vitest

 DEV  v2.1.1 /Users/admin/repos/deno-kitchensink/vitest-test

 ✓ sum.test.ts (1)
   ✓ adds 1 + 2 to equal 3

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  00:37:05
   Duration  473ms (transform 15ms, setup 0ms, collect 15ms, tests 1ms, environment 317ms, prepare 41ms)


 PASS  Waiting for file changes...
       press h to show help, press q to quit

bartlomieju · 2024-09-22T22:44:22Z

Yes, globalThis.location is undefined and writeable in Deno (unless you run with --location flag). So vitest should be able to redeclare it.

birkskyum · 2024-09-22T22:52:50Z

It uses a Object.defineProperty(global, 'location', value) syntax and a delete global2['location']; and they both break. When it says it's a TypeError, is it then the compilation that breaks, and not the code running?

bartlomieju · 2024-09-22T23:02:10Z

Executing script like this:

console.log(globalThis.location);
Object.defineProperty(globalThis, "location", { value: "https://deno.com" });
console.log(globalThis.location);
delete globalThis["location"];
console.log(globalThis.location);

I get this on Deno v2.0.0-rc.4

undefined
https://deno.com
error: Uncaught (in promise) TypeError: Cannot delete property 'location' of #<Window>
delete globalThis["location"];
^
    at file:///Users/ib/dev/deno/try.js:4:1

So indeed the deletion code might be the problem here.

@birkskyum could you provide a reproduction how you got this error?

birkskyum · 2024-09-22T23:04:56Z

@bartlomieju , it's by following these steps.

bartlomieju · 2024-09-22T23:13:13Z

Sorry, missed that. I can indeed reproduce the problem here. PR incoming, should be a simple fix provided that WPT doesn't complain.

bartlomieju · 2024-09-22T23:19:42Z

See #25812

This commit changes `globalThis.location` property to be configurable so that packages wanting to override it (or delete it) work properly. Towards #23882 This change makes reproduction from #23882 (comment) pass properly.

marvinhagemeister · 2024-09-24T08:21:53Z

Using the reproduction posted earlier here #23882 (comment) the test run passes in watch mode without issues. But when running the tests with vitest's run command like deno task test run, the program exits with:

error: Uncaught (in promise) BrokenPipe: Broken pipe (os error 32)

birkskyum · 2024-10-18T22:11:48Z

any idea why the test run is breaking?

bartlomieju · 2024-10-18T22:13:54Z

@birkskyum do you get any error there?

birkskyum · 2024-10-18T22:17:15Z

@bartlomieju i get the one here #23882 (comment)

birkskyum · 2024-10-18T22:18:28Z

it runs the test fine, but gives this error after:

bartlomieju · 2024-10-18T22:19:44Z

I believe this might be related to IPC pipes handling that @nathanwhit worked on a few weeks back.

Fixes the issue described in #23882 (comment). The parent was starting to send a message right before the process would exit, and the channel closed in the middle of the write. Unlike with reads, we weren't cancelling the pending writes, which resulted in a `Broken pipe` error surfacing to the user.

nathanwhit · 2024-10-24T17:02:50Z

any idea why the test run is breaking?

@birkskyum this should be fixed on canary now

birkskyum · 2024-10-24T17:40:19Z

@nathanwhit , awesome! tried it successfully just now, thanks!

birkskyum · 2024-10-24T17:43:51Z

With basic deno task test run working, I tried was using it for Coverage Reporting deno task test run --coverage and it appear to break with:

⇣9% ➜ deno run test run --coverage
Task test vitest "run" "--coverage"
Loaded  vitest@2.1.1  and  @vitest/coverage-v8@2.1.3 .
Running mixed versions is not supported and may lead into bugs
Update your dependencies and make sure the versions match.

 RUN  v2.1.1 
      Coverage enabled with v8

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

Error: Not implemented: inspector.Session.prototype.connect
 ❯ notImplemented ext:deno_node/_utils.ts:9:9
 ❯ Session.connect node:inspector:12:5
 ❯ Object.startCoverage node_modules/.deno

There's a ticket for that specifically in here too:

marvinhagemeister · 2024-10-24T18:35:10Z

This error occurs because our current node:inspector code is mostly stubs. The proper implementation is being worked on here #26471

Fixes the issue described in #23882 (comment). The parent was starting to send a message right before the process would exit, and the channel closed in the middle of the write. Unlike with reads, we weren't cancelling the pending writes, which resulted in a `Broken pipe` error surfacing to the user.

birkskyum · 2024-10-26T16:39:08Z

Hit a new issue today with Deno + Vitest + SolidJS

node worker_threads --conditions bug (vitest breaks for solidjs) #26567

birkskyum · 2024-11-10T22:11:28Z

for deno 2.0.6

vitest with coverage - TypeError: unknown system info kind "inspector" #26809

lishaduck · 2024-11-22T16:20:01Z

Coverage no longer crashes! It currently reports it all as zero, but that's better than it was!

See #27003.

lishaduck · 2024-11-24T22:56:52Z

Bumped into another issue:
If you use a package.json + .npmrc to resolve JSR deps, everything works fine (they're put in node_modules), but when using a deno.json and native jsr: specifiers, they don't get put in the node_modules folder, so Vitest fails, even when using npm:@deno/vite-plugin. I thought it might be because I'm using workspaces (because of denoland/deno-vite-plugin#19), but hoisting the dependency in question (jsr:@std/path) to the root didn't help, so I assume it's an issue with the Vitest runner. If not, what repo should I file this in?

birkskyum · 2024-11-24T23:08:13Z

Bumped into another issue: If you use a package.json + .npmrc to resolve JSR deps, everything works fine (they're put in node_modules), but when using a deno.json and native jsr: specifiers, they don't get put in the node_modules folder, so Vitest fails, even when using npm:@deno/vite-plugin. I thought it might be because I'm using workspaces (because of denoland/deno-vite-plugin#19), but hoisting the dependency in question (jsr:@std/path) to the root didn't help, so I assume it's an issue with the Vitest runner. If not, what repo should I file this in?

@lishaduck Does it help to add in the deno.json { "nodeModulesDir": "auto" }, to generate the node_modules (docs)?

lishaduck · 2024-11-24T23:22:29Z

@lishaduck Does it help to add in the deno.json { "nodeModulesDir": "auto" }, to generate the node_modules (docs)?

I'm using "nodeModulesDir": "manual", but I installed via deno install first.
Here's some ci logs.

marvinhagemeister added the node compat label May 18, 2024

lishaduck mentioned this issue Jul 9, 2024

Support Deno. How to get about implementing platform-deno? Effect-TS/effect#1806

Open

nathanwhit mentioned this issue Jul 27, 2024

fix(node): Rework node:child_process IPC #24763

Merged

yasaichi mentioned this issue Aug 4, 2024

Bug: Vitest fails with Failed to cast callsite method 'getFileName' return value #24869

Closed

yasaichi mentioned this issue Aug 13, 2024

Bug: vitest run --coverage throws Not implemented: inspector.Session.prototype.connect #25004

Closed

birkskyum mentioned this issue Sep 11, 2024

Move from package.json to deno.json -> All Vite projects complain about deprecated CJS mode #25574

Closed

bartlomieju mentioned this issue Sep 22, 2024

feat: make 'globalThis.location' a configurable property #25812

Merged

nathanwhit mentioned this issue Oct 24, 2024

fix(ext/node): cancel pending ipc writes on channel close #26504

Merged

birkskyum mentioned this issue Nov 10, 2024

vitest with coverage - TypeError: unknown system info kind "inspector" #26809

Closed

This was referenced Nov 13, 2024

Deno Support vitest-dev/vscode#528

Open

Deno support Effect-TS/effect#754

Closed

lishaduck mentioned this issue Nov 22, 2024

Vitest Coverage v8 & Istanbul Providers Broken #27003

Open

syhol mentioned this issue Nov 29, 2024

[Proposal] Become a Deno first library fabian-hiller/valibot#183

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Supporting `vitest` tracking issue #23882

Supporting `vitest` tracking issue #23882

marvinhagemeister commented May 18, 2024 •

edited

Loading

birkskyum commented Jun 30, 2024 •

edited

Loading

birkskyum commented Jul 31, 2024 •

edited

Loading

birkskyum commented Sep 10, 2024 •

edited

Loading

birkskyum commented Sep 22, 2024 •

edited

Loading

bartlomieju commented Sep 22, 2024

birkskyum commented Sep 22, 2024 •

edited

Loading

bartlomieju commented Sep 22, 2024 •

edited

Loading

birkskyum commented Sep 22, 2024

bartlomieju commented Sep 22, 2024

bartlomieju commented Sep 22, 2024

marvinhagemeister commented Sep 24, 2024

birkskyum commented Oct 18, 2024 •

edited

Loading

bartlomieju commented Oct 18, 2024

birkskyum commented Oct 18, 2024 •

edited

Loading

birkskyum commented Oct 18, 2024

bartlomieju commented Oct 18, 2024

nathanwhit commented Oct 24, 2024

birkskyum commented Oct 24, 2024

birkskyum commented Oct 24, 2024 •

edited

Loading

marvinhagemeister commented Oct 24, 2024 •

edited

Loading

birkskyum commented Oct 26, 2024 •

edited

Loading

birkskyum commented Nov 10, 2024 •

edited

Loading

lishaduck commented Nov 22, 2024

lishaduck commented Nov 24, 2024 •

edited

Loading

birkskyum commented Nov 24, 2024 •

edited

Loading

lishaduck commented Nov 24, 2024

Supporting vitest tracking issue #23882

Supporting vitest tracking issue #23882

Comments

marvinhagemeister commented May 18, 2024 • edited Loading

birkskyum commented Jun 30, 2024 • edited Loading

birkskyum commented Jul 31, 2024 • edited Loading

birkskyum commented Sep 10, 2024 • edited Loading

Repro

birkskyum commented Sep 22, 2024 • edited Loading

bartlomieju commented Sep 22, 2024

birkskyum commented Sep 22, 2024 • edited Loading

bartlomieju commented Sep 22, 2024 • edited Loading

birkskyum commented Sep 22, 2024

bartlomieju commented Sep 22, 2024

bartlomieju commented Sep 22, 2024

marvinhagemeister commented Sep 24, 2024

birkskyum commented Oct 18, 2024 • edited Loading

bartlomieju commented Oct 18, 2024

birkskyum commented Oct 18, 2024 • edited Loading

birkskyum commented Oct 18, 2024

bartlomieju commented Oct 18, 2024

nathanwhit commented Oct 24, 2024

birkskyum commented Oct 24, 2024

birkskyum commented Oct 24, 2024 • edited Loading

marvinhagemeister commented Oct 24, 2024 • edited Loading

birkskyum commented Oct 26, 2024 • edited Loading

birkskyum commented Nov 10, 2024 • edited Loading

lishaduck commented Nov 22, 2024

lishaduck commented Nov 24, 2024 • edited Loading

birkskyum commented Nov 24, 2024 • edited Loading

lishaduck commented Nov 24, 2024

Supporting `vitest` tracking issue #23882

Supporting `vitest` tracking issue #23882

marvinhagemeister commented May 18, 2024 •

edited

Loading

birkskyum commented Jun 30, 2024 •

edited

Loading

birkskyum commented Jul 31, 2024 •

edited

Loading

birkskyum commented Sep 10, 2024 •

edited

Loading

birkskyum commented Sep 22, 2024 •

edited

Loading

birkskyum commented Sep 22, 2024 •

edited

Loading

bartlomieju commented Sep 22, 2024 •

edited

Loading

birkskyum commented Oct 18, 2024 •

edited

Loading

birkskyum commented Oct 18, 2024 •

edited

Loading

birkskyum commented Oct 24, 2024 •

edited

Loading

marvinhagemeister commented Oct 24, 2024 •

edited

Loading

birkskyum commented Oct 26, 2024 •

edited

Loading

birkskyum commented Nov 10, 2024 •

edited

Loading

lishaduck commented Nov 24, 2024 •

edited

Loading

birkskyum commented Nov 24, 2024 •

edited

Loading