Setting up detox with Vitest as the test runner

[mackwill]

Description

I've just recently set up Detox in our project, but we're looking strip jest from our code base completely and we're in the process of migrating to vitest instead.

The detox documentation states that there is support for third party test runners so i tried to set up detox with vitest

My vitest setup is fairly simple at the moment

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    include: ["./__test__/**/*.test.ts"],
    globalSetup: "./node_modules/detox/runners/jest/globalSetup",
  },
});

and i've updated the .detoxrc configuration to use vitest as well

module.exports = {
  testRunner: {
    args: {
      $0: "vitest",
      config: "vitest.config.js",
    },
  },
....

Building on iOS is fine, but when i try to run a test i get the following error

    DetoxRuntimeError: Detox worker instance has not been installed in this context (DetoxSecondaryContext).

    HINT: If you are using Detox with Jest according to the latest guide, please report this issue on our GitHub tracker:
    https://github.com/wix/Detox/issues
    Otherwise, make sure you call detox.installWorker() beforehand.

      3 | describe('Example', () => {
      4 |   beforeAll(async () => {
    > 5 |     await device.launchApp()
        |                  ^
      6 |   })
      7 |
      8 |   beforeEach(async () => {

I havent found any other documentation online or examples of using a third party test runner, so im coming here to see if there is anyone that has done this with vitest/knows if this is possible? Or maybe its recommended to not use vitest?

Any help or insight would be appreciated!

NOTE: If i just use the default setup with jest as the test runner, everything runs smoothly

Your environment

Detox version: 20.11.2
React Native version: 0.72.1
Node version: 16
Device model:
OS:
Test-runner (select one): jest / other

[noomorph]

Oh, this ain't a trivial task, I'd say. 😬

You'd need to learn thoroughly how Detox + Jest integration works, and create a similar one for Vitest.

I see that you did not create a custom test environment: https://vitest.dev/config/#environment

So you should extend Vitest's Node environment where you'd call detox.init()/detox.installWorker()/detox.uninstallWorker()/detox.cleanup().

P. S. If I were assigned to create such an implementation, I'd estimate at least a full working week for a quick'n'dirty POC. A good implementation would likely involve some rewrites in Detox's source code, and that would take a month or more. Make sure you have enough time dedicated if you're serious about writing a third-party integration.

[mackwill]

@noomorph yeah i have yet to create a custom environment - i had a feeling that was the route i would need to go but i felt like i needed to check and see if there were other instances somewhere else that i had missed before opening possibly a can of worms implementing this :P

Will detox be closely tied to jest for the forseable future? Are there plans to make integration with third party runners easier?

[noomorph]

Sorry, I missed your mention and by accident found it now.

Yes, it is rather correct that Detox will be focusing specifically on Jest in the foreseeable future, and this is not accidental — its source code is still somewhat messed up due to ad-hoc simultaneous integrations with two test runners back in the day. Healing it takes time.

It might sound like a paradox but the stronger we are integrating Detox with Jest, the cleaner, smaller and simpler its core becomes. At some point (hopefully, Detox 22) we'll totally split the automation interface (device and app interactions) from test runner layer, and the authors of test integrations will be given absolute freedom of how to integrate Detox with a test runner of their choice — i.e. there won't be any specific lifecycle to adhere to, just API with devices and their apps and elements. That way the authors won't have to look back at our docs and various quirks, although it has to be admitted that all these improvements won't reduce the amount of glue code — it should just become more straightforward and obvious to grasp.

I'd love to play with Vitest in spare time and see what can be done, but I know that the community's demand is rather to improve onboarding and troubleshooting experience (especially on Android) and maintain API parity between the platforms as it grows.

So, a bird in the hand is worth two in the bush, speaking of test runners. For example, I am not satisfied with artifacts subsystem and timeout errors currently in Jest, and it is preferable to make one runner work flawlessly than to produce weak integrations with many. The mentioned improvements will bring more insights to "what makes Detox+Test runner integration really good", and that will benefit the community later, I believe.

[noomorph]

Just in case if you have a spare couple of minutes to share why Vitest integration would be a better fit for your needs, I'd be happy to read. Maybe I can learn something for the future or the present.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.

[amehmeto]

@noomorph Vitest really has significant better performance. Which is critical for keeping the testing experience satisfying.

As a TDD dev guy, I really enjoy vitest for giving me fast feedback. I use it for unit test mainly. I am ok with setting Detox with jest for e2e tests, yet this dual configuration adds complexity.

[kyranjamie]

We're also moving away from Jest towards Vitest in the rest of our stack, so we'd love to see the ability to use it here as well

[noomorph]

@amehmeto, @kyranjamie, thanks for bringing this to our attention.

I have a couple of things to say:

1. Our goal to make Detox more core-like and test runner-agnostic today is as valid as it was two years ago. Due to scaling challenges within Wix, our team has been busy with non-public development lately, but reassembling Detox into small contributable building blocks is still in the upper half of the todo list.

2. Jest 30 has already released some alpha builds (the latest being 30.0.0-alpha.4, if I remember correctly) that have addressed some performance issues. I hope that these improvements will also benefit Detox users soon. As I understand it, Jest's node_modules footprint is expected to be more compact, as they now bundle it into one package.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.