Rush monorepo with vitest and pnpm

[vjau]

Issue description or question

I'm trying to get Wallaby to work with vitest with a Rush monorepo using pnpm.

I'm trying to create a Webstorm configuration for one of my subprojects using vitest.

When running the configuration, i get an error about missing @vitest/runner

Per instructions in the docs, i add

public-hoist-pattern[]=*vitest*
public-hoist-pattern[]=vite-node

to my root .npmrc in common/config/rush/.npmrc

I then force rush to reinstall everything.
I can verify that i have a @vitest/runner installed in .pnpm folder located in common/temp/node_modules/.pnpm with a symbolic link in common/temp/node_modules/@vitest/runner

Sadly, this doesn't fix the problem.

I then wonder if setting the wallaby root to as subfolder prevent it from seeing the root packages.

But setting walaby root to the monorepo root produces this error

Automatic Jest configuration error: Module jest-cli is not found in '.'.

We've also tried to automatically configure Wallaby.js for other frameworks
Automatic Vitest configuration error: Vitest dependency not found.

Wallaby diagnostics report

{
  editorVersion: 'WebStorm 2023.2.1',
  pluginVersion: '1.0.274',
  editorType: 'IntelliJ',
  osVersion: 'linux 6.2.0-31-generic',
  nodeVersion: 'v18.17.1',
  coreVersion: '1.0.1467',
  checksum: 'YWRlYjRhODkyMzJmM2Y5NDBjOWMxYTBkZmI1NjVlY2EsMTcxNDAwMzIwMDAwMCww',
  config: {
    diagnostics: {
      vitest: {
        file: { config: 'import { defineConfig } from "vitest/config";\n\nexport default defineConfig({\n  test: {\n    include: ["./src/**/tests/*.js", "./src/**/tests/*.ts"],\n  },\n});\n' },
        config: {
          allowOnly: true,
          watch: true,
          globals: false,
          environment: 'node',
          threads: true,
          clearMocks: false,
          restoreMocks: false,
          mockReset: false,
          include: [ './src/**/tests/*.js', './src/**/tests/*.ts' ],
          exclude: [
            '**/node_modules/**',
            '**/dist/**',
            '**/cypress/**',
            '**/.{idea,git,cache,output,temp}/**',
            '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*'
          ],
          testTimeout: 5000,
          hookTimeout: 10000,
          teardownTimeout: 10000,
          isolate: true,
          watchExclude: [ '**/node_modules/**', '**/dist/**' ],
          forceRerunTriggers: [ '**/package.json/**', '**/{vitest,vite}.config.*/**' ],
          update: false,
          reporters: [ 'default' ],
          silent: false,
          hideSkippedTests: false,
          ui: false,
          uiBase: '/__vitest__/',
          open: true,
          css: { include: [], modules: { classNameStrategy: 'stable' } },
          coverage: {
            provider: 'v8',
            enabled: false,
            clean: true,
            cleanOnRerun: true,
            reportsDirectory: './coverage',
            exclude: [
              'coverage/**',
              'dist/**',
              'packages/*/test?(s)/**',
              '**/*.d.ts',
              '**/virtual:*',
              '**/__x00__*',
              '**/\x00*',
              'cypress/**',
              'test?(s)/**',
              'test?(-*).?(c|m)[jt]s?(x)',
              '**/*{.,-}{test,spec}.?(c|m)[jt]s?(x)',
              '**/__tests__/**',
              '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*',
              '**/vitest.{workspace,projects}.[jt]s?(on)',
              '**/.{eslint,mocha,prettier}rc.{?(c|m)js,yml}'
            ],
            reportOnFailure: false,
            reporter: [ [ 'text', {} ], [ 'html', {} ], [ 'clover', {} ], [ 'json', {} ] ],
            extension: [
              '.js',  '.cjs',
              '.mjs', '.ts',
              '.mts', '.cts',
              '.tsx', '.jsx',
              '.vue', '.svelte'
            ],
            allowExternal: false
          },
          fakeTimers: { loopLimit: 10000, shouldClearNativeTimers: true, toFake: [ 'setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'setImmediate', 'clearImmediate', 'Date' ] },
          maxConcurrency: 5,
          dangerouslyIgnoreUnhandledErrors: false,
          typecheck: {
            checker: 'tsc',
            include: [ '**/*.{test,spec}-d.?(c|m)[jt]s?(x)' ],
            exclude: [
              '**/node_modules/**',
              '**/dist/**',
              '**/cypress/**',
              '**/.{idea,git,cache,output,temp}/**',
              '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*'
            ]
          },
          slowTestThreshold: 300,
          config: '<homeDir>/dev/cadendar-monorepo/apps/compta-trpc-server/vitest.config.ts',
          defines: {},
          root: '<homeDir>/dev/cadendar-monorepo/apps/compta-trpc-server',
          mode: [],
          inspect: false,
          inspectBrk: false,
          singleThread: false,
          deps: {
            moduleDirectories: [ '/node_modules/' ],
            optimizer: { ssr: { enabled: true }, web: { enabled: true } },
            web: { transformAssets: true, transformCss: true, transformGlobPattern: [] }
          },
          server: { deps: { inline: [ {}, {}, {}, '@nuxt/test-utils' ], moduleDirectories: [ '/node_modules/' ], cacheDir: 'node_modules/.vitest' } },
          snapshotOptions: { snapshotFormat: {}, updateSnapshot: 'new', snapshotEnvironment: null },
          experimentalVmWorkerMemoryLimit: 1370493952,
          setupFiles: [],
          cache: { dir: '<homeDir>/dev/cadendar-monorepo/apps/compta-trpc-server/node_modules/.vitest' },
          sequence: { hooks: 'parallel' },
          environmentMatchGlobs: [],
          browser: { enabled: false, headless: false, slowHijackESM: true, api: { port: 63315 } },
          testTransformMode: {},
          package: {
            version: '0.34.3',
            urls: { hooks: 'file://<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/runners/node/hooks.mjs' },
            paths: {
              root: '<homeDir>/dev/cadendar-monorepo/common/temp/node_modules/.pnpm/vitest@0.34.3/node_modules/vitest',
              dist: '<homeDir>/dev/cadendar-monorepo/common/temp/node_modules/.pnpm/vitest@0.34.3/node_modules/vitest/dist'
            }
          }
        }
      }
    },
    testFramework: { version: 'vitest@0.14.0', configurator: 'vitest@0.14.0', reporter: 'vitest@0.14.0', starter: 'vitest@0.14.0', autoDetected: true },
    preserveComments: false,
    extractComments: true,
    files: [
      { pattern: 'src/**/tests/*.js', ignore: true, trigger: true, load: true, file: true },
      { pattern: 'src/**/tests/*.ts', ignore: true, trigger: true, load: true, file: true },
      { pattern: '**/node_modules/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/dist/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/cypress/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/.{idea,git,cache,output,temp}/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/*.*', ignore: false, trigger: true, load: true, order: 1 }
    ],
    tests: [
      { pattern: '**/node_modules/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/dist/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/cypress/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/.{idea,git,cache,output,temp}/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build}.config.*', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: 'src/**/tests/*.js', ignore: false, trigger: true, load: true, test: true, order: 2 },
      { pattern: 'src/**/tests/*.ts', ignore: false, trigger: true, load: true, test: true, order: 3 }
    ],
    workers: { initial: 1, regular: 1, recycle: false },
    captureConsoleLog: true,
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    compilers: {},
    logLimits: { inline: { depth: 5, elements: 5000 }, values: { default: { stringLength: 200 }, autoExpand: { elements: 5000, stringLength: 8192, depth: 10 } } },
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore|c8 ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    env: {
      type: 'node',
      params: { runner: '--experimental-loader=file://<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/runners/node/hooks.mjs' },
      runner: '/usr/bin/node',
      viewportSize: { width: 800, height: 600 },
      options: { width: 800, height: 600 },
      bundle: true
    },
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    runAllTestsWhenNoAffectedTests: true,
    symlinkNodeModules: undefined,
    configCode: 'auto.detect#-1857531474'
  },
  packageJSON: {
    dependencies: {
      '@fastify/cors': '~8.3.0',
      fastify: '~4.17.0',
      '@fastify/sensible': '~5.2.0',
      '@trpc/server': '^10.27.1',
      superjson: '~1.12.4',
      '@fastify/secure-session': '~6.1.0',
      mongodb: '^5.1.0',
      dotenv: '^16.0.3',
      zod: '~3.21.4',
      bcrypt: '~5.1.0',
      luxon: '^3.3.0',
      lodash: '~4.17.21',
      pdfjs: '~2.4.7',
      'vincent-utils': 'workspace:*',
      iconv: '~3.0.1',
      underts: 'workspace:*',
      uuid: '~9.0.0',
      pdf2json: '~3.0.4',
      xlsx: '~0.18.5'
    },
    devDependencies: {
      '@types/bcrypt': '^5.0.0',
      '@types/node': '^18.15.11',
      '@types/luxon': '^3.3.0',
      '@types/iconv': '^3.0.1',
      '@types/uuid': '~9.0.2',
      nodemon: '^2.0.22',
      pino: '^8.12.1',
      'pino-pretty': '^10.0.0',
      prettier: '~2.8.8',
      typescript: '^5.0.4',
      vitest: '^0.34.3',
      vite: '^4.4.9'
    }
  },
  fs: { numberOfFiles: 89 },
  debug: [
    '2023-09-07T20:56:22.198Z config Attempting automatic configuration for angular\n',
    '2023-09-07T20:56:22.199Z angular/cli config Angular CLI not found.\n',
    '2023-09-07T20:56:22.199Z config Finished attempting automatic configuration for angular (1ms)\n',
    '2023-09-07T20:56:22.200Z config Attempting automatic configuration for jest\n',
    "2023-09-07T20:56:22.200Z jest/config Error: Module jest-cli is not found in '<homeDir>/dev/cadendar-monorepo/apps/compta-trpc-server'.\n" +
      '    at Object.loadJest (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:496:1070)\n' +
      '    at n (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:184:23339)\n' +
      '    at Object.configure (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:184:26449)\n' +
      '    at automaticConfigurationProviders.reduce.Promise.resolve.success (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:132:12015)\n' +
      '    at async automaticConfigurationProviders.reduce.Promise.resolve.success (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:132:11892)\n' +
      '    at async Config.load (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/server.js:132:11832)\n',
    '2023-09-07T20:56:22.200Z config Finished attempting automatic configuration for jest (0ms)\n',
    '2023-09-07T20:56:22.200Z config Attempting automatic configuration for vitest\n',
    '2023-09-07T20:56:22.202Z vitest/config Detected Vitest (0.34.3).\n',
    '2023-09-07T20:56:22.494Z config Finished attempting automatic configuration for vitest (294ms)\n',
    '2023-09-07T20:56:22.496Z project Wallaby Node version: v18.17.1\n',
    '2023-09-07T20:56:22.496Z project Wallaby config: <homeDir>/dev/cadendar-monorepo/apps/compta-trpc-server/auto.detect\n',
    '2023-09-07T20:56:22.520Z fs File system starting\n',
    '2023-09-07T20:56:22.560Z fs File system scan completed\n',
    '2023-09-07T20:56:22.563Z project File cache: <homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/projects/2e299c6745bab18e\n',
    '2023-09-07T20:56:22.595Z uiService Listening port 51235\n',
    '2023-09-07T20:56:22.600Z project package.json file change detected, invalidating local cache\n',
    '2023-09-07T20:56:22.604Z workers Parallelism for initial run: 1, for regular run: 1\n',
    '2023-09-07T20:56:22.604Z workers Starting run worker instance #0\n',
    '2023-09-07T20:56:22.604Z workers Web server is listening at 33513\n',
    '2023-09-07T20:56:22.604Z project File cache requires some updates, waiting required files from IDE\n',
    '2023-09-07T20:56:22.700Z project Stopping process pool\n',
    '2023-09-07T20:56:22.702Z project Test run started; run priority: 3\n',
    '2023-09-07T20:56:22.703Z project Running all tests\n',
    '2023-09-07T20:56:22.704Z workers Starting test run, priority: 3\n',
    '2023-09-07T20:56:22.704Z nodeRunner Starting sandbox [worker #0, session #og8p4]\n',
    '2023-09-07T20:56:22.704Z nodeRunner Preparing sandbox [worker #0, session #og8p4]\n',
    '2023-09-07T20:56:22.710Z workers Started run worker instance (delayed) #0\n',
    '2023-09-07T20:56:22.710Z nodeRunner Prepared sandbox [worker #0, session #og8p4]\n',
    '2023-09-07T20:56:22.710Z workers [worker #0, session #og8p4] Running tests in sandbox\n',
    '2023-09-07T20:56:22.717Z workers Sandbox (active) [og8p4] error: Failed to initialize wallaby vitest.\n' +
      "Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm)\n" +
      "Error: Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm)\n",
    '2023-09-07T20:56:22.819Z workers [og8p4] Run 0 test(s), skipped 0 test(s)\n',
    '2023-09-07T20:56:22.824Z workers [og8p4] Sandbox is responsive, closing it\n',
    '2023-09-07T20:56:22.826Z workers Failed to map the stack to user code, entry message: Failed to initialize wallaby vitest.\n' +
      "Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm)\n" +
      "Error: Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm), stack: Failed to initialize wallaby vitest.\n" +
      "Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm)\n" +
      "Error: Required dependency '@vitest/runner' not found. If you are using pnpm, please refer to our docs: (https://wallabyjs.com/docs/integration/vitest.html#using-pnpm)\n" +
      '    at initializeVitest (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/runners/node/vitest@0.14.0/initializer.js:14:5104)\n' +
      '    at v._onStart (<homeDir>/.cache/JetBrains/WebStorm2023.2/wallaby/wallaby/runners/node/vitest@0.14.0/initializer.js:14:25562)\n',
    '2023-09-07T20:56:22.827Z project Test run finished\n',
    '2023-09-07T20:56:22.827Z project Processed console.log entries\n',
    '2023-09-07T20:56:22.827Z project Processed loading sequences\n',
    '2023-09-07T20:56:22.827Z project Processed executed tests\n',
    '2023-09-07T20:56:22.828Z project Processed code coverage\n',
    '2023-09-07T20:56:22.835Z project Test run result processed and sent to IDE\n'
  ]
}

[smcenlly]

We are not very familiar with rush, but have spent some time reviewing their docs and setting up a sample and this is what we found.

• By default, there are is no package.json or dependencies installed in the mono-repo root. Unless you configure dependencies to exist in your project root, Wallaby won't run from the root. This means you will need to update your configuration to tell Wallaby to start on a folder containing a package.json and node_modules. You can do this by specifying the Root path in your Run Configuration.

• The .npmrc settings only apply to the project root (e.g. see [rush] set public-hoist-pattern but vs code not detect? microsoft/rushstack#3324), which means you will need to: "manually add phantom dependencies in package.json". There seems to be an open issue in the rush repo to fix this limitation: [rush] Add an option to disable hoisting for indirect dependencies. microsoft/rushstack#3542.

In a fresh sample repo, we were able to get Wallaby working with vitest by explicitly adding these dependencies to the project with vitest installed:

{
...
    "@vitest/runner": "~0.34.3",
    "@vitest/snapshot": "~0.34.3"
...
}

Once you've installed the dependencies and updated your your Run Configuration to set the Root path to your project folder, it should work for you.

It's possible that you will need other dependencies (in addition to @vitest/runner and @vitest/snapshot). If you don't need them right now, it's also possible you will need them in the future with vitest updates. Wallaby will tell you which dependency is missing so you can explicitly install it.

[vjau]

I can confirm this is working by adding those two phantom dependencies.