Support jest-circus

[njmaeff]

Issue description or question

I'm trying to use the jest-circus test runner but I receive an error. Wallaby is referencing jasmine which does not exist.

I have created a sample repo.

Wallaby diagnostics report

{
  editorVersion: 'WebStorm 2020.2.3',
  pluginVersion: '1.0.203',
  editorType: 'IntelliJ',
  osVersion: 'linux 5.4.0-52-generic',
  nodeVersion: 'v12.18.3',
  coreVersion: '1.0.991',
  checksum: 'OTBlNzM4ZDM2MTc4Yzk4MzY1ZjJjMTdhOGRlYmYyZWYsMTYxMzY5MjgwMDAwMCww',
  config: {
    files: [
      { pattern: 'src/**/*.test.js', ignore: true, trigger: true, load: true, instrument: true },
      { pattern: 'src/**/*.js', ignore: false, trigger: true, load: true, instrument: true, order: 1 },
      { pattern: 'src/**/*.test.js.snap', ignore: false, instrument: false, trigger: true, load: true, order: 2 },
      { pattern: 'package.json', ignore: false, instrument: false, trigger: true, load: true, order: 3 }
    ],
    tests: [ { pattern: 'src/**/*.test.js', ignore: false, trigger: true, load: true, test: true, order: 4 } ],
    env: { type: 'node', runner: 'node', params: {}, viewportSize: { width: 800, height: 600 }, options: { width: 800, height: 600 }, bundle: true },
    testFramework: { version: 'jest@0.4.3', configurator: 'jest@0.4.3', reporter: 'jest@0.4.3', starter: 'jest@0.4.3' },
    diagnostics: {},
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    compilers: { '**/*.?(lit)coffee?(.md)': [Function] },
    preprocessors: { 'package.json': [Function] },
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    workers: { initial: 0, regular: 0, recycle: false },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    extensions: {},
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: true,
    configCode: 'module.exports = function (wallaby) {\n' +
      '    return {\n' +
      '        files: [\n' +
      '            "!src/**/*.test.js",\n' +
      '            "src/**/*.js",\n' +
      '        ],\n' +
      '\n' +
      '        tests: ["src/**/*.test.js"],\n' +
      '\n' +
      '        env: {\n' +
      '            type: "node",\n' +
      '            runner: "node",\n' +
      '        },\n' +
      '\n' +
      '        testFramework: "jest",\n' +
      '\n' +
      '        setup(wallaby) {\n' +
      '            wallaby.testFramework.configure({testRunner: "jest-circus/runner"});\n' +
      '        },\n' +
      '    };\n' +
      '};\n'
  },
  packageJSON: { dependencies: undefined, devDependencies: { '@types/jest': '^26.0.15', jest: '^26.6.3', 'jest-circus': '^26.6.3' } },
  fs: { numberOfFiles: 3 },
  debug: [
    '2020-11-27T02:39:51.311Z project Wallaby Node version: v12.18.3\n',
    '2020-11-27T02:39:51.311Z project Wallaby config: <homeDir>/repos/contrib/wallaby/jest-circus/wallaby-circus.js\n',
    '2020-11-27T02:39:51.891Z project File cache: <homeDir>/.cache/JetBrains/WebStorm2020.2/wallaby/projects/569adc4d63e9ca86\n',
    '2020-11-27T02:39:51.898Z uiService Listening port 51235\n',
    '2020-11-27T02:39:51.927Z workers Parallelism for initial run: 14, for regular run: 7\n',
    '2020-11-27T02:39:51.927Z workers Starting run worker instance #0\n',
    '2020-11-27T02:39:51.927Z workers Starting run worker instance #1\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #2\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #3\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #4\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #5\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #6\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #7\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #8\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #9\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #10\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #11\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #12\n',
    '2020-11-27T02:39:51.928Z workers Starting run worker instance #13\n',
    '2020-11-27T02:39:51.934Z workers Web server is listening at 36239\n',
    '2020-11-27T02:39:52.031Z project Stopping process pool\n',
    '2020-11-27T02:39:52.031Z project File cache is up-to-date, starting full test run\n',
    '2020-11-27T02:39:52.037Z project Running postprocessor\n',
    '2020-11-27T02:39:52.040Z postprocessor New TypeScript language service is required\n',
    "2020-11-27T02:39:52.040Z postprocessor TypeScript postprocessor will be removed because in 3 project files none were found by patterns: [ '**/*.ts?(x)' ]\n",
    '2020-11-27T02:39:52.043Z project Postprocessor execution finished\n',
    '2020-11-27T02:39:52.043Z project Postprocessor is removed as requested by itself\n',
    '2020-11-27T02:39:52.043Z project Test run started; run priority: 3\n',
    '2020-11-27T02:39:52.044Z project Running all tests\n',
    '2020-11-27T02:39:52.045Z workers Starting test run, priority: 3\n',
    '2020-11-27T02:39:52.045Z nodeRunner Starting sandbox [worker #0, session #44htj]\n',
    '2020-11-27T02:39:52.046Z nodeRunner Preparing sandbox [worker #0, session #44htj]\n',
    '2020-11-27T02:39:52.181Z workers Started run worker instance (delayed) #1\n',
    '2020-11-27T02:39:52.181Z workers Started run worker instance (delayed) #3\n',
    '2020-11-27T02:39:52.182Z workers Started run worker instance (delayed) #2\n',
    '2020-11-27T02:39:52.185Z workers Started run worker instance (delayed) #0\n',
    '2020-11-27T02:39:52.186Z nodeRunner Prepared sandbox [worker #0, session #44htj]\n',
    '2020-11-27T02:39:52.186Z workers [worker #0, session #44htj] Running tests in sandbox\n',
    '2020-11-27T02:39:52.187Z workers Started run worker instance (delayed) #4\n',
    '2020-11-27T02:39:52.202Z workers Started run worker instance (delayed) #6\n',
    '2020-11-27T02:39:52.203Z workers Started run worker instance (delayed) #5\n',
    '2020-11-27T02:39:52.207Z workers Started run worker instance (delayed) #7\n',
    '2020-11-27T02:39:52.212Z workers Started run worker instance (delayed) #8\n',
    '2020-11-27T02:39:52.223Z workers Started run worker instance (delayed) #10\n',
    '2020-11-27T02:39:52.224Z workers Started run worker instance (delayed) #9\n',
    '2020-11-27T02:39:52.239Z workers Started run worker instance (delayed) #11\n',
    '2020-11-27T02:39:52.243Z workers Started run worker instance (delayed) #12\n',
    '2020-11-27T02:39:52.255Z workers Started run worker instance (delayed) #13\n',
    '2020-11-27T02:39:53.220Z workers Sandbox (active) [44htj] error: ReferenceError: jasmine is not defined\n',
    '2020-11-27T02:39:53.222Z workers Failed to map the stack to user code, entry message: ReferenceError: jasmine is not defined, stack: ReferenceError: jasmine is not defined\n' +
      '    at Object.<anonymous> (<homeDir>/.cache/JetBrains/WebStorm2020.2/wallaby/wallaby/runners/node/jest@0.4.3/jasmineInitializer.js:14:33)\n' +
      '    at Runtime._execModule (<homeDir>/repos/contrib/wallaby/jest-circus/node_modules/jest-runtime/build/index.js:1299:24)\n' +
      '    at Runtime._loadModule (<homeDir>/repos/contrib/wallaby/jest-circus/node_modules/jest-runtime/build/index.js:898:12)\n' +
      '    at Runtime.requireModule (<homeDir>/repos/contrib/wallaby/jest-circus/node_modules/jest-runtime/build/index.js:746:10)\n' +
      '    at Runtime.p._wallabyReplaced.c.requireModule (<homeDir>/.cache/JetBrains/WebStorm2020.2/wallaby/wallaby/runners/node/jest@0.4.3/initializer.js:14:29085)\n' +
      '    at jestAdapter (<homeDir>/repos/contrib/wallaby/jest-circus/node_modules/jest-circus/build/legacy-code-todo-rewrite/jestAdapter.js:93:15)\n' +
      '    at processTicksAndRejections (internal/process/task_queues.js:97:5)\n' +
      '    at runTestInternal (<homeDir>/repos/contrib/wallaby/jest-circus/node_modules/jest-runner/build/runTest.js\n',
    '2020-11-27T02:39:53.226Z workers [44htj] Run 0 test(s), skipped 0 test(s)\n',
    '2020-11-27T02:39:53.228Z workers [44htj] Sandbox is responsive, closing it\n',
    '2020-11-27T02:39:53.229Z project Test run finished\n',
    '2020-11-27T02:39:53.229Z project Processed console.log entries\n',
    '2020-11-27T02:39:53.229Z project Processed loading sequences\n',
    '2020-11-27T02:39:53.229Z project Processed executed tests\n',
    '2020-11-27T02:39:53.230Z project Processed code coverage\n',
    '2020-11-27T02:39:53.238Z project Test run result processed and sent to IDE\n'
  ]
}

[ArtemGovorov]

Right now Wallaby doesn't support jest-circus with manual config, having said that Wallaby automatic configuration for your project should just work. Automatic configuration is how we recommend to run Jest with Wallaby.

Internally automatic configuration will use jest-jasmine2, but once we add integration with jest-circus, it'll be using it when required, without you having to do anything.

Anyway, the internal test runner should not affect how Jest/Wallaby runs your tests and using jest-jasmine2 should be perfectly fine (and to provide the same results as jest-circus). Is there any specific reason you want to use specifically jest-circus with Wallaby?

[njmaeff]

Thanks. Jest-circus will be a default test runner in the next major version jestjs/jest#6295. Also, jest-jasmine2 has some goofy issues with test setup so I have been exploring jest-circus (jestjs/jest#6527). I will continue using jest-jasmine2. It is possible to configure jest with a custom test runner so maybe I can suggest reporting an more descriptive error when a custom test runner is used?

[ArtemGovorov]

Thanks for your response.

I will continue using jest-jasmine2. It is possible to configure jest with a custom test runner so maybe I can suggest reporting an more descriptive error when a custom test runner is used?

Wallaby automatic configuration for your project should just work even you configure jest-circus as your runner. We'll add some error message for the manual Wallaby config in case if not supported test runner is used, but manual Wallaby config for Jest is maintained as legacy at this stage and we recommend using automatic configuration.

[smcenlly]

We have now added support for jest-circus runner.