Built in shard support

[mohsen1]

Allow running a subset of tests. For instance run first 20% of tests. Or run 4th 20% of the test.

Motivation

Some projects grow so big that we have to shard the test suite meaning we have to run for instance 5 CI jobs and each job running 20% of the tests.

This is usually done by manually configuring each job to roughly run only a subset of tests that comes close to 20%. Over time those jobs can become larger or smaller with code changes.

Example

A CLI arguments can look like test

jest --shard-from=0 --shard-to=0.2

• --shard-* are numbers from 0 to 1 representing range of all tests suites found from start to end.
• --shard-from defaults to 0
• --shard-to defaults to 1

Pitch

This is unlike #5048 that is asking to run tests in parallel. I want to run the test in parallel CI jobs.

[ahnpnl]

Workaround is use a separate jest config for each CI job which has specific https://jestjs.io/docs/configuration#testpathignorepatterns-arraystring . Even though won’t work like your proposal but should help your case somehow.

[SimenB]

Is this the same as #6270?

[mohsen1]

@SimenB yes. I actually like --chunk=2/10 better than "shard". Is this something that can make it to Jest? I can work on it if you agree.

[SimenB]

We currently allow you to plug in a custom test sequencer. While it's not a CLI flag to chunk, I wonder if that's good enough to, via env variables or otherwise, split them up without jest needing to change?

[mohsen1]

@SimenB That works perfectly. Thank you!

const Sequencer = require('@jest/test-sequencer').default;

class ParallelSequencer extends Sequencer {
  jobsCount = parseInt(process.env.CI_JOB_COUNT, 10);

  jobIndex = parseInt(process.env.CI_JOB_INDEX, 10);

  /**
   * Select test files that should run in this machine
   * @param {import('jest-runner').Test[]} tests
   * @returns {import('jest-runner').Test[]}
   */
  sort(tests) {
    if (!process.env.CI) return tests;

    const chunkSize = Math.ceil(tests.length / this.jobsCount);
    const minIndex = this.jobIndex * chunkSize;
    const maxIndex = minIndex + chunkSize;

    return Array.from(tests).filter((_, index) => {
      return index >= minIndex && index <= maxIndex;
    });
  }
}

module.exports = ParallelSequencer;

[SimenB]

Awesome 👍 Thoughts on sticking that example in the docs along with the sorted one?

Also, is there a potential bug if tests is not sorted the same way across all workers? Can it be different? I think the order might be different based on data from older runs (e.g. runtime and if it failed), which might vary between workers?

---

I wonder if it'd be worth adding --chunk like you say... 🤔

[mohsen1]

Good point! A simpler sort function will look like this.

sort(tests) {
    if (!process.env.CI) return tests;

    return Array.from(tests)
      .sort((testA, testB) => (testA.path > testB.path ? 1 : -1))
      .filter((_, i) => i % this.jobsCount === this.jobIndex);
}

---

I still support --chunk!

[WeiAnAn]

Use slice would be better.

sort(tests) {
    if (!process.env.CI) return tests;

    const chunkSize = Math.ceil(tests.length / this.jobsCount);
    const minIndex = this.jobIndex * chunkSize;
    const maxIndex = minIndex + chunkSize;

    return Array.from(tests)
      .sort((testA, testB) => (testA.path > testB.path ? 1 : -1))
      .slice(minIndex, maxIndex)
}

---

I think built-in --chunk support will be nice!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.