[Documentation]: CLI --filter option might no be clear enough / Filter {filterPath} did not return a valid test list

[gramsco]

Version

27.4.5

Steps to reproduce

Steps :
(1) Go to the documentation about filter file
(2) Create a working project with a valid jest config
(3) Create a filter file as stated in the doc to be used with bash --filter=<filename>
(4) Try a few functions
(5) Receive "Filter {filterPath} did not return a valid test list"

Expected behavior

I expected the documentation to provide what I was supposed to write in the file, that is :

"[a] Path to a module exporting a filtering function."

One could expect a function of the type

function filteringFunction(files:string[]):string[]{
  return files.filter( /** logic **/ ) 
}

Or even :

function isFileValid(file:string):boolean {
  return applyLogic(file)
}

Actual behavior

But, because the code is the following :

if (filter) { 
      const tests = searchResult.tests;
      const filterResult = await filter(tests.map(test => test.path));

      console.log({ filterResult })

      if (!Array.isArray(filterResult.filtered)) {
        console.log(filterResult.filtered)
        throw new Error(
          `Filter ${filterPath} did not return a valid test list`
        );
 }

and also

 const filteredSet = new Set(
        filterResult.filtered.map(result => result.test),
      );

      return {
        ...searchResult,
        tests: tests.filter(test => filteredSet.has(test.path)),
      };

I got it to work by using the very undocumented:

module.exports = function filter(testPaths) {
  const filtered = testPaths
    .filter((testPath) => isFiltered(testPath))
    .map((testPath) => ({ test: testPath }));

  return {
    filtered,
  };
};

I also discovered it was AN ASYNC function :)

Additional context

I would suggest first to maybe update the documentation with something of the sort:

--filter=<file>

Path to a module exporting a filtering function. This asynchronous function receives a list of paths which can be manipulated to exclude tests from running by returning an object with the "filtered" property. Especially useful when used in conjunction with a testing infrastructure to filter known broken.

Example:

// my-filter.js
module.exports = (testPaths) => {
  const allowedPaths = filterPaths(testPaths) // ["path1.spec.js", "path2.spec.js", etc]
  return {
      filtered : allowedPaths,
   }
}

And (but I don't know if it's possible), to edit the set creation

const filteredSet = new Set(
   ❌ filterResult.filtered.map(result => result.test) 
   ✅ filterResult.filtered
);

Environment

System:
    OS: macOS 12.0.1
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
  Binaries:
    Node: 14.17.0 - ~/.nvm/versions/node/v14.17.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 6.14.13 - ~/.nvm/versions/node/v14.17.0/bin/npm
  npmPackages:
    jest: ^27.4.5 => 27.4.5

[gramsco]

I would be happy to do a PR for the documentation part.
As for the the new Set part, I would say I am not experienced enough with Jest's codebase!

[renatoargh]

Just wanted to stress that I spent some good time trying to figure out what was wrong until I finally found this issue. Doc'ing how to filter the tests would be ideal in this case

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[gramsco]

I don't think this should be closed but I'm biased :)

[IanVS]

I'm glad to find this issue, because I had no idea how to use this cli argument.

[MakersAll8]

why is testPaths not deterministic? How do we get a deterministic testPaths?

[MakersAll8]

Oops... I could just sort testPaths myself...

[martinblostein]

Thanks for documenting this here at least.

[mrazauskas]

Just to leave a note. Jest’s end-to-end tests is a great usage reference too: https://github.com/facebook/jest/tree/main/e2e/filter

[SimenB]

PR very much welcome for doc changes

[gramsco]

@SimenB Done! Waiting for the CLA.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.