Support Testing with imports of JS files in ESM

[daraclare]

Current Behavior

When running test files with that contain ES6 module imports with Jest, I get the following error:

Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

Is there an existing recommendation for handling ES6/ESNext module imports, if so, could we add documentation around this? If it doesn't exist, could we add recommendation on how to handle it? I'm happy to help if I can. Could this be a TSDX feature, to switch on and off?

Desired Behavior

Either handled by default by TSDX, or there is clear recommendations on how best to handle this.

Suggested Solution

Happy to add docs with the recommendation to resolve, although I'm not sure what the recommendations are as yet.

Who does this impact? Who is this for?

Users importing other packages that use ES6.

Describe alternatives you've considered

I considered using babel-jest and configuring it myself, but this seems to make TSDX a little redundant.

[swyxio]

gd qtn. i honestly have no idea. should we recommend that test files are also in typescript? that would be another angle to this.

[daraclare]

Our tests are written in typescript and work fine, until we try to import another package of ours from the NPM registry, which is compiled to ES6. Then Jest complains about that test, with the error above. It still runs other tests that don't import packages in ES6. I'm assuming this is a common enough issue, maybe it's not 🤔

[jaredpalmer]

Very possible, due to the design goals of tsdx, this doesn’t surprise me. Have you tried setting the allowJs flag in tsconfig to true?

[jaredpalmer]

Sorry I may have misread, tsdx is not compiling node_modules at the moment with babel (like CRA does) IIRC. We might want to fix that.

[daraclare]

What would you recommend as a workaround for now, should we add this to the docs?

[daraclare]

Thanks for the quick replies both, btw!

[swyxio]

fyi CRA stopped compiling node_modules in v2

[daraclare]

I don't think we'd want to compile all our node_modules, just ones that are imported into tests, is that possible I wonder? What would be the best practice for resolving this with TSDX?

[mikecousins]

Yeah this one is really biting me currently,as the library that I develop wraps up an es6 module so none of my tests work. If anyone has any hints for how to get this working that would be great.

[zhaoyao91]

A quite common case: I use lodash-es in my project, so the test just boomed.

[agilgur5]

So this was fixed by my #486 very recently. Though if you want to transpile something from node_modules in your tests, you'll have to override transformIgnorePatterns in a jest.config.js as by default it doesn't transpile node_modules (that's part of Jest's default behavior)

[neilchaudhuri]

I am seeing what I believe is a similar issue that I could use some guidance with. I described it in some detail on Stack Overflow. The bottom line is that I am importing the Chakra theme.js into my test and getting the same SyntaxError: Unexpected token 'export' error.

This happens whether theme.js is at the top level or, just for experimentation, copied into the test folder where my tests and renderWithTheme utility reside.

I'm sure this is a simple matter of configuration, but I've been spinning my wheels for days now trying to figure it out. Any guidance is appreciated.

[agilgur5]

@neilchaudhuri TypeScript doesn't use file extensions, so I'd try removing the .js to start

[neilchaudhuri]

@agilgur5 Thanks for the suggestion. That is how I had it initially, but one of the error messages I got along the way prompted me to be more explicit. In all cases I get the same error.

I should note also importing theme (with or without the extension) works just fine in the stories

[karlhorky]

TypeScript doesn't use file extensions

This is not true:

1. TypeScript supports using a .js extension in the import - also just learned this recently
2. If you want to export to ESM, you need to use the .js extension in your imports

[karlhorky]

So this was fixed by my #486 very recently

I don't believe this is fixed yet and this should be reopened.

Using the latest tsdx@0.14.1, Jest tests fail when they import something in ESM (eg. p-map) from node_modules, also when using the NODE_OPTIONS=--experimental-vm-modules option as described in the docs:

Repo: https://github.com/karlhorky/tsdx-test-esm-fail
Repl.it (just hit the Run button at the top): https://replit.com/@karlhorky/tsdx-test-esm-fail

$ yarn test     
yarn run v1.22.10
$ tsdx test
 FAIL  test/blah.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    tsdx-test-esm-fail/node_modules/p-map/index.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){import AggregateError from 'aggregate-error';
                                                                                             ^^^^^^

    SyntaxError: Cannot use import statement outside a module

    > 1 | import pMap from 'p-map';
        | ^
      2 |
      3 | console.log('p-map', pMap);
      4 |

      at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1258:14)
      at Object.<anonymous> (src/index.ts:1:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        1.326s, estimated 2s

[karlhorky]

Maybe one option for fixing this would be to add the @babel/plugin-transform-modules-commonjs to the TSDX Babel preset (only in the test environment), so that at least users can choose to transform files using the jest.config.js escape hatch (from the repo above):

module.exports = {
  transformIgnorePatterns: [
    '[/\\\\]node_modules[/\\\\](?!aggregate-error|clean-stack|escape-string-regexp|indent-string|p-map).+\\.(js|jsx)$',
  ],
};

It doesn't seem possible right now using a .babelrc or .babelrc.js file (TSDX readme is inconsistent with the details of which filename should be used)

[karlhorky]

Currently ESM support for tsdx test looks broken with no way out except for patching the package using something like patch-package.

[karlhorky]

Oh, it seems a babel.config.js or babel.config.cjs file works 🤪 , instead of .babelrc or .babelrc.js mentioned in the readme:

https://github.com/karlhorky/tsdx-test-esm-fail/pull/1/files

Opened a PR to document this: #1019

[karlhorky]

I guess jest.config.cjs is not supported currently (and jest.config.js written in ESM also not), which means that "type": "module" in package.json is not currently supported, unless you write your Jest config in the package.json:

#1020

[lwazevedo]

Hi,

Any solution to this problem?

I'm going through the same situation.

I have a third-party file in my project that internally imports two libs in js and when I run the test I get this error:

Jest encountered an unexpected token

This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

Here's what you can do:
 • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
 • If you need a custom transformation specify a "transform" option in your config.
 • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/en/configuration.html

I solved it using @karlhorky suggestion documented in pr #1019, but I would like to know if this problem will be solved.