Usage of Node CLI's --loader option with PnP

[tavoyne]

Hi there,

I'm using Yarn v3.2.0-rc.13. In order to execute Typescript/ESM files from the console, I take advantage of ts-node and its ts-node/esm loader via the following command:

yarn node --loader ts-node/esm scripts/build.ts

Although ts-node is installed, I'm getting an [ERR_MODULE_NOT_FOUND]: Cannot find package 'ts-node' error.

It seems that what's specified via the --loader option is not resolved via PnP.

Any workaround? Should I open an issue?

Note: everything works well if I unplug ts-node or if I set nodeLinker to node-modules.

[arcanis]

This is a current limitation of the Node loader API, Yarn itself cannot address that. You can follow the discussion on the Node repo:

• Loader chaining *while loading loaders* nodejs/loaders#56
• esm: add chaining to loaders nodejs/node#42623

[tavoyne]

Hi @arcanis, do you know if there's any progress on that? I see that the linked PR was merged but I still can't run yarn node --loader ts-node/esm scripts/build.ts (I'm on yarn@4.0.0-rc.5 now).

[merceyz]

The upstream PR is merged but not released yet, even when it's released it probably wont work since Node.js doesn't use the loaders to load other loaders.

[tavoyne]

Oh I see... Any workaround for using ts-node with PnP and ESM?

[josuevalrob]

updates?

[merceyz]

You can follow nodejs/node#43772

[koistya]

@tavoyne you can use babel-node to run .ts scripts for now (using @babel/node instead of ts-node, converting TypeScript into CommonJS behind the hood):

$ babel-node -x .ts ./scripts/build.ts

You can find a complete example in Relay Starter Kit → db (scripts)

[tavoyne]

Will try that, thanks!

[tavoyne]

@babel/node doesn't work with ESM yet 😥

See:

• Support ES Modules in @babel/preset-typescript babel/babel#14219
• [Feat]: Support "type": "module" in package.json for @babel/node babel/babel#14312

[marchaos]

loader chaining was merged recently - see nodejs/node#42623

[merceyz]

We are aware, see #4044 (reply in thread)

[VulgarnyKarlson]

https://github.com/nodejs/node/releases/tag/v16.17.0
updated at 16 august ( merged )

chaining loader are now supporting by 16.17.0
Can you are update ?

[VulgarnyKarlson]

so yarn berry will not support ts-node until node will not release loading loaders in loaders ? @merceyz

[VulgarnyKarlson]

i using currently nodeLinker: node-modules but .yarn storage seems like much better

[VulgarnyKarlson]

@merceyz as i correctly understand - you are can't update casue it's not even released , but at 16 august it's released in v16.17.0

[merceyz]

Something like nodejs/node#43772 needs to land upstream for this to work.

[VulgarnyKarlson]

https://github.com/nodejs/node/releases/tag/v19.6.0
Looks like we got update of ts-loader, is it enough to implement ?

[faces-of-eth]

Bummer that we cannot use Yarn PNP with ts-node, or do much to change things until other projects get some things landed. Is there any workaround currently for running Typescript command-line scripts while using Yarn PNP?

[neongreen]

A workaround that works for me is using esbuild, which supports Yarn PnP and transpiling single modules (unlike tsc).

yarn add -D esbuild    # NB: 0.15 didn't work for me, 0.17 did
yarn esbuild tools/whatever.ts --bundle --platform=node | yarn node

It will pick up the tsconfig automatically, but you can also give it --tsconfig=....

Here's an example from my own project, the "scripts" section in package.json:

{
  "name": "ches",
  "scripts": {
    ...,
    "sample-games": "rm sample-games/*.txt; esbuild tools/sample-games.ts --bundle --platform=node | node"
  },
  "devDependencies": {
    ...,
    "esbuild": "^0.17.0",
  },

[rtritto]

ts-import should be a workaround (the entrypoint file in script needs to be converted to .js).

[vonagam]

Since #43772 was merged in node 19.6.0 it seems to be possible to run esm typescript with ts-node and yarn pnp:

yarn node --loader ts-node/esm --loader .pnp.loader.mjs file.ts

But to import other ts/tsx files by a full path with ".js" extension (as esm requires) had to write a small custom loader:

Custom loader code

// .pnp-ts.loader.mjs
import Path from 'node:path';
import URL from 'node:url';
import FS from 'node:fs';


const mapping = new Map([
  ['.js', ['.js', '.ts', '.tsx', '.jsx']],
  ['.cjs', ['.cjs', '.cts']],
  ['.mjs', ['.mjs', '.mts']],
  ['.jsx', ['.jsx', '.tsx']],
]);

export function resolve(specifier, context, next) {
  if (!specifier.startsWith('.')) {
    return next(specifier, context);
  }

  const parentURL = context.parentURL;
  if (!parentURL || !parentURL.startsWith('file:') || parentURL.includes('/.yarn/')) {
    return next(specifier, context);
  }

  const specifiedExtension = Path.extname(specifier);
  const sourceExtensions = mapping.get(specifiedExtension);
  if (!sourceExtensions) {
    return next(specifier, context);
  }

  const location = Path.dirname(URL.fileURLToPath(parentURL));
  const required = specifier.slice(0, -specifiedExtension.length);
  const path = Path.join(location, required);
  for (const sourceExtension of sourceExtensions) {
    if (FS.existsSync(path + sourceExtension)) {
      return next(required + sourceExtension, context);
    }
  }

  return next(specifier, context);
}

Also, there is @esbuild-kit/esm-loader which transpiles typescript into esm on the fly (without type validations, but super fast) and now it is compatible with yarn without custom things for that purpose:

yarn node --loader @esbuild-kit/esm-loader file.ts

[EmilIvanichkovv]

Awesome!
Thank you for sharing!!!

@arcanis Can you mark this comment as answer?

[lizthegrey]

Confirmed that using a version of Node compiled with nodejs/node#43772 results in chain loading Yarn PNP and another loader (in our case, it's for hotpatching ESM modules with https://github.com/DataDog/import-in-the-middle) succeeding.

[RomainLanz]

What is the recommended solution today, given that the --loader flag has been deprecated in Node.js 20 but is still required by Yarn PnP?

Linked issue : #6589

[CraigMcCahill]

I've been trying to solve this problem this week. We are using Yarn PNP with Node, my task was to move the project to ESM.

Based on the docs here the --import flag replaces --loader flag. BUT I haven't managed to get this to work. I have also tried using --experimental-loader again with no success.

[clemyan]

yarn node --import ... should just work. Internally we use --experimental-loader to inject the PnP loader but that's just implementation detail.

--import is not a drop-in replacement for --loader. Migrating from --loader to --import requires code changes in the loader itself, but the APIs necessary for migration was introduced in 18.19.0 and --import itself was introduced in 18.18.0. Yarn v4 supports all the way down to 18.12.0, so migrating now would just be a lot of unnecessary churn, especially given that --experimental-loader is probably not going away anytime soon.

[CraigMcCahill]

@clemyan thank you for your quick and knowledgable reply. Can I ask then what is the current recommended replacement for ESM for this command which worked with commonjs please?

node -r ./.pnp.cjs ./packages/my-package/dist/main.js

I am using Node 22

[clemyan]

For maximum forward-compatibility, yarn node ./packages/my-package/dist/main.js. The yarn node command, by design, will set up the node call for based on your workspace and configuration (e.g. putting bins of dependencies in PATH, setting up the PnP CJS hook when using the PnP linker, setting up the PnP ESM loader if it exists, etc).

If you need to set up PnP manually, then node --require ./.pnp.cjs --experimental-loader ./.pnp.loader.mjs ./packages/my-package/dist/main.js also just works

To migrate that command to --import, node's own experimental warning suggests using an inline module to register the loader:

node --require ./.pnp.cjs --import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("./.pnp.loader.mjs", pathToFileURL("./"));'

Alternatively, you can commit that module as a file on disk

// register.mjs
import { register } from "node:module";
import { pathToFileURL } from "node:url";
register("./.pnp.loader.mjs", import.meta.url);

then you can just --import it:

node --require ./.pnp.cjs --import ./register.mjs`

[CraigMcCahill]

Thank you again, @clemyan this now works. EDIT - the problem I had below was caused by running this inside a docker container and only copying the /dist directory and not the package.json as well with "type": "module". Include both and it should work either with --experimental-loader or --import

If you get this error check your package.json.

Error [ERR_REQUIRE_CYCLE_MODULE]: Cannot require() ES Module /app/packages/my-package/dist/main.js in a cycle.
    at ModuleLoader.importSyncForRequire (node:internal/modules/esm/loader:315:15)
    at loadESMFromCJS (node:internal/modules/cjs/loader:1414:24)
    at Module._compile (node:internal/modules/cjs/loader:1547:5)
    at Object..js (node:internal/modules/cjs/loader:1708:10)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)
    at cjsLoader (node:internal/modules/esm/translators:263:5)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:196:7) {
  code: 'ERR_REQUIRE_CYCLE_MODULE'
}

Node.js v22.12.0