esm: consolidate ESM Loader methods #37468

JakobJingleheimer · 2021-02-21T18:29:31Z

Implements step 1 of #36396

Consolidate loader hooks:
- resolve = resolve [+getFormat]
- load = getFormat + getSource + transformSource
- globalPreload = getGlobalPreload
Consolidate node's internal default hooks:
- new load (wrap existing get_format & get_source; delete transform_source)
- resolve
Update when unknown format type (ERR_UNKNOWN_MODULE_FORMAT) is checked / thrown from after resolve to after load

For code changes:

Include tests for any bug fixes or new features.
Update documentation if relevant.
Ensure that make -j4 test (UNIX), or vcbuild test (Windows) passes.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

JakobJingleheimer · 2021-02-21T18:32:46Z

I ordinarily try to avoid touching lines that don't otherwise have changes, but I had to tidy up code that was otherwise untouched because it was very difficult to read & follow (it looked like it had been through an uglifier—perhaps some kind of lint autofixer?).

JakobJingleheimer · 2021-02-21T23:23:05Z

@GeoffreyBooth I have a few questions:

There're very few code comments (at least in the ESM section), and it's very difficult to figure out what anything is doing / supposed to do. I haven't come across any docs that seem to cover this aside from "try to avoid unnecessary breaking changes".

I know this particular piece is intended to merely consolidate some methods, but as far as I've managed to glean, a lot of the ESM code needs ground-up change (ex how the Loader class is designed and used is…very strange). How much change there might you/y'all be open to?

Happy to jump on a call to discuss. I'm EST (Mondays and Fridays after ~17:30 would be good for me).

GeoffreyBooth · 2021-02-21T23:57:34Z

I know this particular piece is intended to merely consolidate some methods, but as far as I’ve managed to glean, a lot of the ESM code needs ground-up change (ex how the Loader class is designed and used is…very strange). How much change there might you/y’all be open to?

@guybedford is the expert on this part of the codebase, I think. Perhaps he can answer that. I would be happy to join a call but my schedule is difficult and I don’t know that part of the codebase too well either (just the hooks, for the most part).

I assume you’ve reviewed #35524? I was thinking that the first PR would just be completing what was started on that branch.

JakobJingleheimer · 2021-02-22T00:02:25Z

Yes, I reviewed it to help figure out what the code is doing. I'm still not 100% (probably closer to 70%). I'd like to run an approach by someone more familiar.

devsnek · 2021-02-22T00:42:25Z

@JakobJingleheimer the code is split into explicit phases (resolve, getformat, getsource, transform). In terms of the system model of the loader, I'm not sure this is clearer. Additionally, you should run a lint pass on this code (make lint-js) locally, as it looks like it has a few issues.

This is also going to need a review for primordials. For example, array.push(v) should be ArrayPrototypePush(array, v).

devsnek · 2021-02-22T00:46:56Z

Actually I'm now realizing this is changing the public loader API. I will remove myself from the discussion.

JakobJingleheimer · 2021-02-22T00:47:00Z

@devsnek i gathered about the phases ;)

It's a WIP commit; I didn't bother with the non-IDE-integrating linter yet.

On a related note, why 80 columns? Are we devving on iPads?

GeoffreyBooth · 2021-02-22T04:19:51Z

On a related note, why 80 columns? Are we devving on iPads?

The Node codebase is over ten years old, so some things you just have to live with at this point 😄 My biggest pet peeve is that its test suite is written in Python.

@devsnek is also a good resource for this part of the codebase, he knows it better than I do.

GeoffreyBooth · 2021-02-22T06:07:20Z

(Huge) issue for another time: would it be feasible and desirable to port the test suite to something JavaScript-based like Mocha? I’m pretty savvy at unit tests (but I dunno if I’m up for a Sisyphean labour on my own).

The tests themselves are mostly in JavaScript. The runner is Python. Remember, the test runner for Node was written before any Node-based test runner existed 😉 You won’t need to know any Python to write tests.

I wouldn’t spend time on things like trying to change Node’s linting rules. The project is very mature at this point and seemingly little things like that are big efforts now because of the number of open branches and collaborators and release lines that things need to be backported to, etc. Also changes like that would need sign-off by a huge number of people, and it’s just not worth the effort if it’s not blocking you from the immediate task you’re trying to accomplish.

GeoffreyBooth · 2021-02-22T06:10:40Z

Actually I’m now realizing this is changing the public loader API. I will remove myself from the discussion.

@devsnek The goal is to change both the public and private loader. See #35524. The goal is to complete that PR.

You’re right about primordials, but we can fix all those at the end once we have a working PR.

JakobJingleheimer · 2021-02-22T06:48:24Z

@devsnek Thanks for the tip about array.push → primordial!

Where do primordials come from? They appear to be magically available (no import/require). This morning I tried to find them via a file search on the repo, but didn't find where they're defined.

JakobJingleheimer · 2021-03-06T19:05:02Z

Okay, so since @devsnek and @guybedford aren't available for that call to discuss, I'll try to summarise here:

esm_loader.js instantiates ESMLoader¹ twice: first to use esmLoader.import(), then immediately instantiate a new instance (overwriting the previous) to pass the output from esmLoader1.import to esmLoader2.hooks (abbreviated / pseudo code):

hooks = (new ESMLoader()).import(userLoaderSpecifier)
(new ESMLoader()).hooks(hooks)
// esmLoader2.runGlobalPreloader()

ESMLoader.import uses:

ESMLoader.resolve()
ESMLoader.getFormat()
ESMLoader.moduleMap

For esmLoader1, resolve and getFormat are node's built-ins.

I think instead:

rename the Loader class to ESMLoader
ESMLoader's constructor should accept/receive user loader specifier(s)
- update esm_loader to instantiate ESMLoader only once
ESMLoader's constructor should handle importing & caching the user loaders
ESMLoader's constructor should automatically call runGlobalPreloadCode() at the very end of its execution
update ESMLoader's hook properties (now globalPreloaders, resolvers, and loaders) to lists, pre-populated with node's default hook handlers

¹ the class itself is actually named Loader but imported as ESMLoader, so for clarity I'm referring to it as ESMLoader everywhere

GeoffreyBooth · 2021-03-08T06:11:41Z

I'm not too familiar with that section of the codebase, but in general this approach seems fine to me.

guybedford · 2021-03-08T08:49:30Z

@JakobJingleheimer happy to answer questions and review, two things would help to do that:

Seeing the docs diff is the most useful thing to tell what part of the public API is actually changing here.
If you have specific refactoring questions, just listing them in a question that won't get lost in the thread would help to not miss answering anything.

update esm_loader to instantiate ESMLoader only once

We have an outer and inner loader to ensure that the hooks do not affect the loader modules themselves.

JakobJingleheimer · 2021-03-08T15:00:50Z

@guybedford sure! I'll update the docs over lunch today. Would you be available to chat about them this evening? (If so, mind DMing me on Twitter or emailing me?)

My questions are mostly around internal approach (I think the changes to the public API are pretty simple and agreed).

GeoffreyBooth · 2021-03-13T00:14:47Z

@DerekNonGeneric You should chat with @JakobJingleheimer here about this PR, I think this could be a good effort to collaborate on.

JakobJingleheimer · 2021-03-13T00:31:57Z

Yes please! I didn't expect this to drag on so long 😓

@guybedford did you see my previous message here? (I didn't receive any follow-up from you—did I miss it somewhere?)

When I try to build, I get the following error, which I don't know how to properly fix. I presume it's because I deleted the obsolete transform_source.js (which was pretty much empty anyway):

make[1]: *** No rule to make target `../lib/internal/modules/esm/transform_source.js', needed by `[…]/node/out/Release/obj/gen/node_javascript.cc'.  Stop.

I see in #37468 that file was replaced with a comment (presumably to circumvent this error). I can do that, but I think that's not the ideal thing to do.

guybedford · 2021-03-13T11:24:52Z

@JakobJingleheimer feel free to DM me further to arrange a discusson, although written questions would always be preferable for context.

JakobJingleheimer · 2021-03-26T02:37:46Z

I'm trying to get the tests updated so I can verify the updates, but I don't recognise what it uses—seems to be something homebrewed?

I tried compiling, but I think it got stuck in a loop (make ran for like 20–30 minutes, and the output seemed to repeat after about 5). Possibly related: I have an M1 mac (and I know that's experimental).

I have tomorrow afternoon I can work on this. It would be a lot more productive not having to discover the context around the ESM subsystem (and test framework).

I didn't hear back from @guybedford, and I noticed @DerekNonGeneric has a note in his profile that he's taking a break from Node work.

GeoffreyBooth · 2021-03-26T02:48:30Z

@DerekNonGeneric has a note in his profile that he's taking a break from Node work.

I think he's back now.

DerekNonGeneric · 2021-03-26T04:46:52Z

Yeah, thanks for reaching out. I was planning to get around to doing this a bit later next month. We can continue work on your branch, but I am not 100% on the naming you have here. We can bikeshed in the #node-dev Freenode IRC channel if you're up for it (ping me on there tomorrow if you can).

JakobJingleheimer · 2021-03-29T19:35:49Z

Chatted with Derek, and he wants to revisit the consolidation discussion (to potentially reduce the hooks down to just 1). He's planning to return from his break in late April. Conceptually, I'm not sure if the One Hook to Rule Them All is logically possible (aside from that, I am a fan of simplicity).

I didn't get any answers to my questions so far, so I'm still groping around in the dark.

And now with the potential discussion revisit, I'm not sure what the status of this PR is and whether I should still try to figure out the unit tests.

DerekNonGeneric · 2021-03-29T20:00:05Z

@JakobJingleheimer, please keep this as a draft for now. I will address your questions momentarily too.

nodejs/node#37468

* chore: bump node in DEPS to v16.12.0 * chore: update patches * esm: consolidate ESM Loader methods nodejs/node#37468 Co-authored-by: electron-roller[bot] <84116207+electron-roller[bot]@users.noreply.github.com> Co-authored-by: PatchUp <73610968+patchup[bot]@users.noreply.github.com> Co-authored-by: Shelley Vohr <shelley.vohr@gmail.com>

https://build.opensuse.org/request/show/930406 by user adamm + dimstar_suse - BR python 3.6+ - Update to 16.13.0: * Experimental ESM Loader Hooks API nodejs/node#37468 * deps: upgrade npm to 8.1.0 (npm team) * vm: add support for import assertions in dynamic imports - Changes in 16.11.1: * deps: update llhttp to 6.0.4 - HTTP Request Smuggling due to spaced in headers (bsc#1191601, CVE-2021-22959) - HTTP Request Smuggling when parsing the body (bsc#1191602, CVE-2021-22960) - Changes in 16.11.0: * deps: update nghttp2 to v1.45.1 - Changes in 16.10.0: * crypto: add rsa-pss keygen parameters * fs: make open and close stream override optional when unused * http: limit requests per connection The maximum number of requests a socket can handle before closing keep alive connection can be set with server

Fixes: #41189 PR-URL: #41198 Refs: #37468 Reviewed-By: Derek Lewis <DerekNonGeneric@inf.is> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de>

This allows `ts-node` to be used to run TypeScript files without having to compile them with `tsc` first. It also adds the necessary configs, including `tsconfig-paths` for `paths` import-alias resolution. Unfortunately, this means we have to remove the `--experimental-module-resolution=node` since `ts-node` uses its own loader and thus form of resolving modules. See: Setup * https://medium.com/@jimcraft123hd/setting-up-path-alias-in-typescript-and-tsc-build-without-error-9f1dbc0bccd2 Issues with `ts-node`, ESM, and aliases * TypeStrong/ts-node#1007 - TypeStrong/ts-node#476 - dividab/tsconfig-paths#122 (comment) - TypeStrong/ts-node#1450 (comment) * TypeStrong/ts-node#1414 * TypeStrong/ts-node#995 - TypeStrong/ts-node#639 Node issues with ESM * https://nodejs.org/api/packages.html#determining-module-system * nodejs/node#37468

Fixes: #41189 PR-URL: #41198 Refs: #37468 Reviewed-By: Derek Lewis <DerekNonGeneric@inf.is> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de>

Fixes: nodejs#41189 PR-URL: nodejs#41198 Refs: nodejs#37468 Reviewed-By: Derek Lewis <DerekNonGeneric@inf.is> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de>

Fixes: #41189 PR-URL: #41198 Refs: #37468 Reviewed-By: Derek Lewis <DerekNonGeneric@inf.is> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Richard Lau <rlau@redhat.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de>

# Modules: ECMAScript modules    > Stability: 2 - Stable ## Introduction  ECMAScript modules are [the official standard format][] to package JavaScript code for reuse. Modules are defined using a variety of [`import`][] and [`export`][] statements. The following example of an ES module exports a function: ```js // addTwo.mjs function addTwo(num) { return num + 2; } export { addTwo }; ``` The following example of an ES module imports the function from `addTwo.mjs`: ```js // app.mjs import { addTwo } from './addTwo.mjs'; // Prints: 6 console.log(addTwo(4)); ``` Node.js fully supports ECMAScript modules as they are currently specified and provides interoperability between them and its original module format, [CommonJS][].  ## Enabling  Node.js has two module systems: [CommonJS][] modules and ECMAScript modules. Authors can tell Node.js to use the ECMAScript modules loader via the `.mjs` file extension, the `package.json` [`"type"`][] field, or the [`--input-type`][] flag. Outside of those cases, Node.js will use the CommonJS module loader. See [Determining module system][] for more details.  ## Packages This section was moved to [Modules: Packages](packages.md). ## `import` Specifiers ### Terminology The _specifier_ of an `import` statement is the string after the `from` keyword, e.g. `'path'` in `import { sep } from 'path'`. Specifiers are also used in `export from` statements, and as the argument to an `import()` expression. There are three types of specifiers: * _Relative specifiers_ like `'./startup.js'` or `'../config.mjs'`. They refer to a path relative to the location of the importing file. _The file extension is always necessary for these._ * _Bare specifiers_ like `'some-package'` or `'some-package/shuffle'`. They can refer to the main entry point of a package by the package name, or a specific feature module within a package prefixed by the package name as per the examples respectively. _Including the file extension is only necessary for packages without an [`"exports"`][] field._ * _Absolute specifiers_ like `'file:///opt/nodejs/config.js'`. They refer directly and explicitly to a full path. Bare specifier resolutions are handled by the [Node.js module resolution algorithm][]. All other specifier resolutions are always only resolved with the standard relative [URL][] resolution semantics. Like in CommonJS, module files within packages can be accessed by appending a path to the package name unless the packageâ€™s [`package.json`][] contains an [`"exports"`][] field, in which case files within packages can only be accessed via the paths defined in [`"exports"`][]. For details on these package resolution rules that apply to bare specifiers in the Node.js module resolution, see the [packages documentation](packages.md). ### Mandatory file extensions A file extension must be provided when using the `import` keyword to resolve relative or absolute specifiers. Directory indexes (e.g. `'./startup/index.js'`) must also be fully specified. This behavior matches how `import` behaves in browser environments, assuming a typically configured server. ### URLs ES modules are resolved and cached as URLs. This means that special characters must be [percent-encoded][], such as `#` with `%23` and `?` with `%3F`. `file:`, `node:`, and `data:` URL schemes are supported. A specifier like `'https://example.com/app.js'` is not supported natively in Node.js unless using a [custom HTTPS loader][]. #### `file:` URLs Modules are loaded multiple times if the `import` specifier used to resolve them has a different query or fragment. ```js import './foo.mjs?query=1'; // loads ./foo.mjs with query of "?query=1" import './foo.mjs?query=2'; // loads ./foo.mjs with query of "?query=2" ``` The volume root may be referenced via `/`, `//` or `file:///`. Given the differences between [URL][] and path resolution (such as percent encoding details), it is recommended to use [url.pathToFileURL][] when importing a path. #### `data:` Imports  [`data:` URLs][] are supported for importing with the following MIME types: * `text/javascript` for ES Modules * `application/json` for JSON * `application/wasm` for Wasm `data:` URLs only resolve [_Bare specifiers_][Terminology] for builtin modules and [_Absolute specifiers_][Terminology]. Resolving [_Relative specifiers_][Terminology] does not work because `data:` is not a [special scheme][]. For example, attempting to load `./foo` from `data:text/javascript,import "./foo";` fails to resolve because there is no concept of relative resolution for `data:` URLs. An example of a `data:` URLs being used is: ```js import 'data:text/javascript,console.log("hello!");'; import _ from 'data:application/json,"world!"'; ``` #### `node:` Imports  `node:` URLs are supported as an alternative means to load Node.js builtin modules. This URL scheme allows for builtin modules to be referenced by valid absolute URL strings. ```js import fs from 'node:fs/promises'; ``` ## Import assertions  > Stability: 1 - Experimental The [Import Assertions proposal][] adds an inline syntax for module import statements to pass on more information alongside the module specifier. ```js import fooData from './foo.json' assert { type: 'json' }; const { default: barData } = await import('./bar.json', { assert: { type: 'json' } }); ``` Node.js supports the following `type` values, for which the assertion is mandatory: | Assertion `type` | Needed for | | ---------------- | ---------------- | | `'json'` | [JSON modules][] | ## Builtin modules [Core modules][] provide named exports of their public API. A default export is also provided which is the value of the CommonJS exports. The default export can be used for, among other things, modifying the named exports. Named exports of builtin modules are updated only by calling [`module.syncBuiltinESMExports()`][]. ```js import EventEmitter from 'events'; const e = new EventEmitter(); ``` ```js import { readFile } from 'fs'; readFile('./foo.txt', (err, source) => { if (err) { console.error(err); } else { console.log(source); } }); ``` ```js import fs, { readFileSync } from 'fs'; import { syncBuiltinESMExports } from 'module'; import { Buffer } from 'buffer'; fs.readFileSync = () => Buffer.from('Hello, ESM'); syncBuiltinESMExports(); fs.readFileSync === readFileSync; ``` ## `import()` expressions [Dynamic `import()`][] is supported in both CommonJS and ES modules. In CommonJS modules it can be used to load ES modules. ## `import.meta` * {Object} The `import.meta` meta property is an `Object` that contains the following properties. ### `import.meta.url` * {string} The absolute `file:` URL of the module. This is defined exactly the same as it is in browsers providing the URL of the current module file. This enables useful patterns such as relative file loading: ```js import { readFileSync } from 'fs'; const buffer = readFileSync(new URL('./data.proto', import.meta.url)); ``` ### `import.meta.resolve(specifier[, parent])`  > Stability: 1 - Experimental This feature is only available with the `--experimental-import-meta-resolve` command flag enabled. * `specifier` {string} The module specifier to resolve relative to `parent`. * `parent` {string|URL} The absolute parent module URL to resolve from. If none is specified, the value of `import.meta.url` is used as the default. * Returns: {Promise} Provides a module-relative resolution function scoped to each module, returning the URL string.  ```js const dependencyAsset = await import.meta.resolve('component-lib/asset.css'); ``` `import.meta.resolve` also accepts a second argument which is the parent module from which to resolve from:  ```js await import.meta.resolve('./dep', import.meta.url); ``` This function is asynchronous because the ES module resolver in Node.js is allowed to be asynchronous. ## Interoperability with CommonJS ### `import` statements An `import` statement can reference an ES module or a CommonJS module. `import` statements are permitted only in ES modules, but dynamic [`import()`][] expressions are supported in CommonJS for loading ES modules. When importing [CommonJS modules](#commonjs-namespaces), the `module.exports` object is provided as the default export. Named exports may be available, provided by static analysis as a convenience for better ecosystem compatibility. ### `require` The CommonJS module `require` always treats the files it references as CommonJS. Using `require` to load an ES module is not supported because ES modules have asynchronous execution. Instead, use [`import()`][] to load an ES module from a CommonJS module. ### CommonJS Namespaces CommonJS modules consist of a `module.exports` object which can be of any type. When importing a CommonJS module, it can be reliably imported using the ES module default import or its corresponding sugar syntax:  ```js import { default as cjs } from 'cjs'; // The following import statement is "syntax sugar" (equivalent but sweeter) // for `{ default as cjsSugar }` in the above import statement: import cjsSugar from 'cjs'; console.log(cjs); console.log(cjs === cjsSugar); // Prints: // <module.exports> // true ``` The ECMAScript Module Namespace representation of a CommonJS module is always a namespace with a `default` export key pointing to the CommonJS `module.exports` value. This Module Namespace Exotic Object can be directly observed either when using `import * as m from 'cjs'` or a dynamic import:  ```js import * as m from 'cjs'; console.log(m); console.log(m === await import('cjs')); // Prints: // [Module] { default: <module.exports> } // true ``` For better compatibility with existing usage in the JS ecosystem, Node.js in addition attempts to determine the CommonJS named exports of every imported CommonJS module to provide them as separate ES module exports using a static analysis process. For example, consider a CommonJS module written: ```cjs // cjs.cjs exports.name = 'exported'; ``` The preceding module supports named imports in ES modules:  ```js import { name } from './cjs.cjs'; console.log(name); // Prints: 'exported' import cjs from './cjs.cjs'; console.log(cjs); // Prints: { name: 'exported' } import * as m from './cjs.cjs'; console.log(m); // Prints: [Module] { default: { name: 'exported' }, name: 'exported' } ``` As can be seen from the last example of the Module Namespace Exotic Object being logged, the `name` export is copied off of the `module.exports` object and set directly on the ES module namespace when the module is imported. Live binding updates or new exports added to `module.exports` are not detected for these named exports. The detection of named exports is based on common syntax patterns but does not always correctly detect named exports. In these cases, using the default import form described above can be a better option. Named exports detection covers many common export patterns, reexport patterns and build tool and transpiler outputs. See [cjs-module-lexer][] for the exact semantics implemented. ### Differences between ES modules and CommonJS #### No `require`, `exports` or `module.exports` In most cases, the ES module `import` can be used to load CommonJS modules. If needed, a `require` function can be constructed within an ES module using [`module.createRequire()`][]. #### No `__filename` or `__dirname` These CommonJS variables are not available in ES modules. `__filename` and `__dirname` use cases can be replicated via [`import.meta.url`][]. #### No Native Module Loading Native modules are not currently supported with ES module imports. They can instead be loaded with [`module.createRequire()`][] or [`process.dlopen`][]. #### No `require.resolve` Relative resolution can be handled via `new URL('./local', import.meta.url)`. For a complete `require.resolve` replacement, there is a flagged experimental [`import.meta.resolve`][] API. Alternatively `module.createRequire()` can be used. #### No `NODE_PATH` `NODE_PATH` is not part of resolving `import` specifiers. Please use symlinks if this behavior is desired. #### No `require.extensions` `require.extensions` is not used by `import`. The expectation is that loader hooks can provide this workflow in the future. #### No `require.cache` `require.cache` is not used by `import` as the ES module loader has its own separate cache. ## JSON modules > Stability: 1 - Experimental JSON files can be referenced by `import`: ```js import packageConfig from './package.json' assert { type: 'json' }; ``` The `assert { type: 'json' }` syntax is mandatory; see [Import Assertions][]. The imported JSON only exposes a `default` export. There is no support for named exports. A cache entry is created in the CommonJS cache to avoid duplication. The same object is returned in CommonJS if the JSON module has already been imported from the same path. ## Wasm modules > Stability: 1 - Experimental Importing WebAssembly modules is supported under the `--experimental-wasm-modules` flag, allowing any `.wasm` files to be imported as normal modules while also supporting their module imports. This integration is in line with the [ES Module Integration Proposal for WebAssembly][]. For example, an `index.mjs` containing: ```js import * as M from './module.wasm'; console.log(M); ``` executed under: ```bash node --experimental-wasm-modules index.mjs ``` would provide the exports interface for the instantiation of `module.wasm`. ## Top-level `await`  > Stability: 1 - Experimental The `await` keyword may be used in the top level body of an ECMAScript module. Assuming an `a.mjs` with ```js export const five = await Promise.resolve(5); ``` And a `b.mjs` with ```js import { five } from './a.mjs'; console.log(five); // Logs `5` ``` ```bash node b.mjs # works ``` If a top level `await` expression never resolves, the `node` process will exit with a `13` [status code][]. ```js import { spawn } from 'child_process'; import { execPath } from 'process'; spawn(execPath, [ '--input-type=module', '--eval', // Never-resolving Promise: 'await new Promise(() => {})', ]).once('exit', (code) => { console.log(code); // Logs `13` }); ``` ## Loaders > Stability: 1 - Experimental **Note: This API is currently being redesigned and will still change.**  To customize the default module resolution, loader hooks can optionally be provided via a `--experimental-loader ./loader-name.mjs` argument to Node.js. When hooks are used they apply to the entry point and all `import` calls. They won't apply to `require` calls; those still follow [CommonJS][] rules. ### Hooks #### `resolve(specifier, context, defaultResolve)`  > Note: The loaders API is being redesigned. This hook may disappear or its > signature may change. Do not rely on the API described below. * `specifier` {string} * `context` {Object} * `conditions` {string\[]} * `importAssertions` {Object} * `parentURL` {string|undefined} * `defaultResolve` {Function} The Node.js default resolver. * Returns: {Object} * `format` {string|null|undefined} `'builtin' | 'commonjs' | 'json' | 'module' | 'wasm'` * `url` {string} The absolute url to the import target (such as `file://â€¦`) The `resolve` hook returns the resolved file URL for a given module specifier and parent URL, and optionally its format (such as `'module'`) as a hint to the `load` hook. If a format is specified, the `load` hook is ultimately responsible for providing the final `format` value (and it is free to ignore the hint provided by `resolve`); if `resolve` provides a `format`, a custom `load` hook is required even if only to pass the value to the Node.js default `load` hook. The module specifier is the string in an `import` statement or `import()` expression, and the parent URL is the URL of the module that imported this one, or `undefined` if this is the main entry point for the application. The `conditions` property in `context` is an array of conditions for [package exports conditions][Conditional Exports] that apply to this resolution request. They can be used for looking up conditional mappings elsewhere or to modify the list when calling the default resolution logic. The current [package exports conditions][Conditional Exports] are always in the `context.conditions` array passed into the hook. To guarantee _default Node.js module specifier resolution behavior_ when calling `defaultResolve`, the `context.conditions` array passed to it _must_ include _all_ elements of the `context.conditions` array originally passed into the `resolve` hook. ```js /** * @param {string} specifier * @param {{ * conditions: string[], * parentURL: string | undefined, * }} context * @param {Function} defaultResolve * @returns {Promise<{ url: string }>} */ export async function resolve(specifier, context, defaultResolve) { const { parentURL = null } = context; if (Math.random() > 0.5) { // Some condition. // For some or all specifiers, do some custom logic for resolving. // Always return an object of the form {url: <string>}. return { url: parentURL ? new URL(specifier, parentURL).href : new URL(specifier).href, }; } if (Math.random() < 0.5) { // Another condition. // When calling `defaultResolve`, the arguments can be modified. In this // case it's adding another value for matching conditional exports. return defaultResolve(specifier, { ...context, conditions: [...context.conditions, 'another-condition'], }); } // Defer to Node.js for all other specifiers. return defaultResolve(specifier, context, defaultResolve); } ``` #### `load(url, context, defaultLoad)` > Note: The loaders API is being redesigned. This hook may disappear or its > signature may change. Do not rely on the API described below. > Note: In a previous version of this API, this was split across 3 separate, now > deprecated, hooks (`getFormat`, `getSource`, and `transformSource`). * `url` {string} * `context` {Object} * `format` {string|null|undefined} The format optionally supplied by the `resolve` hook. * `importAssertions` {Object} * `defaultLoad` {Function} * Returns: {Object} * `format` {string} * `source` {string|ArrayBuffer|TypedArray} The `load` hook provides a way to define a custom method of determining how a URL should be interpreted, retrieved, and parsed. It is also in charge of validating the import assertion. The final value of `format` must be one of the following: | `format` | Description | Acceptable types for `source` returned by `load` | | ------------ | ------------------------------ | ----------------------------------------------------- | | `'builtin'` | Load a Node.js builtin module | Not applicable | | `'commonjs'` | Load a Node.js CommonJS module | Not applicable | | `'json'` | Load a JSON file | { [`string`][], [`ArrayBuffer`][], [`TypedArray`][] } | | `'module'` | Load an ES module | { [`string`][], [`ArrayBuffer`][], [`TypedArray`][] } | | `'wasm'` | Load a WebAssembly module | { [`ArrayBuffer`][], [`TypedArray`][] } | The value of `source` is ignored for type `'builtin'` because currently it is not possible to replace the value of a Node.js builtin (core) module. The value of `source` is ignored for type `'commonjs'` because the CommonJS module loader does not provide a mechanism for the ES module loader to override the [CommonJS module return value](#commonjs-namespaces). This limitation might be overcome in the future. > **Caveat**: The ESM `load` hook and namespaced exports from CommonJS modules > are incompatible. Attempting to use them together will result in an empty > object from the import. This may be addressed in the future. > Note: These types all correspond to classes defined in ECMAScript. * The specific [`ArrayBuffer`][] object is a [`SharedArrayBuffer`][]. * The specific [`TypedArray`][] object is a [`Uint8Array`][]. If the source value of a text-based format (i.e., `'json'`, `'module'`) is not a string, it is converted to a string using [`util.TextDecoder`][]. The `load` hook provides a way to define a custom method for retrieving the source code of an ES module specifier. This would allow a loader to potentially avoid reading files from disk. It could also be used to map an unrecognized format to a supported one, for example `yaml` to `module`. ```js /** * @param {string} url * @param {{ format: string, }} context If resolve settled with a `format`, that value is included here. * @param {Function} defaultLoad * @returns {Promise<{ format: string, source: string | ArrayBuffer | SharedArrayBuffer | Uint8Array, }>} */ export async function load(url, context, defaultLoad) { const { format } = context; if (Math.random() > 0.5) { // Some condition. /* For some or all URLs, do some custom logic for retrieving the source. Always return an object of the form { format: <string>, source: <string|buffer>, }. */ return { format, source: '...', }; } // Defer to Node.js for all other URLs. return defaultLoad(url, context, defaultLoad); } ``` In a more advanced scenario, this can also be used to transform an unsupported source to a supported one (see [Examples](#examples) below). #### `globalPreload()` > Note: The loaders API is being redesigned. This hook may disappear or its > signature may change. Do not rely on the API described below. > Note: In a previous version of this API, this hook was named > `getGlobalPreloadCode`. * Returns: {string} Sometimes it might be necessary to run some code inside of the same global scope that the application runs in. This hook allows the return of a string that is run as a sloppy-mode script on startup. Similar to how CommonJS wrappers work, the code runs in an implicit function scope. The only argument is a `require`-like function that can be used to load builtins like "fs": `getBuiltin(request: string)`. If the code needs more advanced `require` features, it has to construct its own `require` using `module.createRequire()`. ```js /** * @param {{ port: MessagePort, }} utilities Things that preload code might find useful * @returns {string} Code to run before application startup */ export function globalPreload(utilities) { return `\ globalThis.someInjectedProperty = 42; console.log('I just set some globals!'); const { createRequire } = getBuiltin('module'); const { cwd } = getBuiltin('process'); const require = createRequire(cwd() + '/<preload>'); // [...] `; } ``` In order to allow communication between the application and the loader, another argument is provided to the preload code: `port`. This is available as a parameter to the loader hook and inside of the source text returned by the hook. Some care must be taken in order to properly call [`port.ref()`][] and [`port.unref()`][] to prevent a process from being in a state where it won't close normally. ```js /** * This example has the application context send a message to the loader * and sends the message back to the application context * @param {{ port: MessagePort, }} utilities Things that preload code might find useful * @returns {string} Code to run before application startup */ export function globalPreload({ port }) { port.onmessage = (evt) => { port.postMessage(evt.data); }; return `\ port.postMessage('console.log("I went to the Loader and back");'); port.onmessage = (evt) => { eval(evt.data); }; `; } ``` ### Examples The various loader hooks can be used together to accomplish wide-ranging customizations of Node.jsâ€™ code loading and evaluation behaviors. #### HTTPS loader In current Node.js, specifiers starting with `https://` are unsupported. The loader below registers hooks to enable rudimentary support for such specifiers. While this may seem like a significant improvement to Node.js core functionality, there are substantial downsides to actually using this loader: performance is much slower than loading files from disk, there is no caching, and there is no security. ```js // https-loader.mjs import { get } from 'https'; export function resolve(specifier, context, defaultResolve) { const { parentURL = null } = context; // Normally Node.js would error on specifiers starting with 'https://', so // this hook intercepts them and converts them into absolute URLs to be // passed along to the later hooks below. if (specifier.startsWith('https://')) { return { url: specifier }; } else if (parentURL && parentURL.startsWith('https://')) { return { url: new URL(specifier, parentURL).href }; } // Let Node.js handle all other specifiers. return defaultResolve(specifier, context, defaultResolve); } export function load(url, context, defaultLoad) { // For JavaScript to be loaded over the network, we need to fetch and // return it. if (url.startsWith('https://')) { return new Promise((resolve, reject) => { get(url, (res) => { let data = ''; res.on('data', (chunk) => data += chunk); res.on('end', () => resolve({ // This example assumes all network-provided JavaScript is ES module // code. format: 'module', source: data, })); }).on('error', (err) => reject(err)); }); } // Let Node.js handle all other URLs. return defaultLoad(url, context, defaultLoad); } ``` ```js // main.mjs import { VERSION } from 'https://coffeescript.org/browser-compiler-modern/coffeescript.js'; console.log(VERSION); ``` With the preceding loader, running `node --experimental-loader ./https-loader.mjs ./main.mjs` prints the current version of CoffeeScript per the module at the URL in `main.mjs`. #### Transpiler loader Sources that are in formats Node.js doesnâ€™t understand can be converted into JavaScript using the [`load` hook][load hook]. Before that hook gets called, however, a [`resolve` hook][resolve hook] hook needs to tell Node.js not to throw an error on unknown file types. This is less performant than transpiling source files before running Node.js; a transpiler loader should only be used for development and testing purposes. ```js // coffeescript-loader.mjs import { readFile } from 'node:fs/promises'; import { dirname, extname, resolve as resolvePath } from 'node:path'; import { cwd } from 'node:process'; import { fileURLToPath, pathToFileURL } from 'node:url'; import CoffeeScript from 'coffeescript'; const baseURL = pathToFileURL(`${cwd()}/`).href; // CoffeeScript files end in .coffee, .litcoffee or .coffee.md. const extensionsRegex = /\.coffee$|\.litcoffee$|\.coffee\.md$/; export async function resolve(specifier, context, defaultResolve) { const { parentURL = baseURL } = context; // Node.js normally errors on unknown file extensions, so return a URL for // specifiers ending in the CoffeeScript file extensions. if (extensionsRegex.test(specifier)) { return { url: new URL(specifier, parentURL).href }; } // Let Node.js handle all other specifiers. return defaultResolve(specifier, context, defaultResolve); } export async function load(url, context, defaultLoad) { // Now that we patched resolve to let CoffeeScript URLs through, we need to // tell Node.js what format such URLs should be interpreted as. Because // CoffeeScript transpiles into JavaScript, it should be one of the two // JavaScript formats: 'commonjs' or 'module'. if (extensionsRegex.test(url)) { // CoffeeScript files can be either CommonJS or ES modules, so we want any // CoffeeScript file to be treated by Node.js the same as a .js file at the // same location. To determine how Node.js would interpret an arbitrary .js // file, search up the file system for the nearest parent package.json file // and read its "type" field. const format = await getPackageType(url); // When a hook returns a format of 'commonjs', `source` is be ignored. // To handle CommonJS files, a handler needs to be registered with // `require.extensions` in order to process the files with the CommonJS // loader. Avoiding the need for a separate CommonJS handler is a future // enhancement planned for ES module loaders. if (format === 'commonjs') { return { format }; } const { source: rawSource } = await defaultLoad(url, { format }); // This hook converts CoffeeScript source code into JavaScript source code // for all imported CoffeeScript files. const transformedSource = CoffeeScript.compile(rawSource.toString(), { bare: true, filename: url, }); return { format, source: transformedSource, }; } // Let Node.js handle all other URLs. return defaultLoad(url, context, defaultLoad); } async function getPackageType(url) { // `url` is only a file path during the first iteration when passed the // resolved url from the load() hook // an actual file path from load() will contain a file extension as it's // required by the spec // this simple truthy check for whether `url` contains a file extension will // work for most projects but does not cover some edge-cases (such as // extension-less files or a url ending in a trailing space) const isFilePath = !!extname(url); // If it is a file path, get the directory it's in const dir = isFilePath ? dirname(fileURLToPath(url)) : url; // Compose a file path to a package.json in the same directory, // which may or may not exist const packagePath = resolvePath(dir, 'package.json'); // Try to read the possibly nonexistent package.json const type = await readFile(packagePath, { encoding: 'utf8' }) .then((filestring) => JSON.parse(filestring).type) .catch((err) => { if (err?.code !== 'ENOENT') console.error(err); }); // Ff package.json existed and contained a `type` field with a value, voila if (type) return type; // Otherwise, (if not at the root) continue checking the next directory up // If at the root, stop and return false return dir.length > 1 && getPackageType(resolvePath(dir, '..')); } ``` ```coffee # main.coffee import { scream } from './scream.coffee' console.log scream 'hello, world' import { version } from 'process' console.log "Brought to you by Node.js version #{version}" ``` ```coffee # scream.coffee export scream = (str) -> str.toUpperCase() ``` With the preceding loader, running `node --experimental-loader ./coffeescript-loader.mjs main.coffee` causes `main.coffee` to be turned into JavaScript after its source code is loaded from disk but before Node.js executes it; and so on for any `.coffee`, `.litcoffee` or `.coffee.md` files referenced via `import` statements of any loaded file. ## Resolution algorithm ### Features The resolver has the following properties: * FileURL-based resolution as is used by ES modules * Support for builtin module loading * Relative and absolute URL resolution * No default extensions * No folder mains * Bare specifier package resolution lookup through node\_modules ### Resolver algorithm The algorithm to load an ES module specifier is given through the **ESM\_RESOLVE** method below. It returns the resolved URL for a module specifier relative to a parentURL. The algorithm to determine the module format of a resolved URL is provided by **ESM\_FORMAT**, which returns the unique module format for any file. The _"module"_ format is returned for an ECMAScript Module, while the _"commonjs"_ format is used to indicate loading through the legacy CommonJS loader. Additional formats such as _"addon"_ can be extended in future updates. In the following algorithms, all subroutine errors are propagated as errors of these top-level routines unless stated otherwise. _defaultConditions_ is the conditional environment name array, `["node", "import"]`. The resolver can throw the following errors: * _Invalid Module Specifier_: Module specifier is an invalid URL, package name or package subpath specifier. * _Invalid Package Configuration_: package.json configuration is invalid or contains an invalid configuration. * _Invalid Package Target_: Package exports or imports define a target module for the package that is an invalid type or string target. * _Package Path Not Exported_: Package exports do not define or permit a target subpath in the package for the given module. * _Package Import Not Defined_: Package imports do not define the specifier. * _Module Not Found_: The package or module requested does not exist. * _Unsupported Directory Import_: The resolved path corresponds to a directory, which is not a supported target for module imports. ### Resolver Algorithm Specification **ESM\_RESOLVE**(_specifier_, _parentURL_) > 1. Let _resolved_ be **undefined**. > 2. If _specifier_ is a valid URL, then > 1. Set _resolved_ to the result of parsing and reserializing > _specifier_ as a URL. > 3. Otherwise, if _specifier_ starts with _"/"_, _"./"_ or _"../"_, then > 1. Set _resolved_ to the URL resolution of _specifier_ relative to > _parentURL_. > 4. Otherwise, if _specifier_ starts with _"#"_, then > 1. Set _resolved_ to the result of > **PACKAGE\_IMPORTS\_RESOLVE**(_specifier_, > _parentURL_, _defaultConditions_). > 5. Otherwise, > 1. Note: _specifier_ is now a bare specifier. > 2. Set _resolved_ the result of > **PACKAGE\_RESOLVE**(_specifier_, _parentURL_). > 6. Let _format_ be **undefined**. > 7. If _resolved_ is a _"file:"_ URL, then > 1. If _resolved_ contains any percent encodings of _"/"_ or _"\\"_ (_"%2F"_ > and _"%5C"_ respectively), then > 1. Throw an _Invalid Module Specifier_ error. > 2. If the file at _resolved_ is a directory, then > 1. Throw an _Unsupported Directory Import_ error. > 3. If the file at _resolved_ does not exist, then > 1. Throw a _Module Not Found_ error. > 4. Set _resolved_ to the real path of _resolved_, maintaining the > same URL querystring and fragment components. > 5. Set _format_ to the result of **ESM\_FILE\_FORMAT**(_resolved_). > 8. Otherwise, > 1. Set _format_ the module format of the content type associated with the > URL _resolved_. > 9. Load _resolved_ as module format, _format_. **PACKAGE\_RESOLVE**(_packageSpecifier_, _parentURL_) > 1. Let _packageName_ be **undefined**. > 2. If _packageSpecifier_ is an empty string, then > 1. Throw an _Invalid Module Specifier_ error. > 3. If _packageSpecifier_ is a Node.js builtin module name, then > 1. Return the string _"node:"_ concatenated with _packageSpecifier_. > 4. If _packageSpecifier_ does not start with _"@"_, then > 1. Set _packageName_ to the substring of _packageSpecifier_ until the first > _"/"_ separator or the end of the string. > 5. Otherwise, > 1. If _packageSpecifier_ does not contain a _"/"_ separator, then > 1. Throw an _Invalid Module Specifier_ error. > 2. Set _packageName_ to the substring of _packageSpecifier_ > until the second _"/"_ separator or the end of the string. > 6. If _packageName_ starts with _"."_ or contains _"\\"_ or _"%"_, then > 1. Throw an _Invalid Module Specifier_ error. > 7. Let _packageSubpath_ be _"."_ concatenated with the substring of > _packageSpecifier_ from the position at the length of _packageName_. > 8. If _packageSubpath_ ends in _"/"_, then > 1. Throw an _Invalid Module Specifier_ error. > 9. Let _selfUrl_ be the result of > **PACKAGE\_SELF\_RESOLVE**(_packageName_, _packageSubpath_, _parentURL_). > 10. If _selfUrl_ is not **undefined**, return _selfUrl_. > 11. While _parentURL_ is not the file system root, > 1. Let _packageURL_ be the URL resolution of _"node\_modules/"_ > concatenated with _packageSpecifier_, relative to _parentURL_. > 2. Set _parentURL_ to the parent folder URL of _parentURL_. > 3. If the folder at _packageURL_ does not exist, then > 1. Continue the next loop iteration. > 4. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_). > 5. If _pjson_ is not **null** and _pjson_._exports_ is not **null** or > **undefined**, then > 1. Return the result of **PACKAGE\_EXPORTS\_RESOLVE**(_packageURL_, > _packageSubpath_, _pjson.exports_, _defaultConditions_). > 6. Otherwise, if _packageSubpath_ is equal to _"."_, then > 1. If _pjson.main_ is a string, then > 1. Return the URL resolution of _main_ in _packageURL_. > 7. Otherwise, > 1. Return the URL resolution of _packageSubpath_ in _packageURL_. > 12. Throw a _Module Not Found_ error. **PACKAGE\_SELF\_RESOLVE**(_packageName_, _packageSubpath_, _parentURL_) > 1. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_parentURL_). > 2. If _packageURL_ is **null**, then > 1. Return **undefined**. > 3. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_). > 4. If _pjson_ is **null** or if _pjson_._exports_ is **null** or > **undefined**, then > 1. Return **undefined**. > 5. If _pjson.name_ is equal to _packageName_, then > 1. Return the result of **PACKAGE\_EXPORTS\_RESOLVE**(_packageURL_, > _packageSubpath_, _pjson.exports_, _defaultConditions_). > 6. Otherwise, return **undefined**. **PACKAGE\_EXPORTS\_RESOLVE**(_packageURL_, _subpath_, _exports_, _conditions_) > 1. If _exports_ is an Object with both a key starting with _"."_ and a key not > starting with _"."_, throw an _Invalid Package Configuration_ error. > 2. If _subpath_ is equal to _"."_, then > 1. Let _mainExport_ be **undefined**. > 2. If _exports_ is a String or Array, or an Object containing no keys > starting with _"."_, then > 1. Set _mainExport_ to _exports_. > 3. Otherwise if _exports_ is an Object containing a _"."_ property, then > 1. Set _mainExport_ to _exports_\[_"."_]. > 4. If _mainExport_ is not **undefined**, then > 1. Let _resolved_ be the result of **PACKAGE\_TARGET\_RESOLVE**( > _packageURL_, _mainExport_, _""_, **false**, **false**, > _conditions_). > 2. If _resolved_ is not **null** or **undefined**, return _resolved_. > 3. Otherwise, if _exports_ is an Object and all keys of _exports_ start with > _"."_, then > 1. Let _matchKey_ be the string _"./"_ concatenated with _subpath_. > 2. Let _resolved_ be the result of **PACKAGE\_IMPORTS\_EXPORTS\_RESOLVE**( > _matchKey_, _exports_, _packageURL_, **false**, _conditions_). > 3. If _resolved_ is not **null** or **undefined**, return _resolved_. > 4. Throw a _Package Path Not Exported_ error. **PACKAGE\_IMPORTS\_RESOLVE**(_specifier_, _parentURL_, _conditions_) > 1. Assert: _specifier_ begins with _"#"_. > 2. If _specifier_ is exactly equal to _"#"_ or starts with _"#/"_, then > 1. Throw an _Invalid Module Specifier_ error. > 3. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_parentURL_). > 4. If _packageURL_ is not **null**, then > 1. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_). > 2. If _pjson.imports_ is a non-null Object, then > 1. Let _resolved_ be the result of > **PACKAGE\_IMPORTS\_EXPORTS\_RESOLVE**( > _specifier_, _pjson.imports_, _packageURL_, **true**, _conditions_). > 2. If _resolved_ is not **null** or **undefined**, return _resolved_. > 5. Throw a _Package Import Not Defined_ error. **PACKAGE\_IMPORTS\_EXPORTS\_RESOLVE**(_matchKey_, _matchObj_, _packageURL_, _isImports_, _conditions_) > 1. If _matchKey_ is a key of _matchObj_ and does not contain _"\*"_, then > 1. Let _target_ be the value of _matchObj_\[_matchKey_]. > 2. Return the result of **PACKAGE\_TARGET\_RESOLVE**(_packageURL_, > _target_, _""_, **false**, _isImports_, _conditions_). > 2. Let _expansionKeys_ be the list of keys of _matchObj_ containing only a > single _"\*"_, sorted by the sorting function **PATTERN\_KEY\_COMPARE** > which orders in descending order of specificity. > 3. For each key _expansionKey_ in _expansionKeys_, do > 1. Let _patternBase_ be the substring of _expansionKey_ up to but excluding > the first _"\*"_ character. > 2. If _matchKey_ starts with but is not equal to _patternBase_, then > 1. Let _patternTrailer_ be the substring of _expansionKey_ from the > index after the first _"\*"_ character. > 2. If _patternTrailer_ has zero length, or if _matchKey_ ends with > _patternTrailer_ and the length of _matchKey_ is greater than or > equal to the length of _expansionKey_, then > 1. Let _target_ be the value of _matchObj_\[_expansionKey_]. > 2. Let _subpath_ be the substring of _matchKey_ starting at the > index of the length of _patternBase_ up to the length of > _matchKey_ minus the length of _patternTrailer_. > 3. Return the result of **PACKAGE\_TARGET\_RESOLVE**(_packageURL_, > _target_, _subpath_, **true**, _isImports_, _conditions_). > 4. Return **null**. **PATTERN\_KEY\_COMPARE**(_keyA_, _keyB_) > 1. Assert: _keyA_ ends with _"/"_ or contains only a single _"\*"_. > 2. Assert: _keyB_ ends with _"/"_ or contains only a single _"\*"_. > 3. Let _baseLengthA_ be the index of _"\*"_ in _keyA_ plus one, if _keyA_ > contains _"\*"_, or the length of _keyA_ otherwise. > 4. Let _baseLengthB_ be the index of _"\*"_ in _keyB_ plus one, if _keyB_ > contains _"\*"_, or the length of _keyB_ otherwise. > 5. If _baseLengthA_ is greater than _baseLengthB_, return -1. > 6. If _baseLengthB_ is greater than _baseLengthA_, return 1. > 7. If _keyA_ does not contain _"\*"_, return 1. > 8. If _keyB_ does not contain _"\*"_, return -1. > 9. If the length of _keyA_ is greater than the length of _keyB_, return -1. > 10. If the length of _keyB_ is greater than the length of _keyA_, return 1. > 11. Return 0. **PACKAGE\_TARGET\_RESOLVE**(_packageURL_, _target_, _subpath_, _pattern_, _internal_, _conditions_) > 1. If _target_ is a String, then > 1. If _pattern_ is **false**, _subpath_ has non-zero length and _target_ > does not end with _"/"_, throw an _Invalid Module Specifier_ error. > 2. If _target_ does not start with _"./"_, then > 1. If _internal_ is **true** and _target_ does not start with _"../"_ or > _"/"_ and is not a valid URL, then > 1. If _pattern_ is **true**, then > 1. Return **PACKAGE\_RESOLVE**(_target_ with every instance of > _"\*"_ replaced by _subpath_, _packageURL_ + _"/"_). > 2. Return **PACKAGE\_RESOLVE**(_target_ + _subpath_, > _packageURL_ + _"/"_). > 2. Otherwise, throw an _Invalid Package Target_ error. > 3. If _target_ split on _"/"_ or _"\\"_ contains any _"."_, _".."_ or > _"node\_modules"_ segments after the first segment, case insensitive and > including percent encoded variants, throw an _Invalid Package Target_ > error. > 4. Let _resolvedTarget_ be the URL resolution of the concatenation of > _packageURL_ and _target_. > 5. Assert: _resolvedTarget_ is contained in _packageURL_. > 6. If _subpath_ split on _"/"_ or _"\\"_ contains any _"."_, _".."_ or > _"node\_modules"_ segments, case insensitive and including percent > encoded variants, throw an _Invalid Module Specifier_ error. > 7. If _pattern_ is **true**, then > 1. Return the URL resolution of _resolvedTarget_ with every instance of > _"\*"_ replaced with _subpath_. > 8. Otherwise, > 1. Return the URL resolution of the concatenation of _subpath_ and > _resolvedTarget_. > 2. Otherwise, if _target_ is a non-null Object, then > 1. If _exports_ contains any index property keys, as defined in ECMA-262 > [6.1.7 Array Index][], throw an _Invalid Package Configuration_ error. > 2. For each property _p_ of _target_, in object insertion order as, > 1. If _p_ equals _"default"_ or _conditions_ contains an entry for _p_, > then > 1. Let _targetValue_ be the value of the _p_ property in _target_. > 2. Let _resolved_ be the result of **PACKAGE\_TARGET\_RESOLVE**( > _packageURL_, _targetValue_, _subpath_, _pattern_, _internal_, > _conditions_). > 3. If _resolved_ is equal to **undefined**, continue the loop. > 4. Return _resolved_. > 3. Return **undefined**. > 3. Otherwise, if _target_ is an Array, then > 1. If \_target.length is zero, return **null**. > 2. For each item _targetValue_ in _target_, do > 1. Let _resolved_ be the result of **PACKAGE\_TARGET\_RESOLVE**( > _packageURL_, _targetValue_, _subpath_, _pattern_, _internal_, > _conditions_), continuing the loop on any _Invalid Package Target_ > error. > 2. If _resolved_ is **undefined**, continue the loop. > 3. Return _resolved_. > 3. Return or throw the last fallback resolution **null** return or error. > 4. Otherwise, if _target_ is _null_, return **null**. > 5. Otherwise throw an _Invalid Package Target_ error. **ESM\_FILE\_FORMAT**(_url_) > 1. Assert: _url_ corresponds to an existing file. > 2. If _url_ ends in _".mjs"_, then > 1. Return _"module"_. > 3. If _url_ ends in _".cjs"_, then > 1. Return _"commonjs"_. > 4. If _url_ ends in _".json"_, then > 1. Return _"json"_. > 5. Let _packageURL_ be the result of **LOOKUP\_PACKAGE\_SCOPE**(_url_). > 6. Let _pjson_ be the result of **READ\_PACKAGE\_JSON**(_packageURL_). > 7. If _pjson?.type_ exists and is _"module"_, then > 1. If _url_ ends in _".js"_, then > 1. Return _"module"_. > 2. Throw an _Unsupported File Extension_ error. > 8. Otherwise, > 1. Throw an _Unsupported File Extension_ error. **LOOKUP\_PACKAGE\_SCOPE**(_url_) > 1. Let _scopeURL_ be _url_. > 2. While _scopeURL_ is not the file system root, > 1. Set _scopeURL_ to the parent URL of _scopeURL_. > 2. If _scopeURL_ ends in a _"node\_modules"_ path segment, return **null**. > 3. Let _pjsonURL_ be the resolution of _"package.json"_ within > _scopeURL_. > 4. if the file at _pjsonURL_ exists, then > 1. Return _scopeURL_. > 3. Return **null**. **READ\_PACKAGE\_JSON**(_packageURL_) > 1. Let _pjsonURL_ be the resolution of _"package.json"_ within _packageURL_. > 2. If the file at _pjsonURL_ does not exist, then > 1. Return **null**. > 3. If the file at _packageURL_ does not parse as valid JSON, then > 1. Throw an _Invalid Package Configuration_ error. > 4. Return the parsed JSON source of the file at _pjsonURL_. ### Customizing ESM specifier resolution algorithm > Stability: 1 - Experimental The current specifier resolution does not support all default behavior of the CommonJS loader. One of the behavior differences is automatic resolution of file extensions and the ability to import directories that have an index file. The `--experimental-specifier-resolution=[mode]` flag can be used to customize the extension resolution algorithm. The default mode is `explicit`, which requires the full path to a module be provided to the loader. To enable the automatic extension resolution and importing from directories that include an index file use the `node` mode. ```console $ node index.mjs success! $ node index # Failure! Error: Cannot find module $ node --experimental-specifier-resolution=node index success! ```  [6.1.7 Array Index]: https://tc39.es/ecma262/#integer-index [CommonJS]: modules.md [Conditional exports]: packages.md#conditional-exports [Core modules]: modules.md#core-modules [Determining module system]: packages.md#determining-module-system [Dynamic `import()`]: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports [ES Module Integration Proposal for WebAssembly]: https://github.com/webassembly/esm-integration [Import Assertions]: #import-assertions [Import Assertions proposal]: https://github.com/tc39/proposal-import-assertions [JSON modules]: #json-modules [Node.js Module Resolution Algorithm]: #resolver-algorithm-specification [Terminology]: #terminology [URL]: https://url.spec.whatwg.org/ [`"exports"`]: packages.md#exports [`"type"`]: packages.md#type [`--input-type`]: cli.md#--input-typetype [`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer [`SharedArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer [`TypedArray`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray [`Uint8Array`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array [`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs [`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export [`import()`]: #import-expressions [`import.meta.resolve`]: #importmetaresolvespecifier-parent [`import.meta.url`]: #importmetaurl [`import`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import [`module.createRequire()`]: module.md#modulecreaterequirefilename [`module.syncBuiltinESMExports()`]: module.md#modulesyncbuiltinesmexports [`package.json`]: packages.md#nodejs-packagejson-field-definitions [`port.ref()`]: https://nodejs.org/dist/latest-v17.x/docs/api/worker_threads.html#portref [`port.unref()`]: https://nodejs.org/dist/latest-v17.x/docs/api/worker_threads.html#portunref [`process.dlopen`]: process.md#processdlopenmodule-filename-flags [`string`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String [`util.TextDecoder`]: util.md#class-utiltextdecoder [cjs-module-lexer]: https://github.com/nodejs/cjs-module-lexer/tree/1.2.2 [custom https loader]: #https-loader [load hook]: #loadurl-context-defaultload [percent-encoded]: url.md#percent-encoding-in-urls [resolve hook]: #resolvespecifier-context-defaultresolve [special scheme]: https://url.spec.whatwg.org/#special-scheme [status code]: process.md#exit-codes [the official standard format]: https://tc39.github.io/ecma262/#sec-modules [url.pathToFileURL]: url.md#urlpathtofileurlpath

GeoffreyBooth added the esm Issues and PRs related to the ECMAScript Modules implementation. label Feb 21, 2021

devsnek added the experimental Issues and PRs related to experimental features. label Feb 22, 2021

This comment has been minimized.

Sign in to view

DerekNonGeneric self-requested a review March 26, 2021 05:00

guybedford force-pushed the master branch from dc5a5da to 8e46568 Compare March 29, 2021 21:33

This was referenced Oct 21, 2021

feat: add support for node >=16.12.0 antfu/esbuild-node-loader#19

Merged

chore: bump esbuild-node-loader teknologi-umum/graphene#68

Closed

codebytere added a commit to electron/electron that referenced this pull request Oct 25, 2021

esm: consolidate ESM Loader methods

cc6b2e1

nodejs/node#37468

gary-kim-bot mentioned this pull request Oct 26, 2021

chore(deps): update node.js to v16 gary-kim/nc-build-script#73

Closed

1 task

aloerina01 mentioned this pull request Nov 1, 2021

Update Node.js to v16 aloerina01/aloerina01.github.io#57

Merged

1 task

Ayase-252 mentioned this pull request Dec 16, 2021

lib: fix checking syntax of esm module #41198

Merged

JakobJingleheimer mentioned this pull request Dec 22, 2021

loader: fix package resolution for edge case #41218

Merged

frandiox mentioned this pull request Dec 26, 2021

Invalid module - code ERR_INVALID_MODULE_SPECIFIER frandiox/vitedge#74

Closed

aduh95 mentioned this pull request Apr 18, 2022

doc: fix version history for Loaders API #42778

Merged

JakobJingleheimer mentioned this pull request Jul 18, 2022

esm: add chaining to loaders #42623

Merged

10 tasks

JakobJingleheimer mentioned this pull request Mar 8, 2023

Move ESM loaders off-thread #44710

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

esm: consolidate ESM Loader methods #37468

esm: consolidate ESM Loader methods #37468

JakobJingleheimer commented Feb 21, 2021 •

edited

Loading

JakobJingleheimer commented Feb 21, 2021

JakobJingleheimer commented Feb 21, 2021

GeoffreyBooth commented Feb 21, 2021

JakobJingleheimer commented Feb 22, 2021

devsnek commented Feb 22, 2021 •

edited

Loading

devsnek commented Feb 22, 2021

JakobJingleheimer commented Feb 22, 2021

GeoffreyBooth commented Feb 22, 2021 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

GeoffreyBooth commented Feb 22, 2021

GeoffreyBooth commented Feb 22, 2021

JakobJingleheimer commented Feb 22, 2021 •

edited

Loading

JakobJingleheimer commented Mar 6, 2021 •

edited

Loading

This comment has been minimized.

GeoffreyBooth commented Mar 8, 2021

guybedford commented Mar 8, 2021

JakobJingleheimer commented Mar 8, 2021 •

edited

Loading

GeoffreyBooth commented Mar 13, 2021

JakobJingleheimer commented Mar 13, 2021

guybedford commented Mar 13, 2021

JakobJingleheimer commented Mar 26, 2021

GeoffreyBooth commented Mar 26, 2021

DerekNonGeneric commented Mar 26, 2021 •

edited

Loading

JakobJingleheimer commented Mar 29, 2021 •

edited

Loading

DerekNonGeneric commented Mar 29, 2021

esm: consolidate ESM Loader methods #37468

esm: consolidate ESM Loader methods #37468

Conversation

JakobJingleheimer commented Feb 21, 2021 • edited Loading

JakobJingleheimer commented Feb 21, 2021

JakobJingleheimer commented Feb 21, 2021

GeoffreyBooth commented Feb 21, 2021

JakobJingleheimer commented Feb 22, 2021

devsnek commented Feb 22, 2021 • edited Loading

devsnek commented Feb 22, 2021

JakobJingleheimer commented Feb 22, 2021

GeoffreyBooth commented Feb 22, 2021 • edited Loading

This comment has been minimized.

This comment has been minimized.

GeoffreyBooth commented Feb 22, 2021

GeoffreyBooth commented Feb 22, 2021

JakobJingleheimer commented Feb 22, 2021 • edited Loading

JakobJingleheimer commented Mar 6, 2021 • edited Loading

This comment has been minimized.

GeoffreyBooth commented Mar 8, 2021

guybedford commented Mar 8, 2021

JakobJingleheimer commented Mar 8, 2021 • edited Loading

GeoffreyBooth commented Mar 13, 2021

JakobJingleheimer commented Mar 13, 2021

guybedford commented Mar 13, 2021

JakobJingleheimer commented Mar 26, 2021

GeoffreyBooth commented Mar 26, 2021

DerekNonGeneric commented Mar 26, 2021 • edited Loading

JakobJingleheimer commented Mar 29, 2021 • edited Loading

DerekNonGeneric commented Mar 29, 2021

JakobJingleheimer commented Feb 21, 2021 •

edited

Loading

devsnek commented Feb 22, 2021 •

edited

Loading

GeoffreyBooth commented Feb 22, 2021 •

edited

Loading

JakobJingleheimer commented Feb 22, 2021 •

edited

Loading

JakobJingleheimer commented Mar 6, 2021 •

edited

Loading

JakobJingleheimer commented Mar 8, 2021 •

edited

Loading

DerekNonGeneric commented Mar 26, 2021 •

edited

Loading

JakobJingleheimer commented Mar 29, 2021 •

edited

Loading