Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

doc: clarify module system selection #41383

Merged
merged 14 commits into from
Jan 17, 2022
Merged

doc: clarify module system selection #41383

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
1b1159a
doc: clarify module system selection
aduh95 Jan 2, 2022
dd7140f
Address review comments
aduh95 Jan 5, 2022
97515fd
fixup!
aduh95 Jan 7, 2022
c19d161
fixup!
aduh95 Jan 7, 2022
ce3edd0
fixup!
aduh95 Jan 8, 2022
be6f791
fixup!
aduh95 Jan 10, 2022
e9c0b15
fixup!
aduh95 Jan 11, 2022
4c697b7
Apply suggestions from code review
aduh95 Jan 13, 2022
2cea6a8
fixup!
aduh95 Jan 13, 2022
21b45bf
fixup!
aduh95 Jan 13, 2022
93d6a23
fixup!
aduh95 Jan 14, 2022
fe05b72
fixup!
aduh95 Jan 14, 2022
5de6698
fixup!
aduh95 Jan 16, 2022
b08ec87
fixup!
aduh95 Jan 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions doc/api/cli.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Command-line options
# Command-line API

<!--introduced_in=v5.9.1-->

Expand All @@ -11,16 +11,40 @@ To view this documentation as a manual page in a terminal, run `man node`.

## Synopsis

`node [options] [V8 options] [script.js | -e "script" | -] [--] [arguments]`
`node [options] [V8 options] [<program-entry-point> | -e "script" | -] [--] [arguments]`

`node inspect [script.js | -e "script" | <host>:<port>] …`
`node inspect [<program-entry-point> | -e "script" | <host>:<port>] …`

`node --v8-options`

Execute without arguments to start the [REPL][].

For more info about `node inspect`, see the [debugger][] documentation.

## Program entry point

The program entry point is a specifier-like string. That string is first passed
through `path.resolve()` and the [CommonJS][] modules loader. If no
corresponding module, an error is thrown.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
corresponding module, an error is thrown.
corresponding module is found, an error is thrown.

A user so new to Node that they’re reading this page is highly unlikely to know what the CommonJS modules loader is, or even path.resolve. Is there a way we can describe what it means for the string to be parsed by those things, that doesn’t assume any prior Node knowledge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm all for simplifying what can be simplified, but I also strongly think docs should not omit details on the only basis that new users won't be familiar. I think it's fair to assume no one that has participated in the discussion in this PR is new to Node.js (euphemism), yet folks have expressed surprise at node behavior regarding entry point specifier: to me, that indicates that it should be documented.

Maybe instead we should try to define what is the CommonJS module loader and link to that?

Regarding path.resolve, should we replace that with if it's not an absolute path, it's resolved as a relative path from the current working directory?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re path.resolve, sure, that sounds good.

Re the loaders, if you want to mention them then I think you need to introduce them. Something like:

Node.js has two module systems: CommonJS and ECMAScript (ES) Modules. A set of rules define whether a particular specifier (the string passed to require or import) should be interpreted as a CommonJS or ES module. Once this determination is made, either the CommonJS or ES module loader Node.js subsystem then resolves the specifier into a file path or URL, and loads the source for that resource.

The last sentence is the key: I’m defining what either of these loaders is by describing what it does. (What it “is” is a Node.js subsystem, but that’s not very informative.)

And within this brief overview, you can link to the detailed sections describing each loader.

Copy link
Contributor Author

@aduh95 aduh95 Jan 7, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A set of rules define whether a particular specifier (the string passed to require or import) should be interpreted as a CommonJS or ES module. Once this determination is made, either the CommonJS or ES module loader Node.js subsystem then resolves the specifier into a file path or URL, and loads the source for that resource.

A specifier passed to import is always resolved and loaded by the ES module loader (which may in turn pass the resolved URL to the CJS loader), and a specifier passed to require is always resolved and loaded by the CJS loader. I must be missing something, I don't understand what you mean in this last sentence...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the user's perspective, saying that import is always handled by the ESM loader implies that the reference would always be treated as ESM. What matters is that Node can treat some imports as CommonJS and some as ESM. As in, what Node is doing is more important than what Node subsystem is doing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That will depend if the user is a loader author (or maybe even a loader user) imho. If loaders were not public, I would agree that's just an implementation detail that would not belong in the docs. Unless we want to make separate docs for loaders, I think we should have this information somewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can say that one loader or the other is responsible, that's fine. My point is to go further and explain what it means that a particular loader handles something: what does it do as a result?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to address that in ce3edd0, PTAL.


If a module is found, its path will be passed to ECMAScript Module loader if:
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

* The file has a `.mjs` extension,
* Or the file nearest parent `package.json` file
contains a top-level [`"type"`][] field with a value of `"module"`,
* Or if the program was started with a command-line flag that forces the entry
point to be loaded with ECMAScript Module loader.
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

Otherwise the file is loaded using the CommonJS modules loader. See
[File modules][] section for more details.
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

### ECMAScript modules loader entry point caveat

When loading the program entry point using [ECMAScript Module loader][], the `node`
command will accept as input only files with `.js`, `.mjs`, or `.cjs`
extensions; and with `.wasm` extensions when
[`--experimental-wasm-modules`][] is enabled.
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

## Options

<!-- YAML
Expand Down Expand Up @@ -1928,15 +1952,19 @@ $ node --max-old-space-size=1536 index.js
```

[Chrome DevTools Protocol]: https://chromedevtools.github.io/devtools-protocol/
[CommonJS]: modules.md
[ECMAScript Module loader]: esm.md#loaders
[File modules]: modules.md#file-modules
[OSSL_PROVIDER-legacy]: https://www.openssl.org/docs/man3.0/man7/OSSL_PROVIDER-legacy.html
[REPL]: repl.md
[ScriptCoverage]: https://chromedevtools.github.io/devtools-protocol/tot/Profiler#type-ScriptCoverage
[Source Map]: https://sourcemaps.info/spec.html
[Subresource Integrity]: https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity
[V8 JavaScript code coverage]: https://v8project.blogspot.com/2017/12/javascript-code-coverage.html
[`"type"`]: packages.md#type
[`--cpu-prof-dir`]: #--cpu-prof-dir
[`--diagnostic-dir`]: #--diagnostic-dirdirectory
[`--experimental-wasm-modules`]: #--experimental-wasm-modules
[`--heap-prof-dir`]: #--heap-prof-dir
[`--openssl-config`]: #--openssl-configfile
[`--redirect-warnings`]: #--redirect-warningsfile
Expand Down
12 changes: 7 additions & 5 deletions doc/api/esm.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ provides interoperability between them and its original module format,

<!-- type=misc -->

Node.js treats JavaScript code as CommonJS modules by default.
Authors can tell Node.js to treat JavaScript code as ECMAScript modules
Node.js has two module systems: [CommonJS][] modules and ECMAScript modules.

Authors can tell Node.js to use the ECMAScript modules loader
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Authors can tell Node.js to use the ECMAScript modules loader
Authors can tell Node.js to treat JavaScript code as ECMAScript modules

I just don’t think the docs should have any references to “the CommonJS modules loader” or the “ECMAScript modules loader”—no average user knows what those are. There’s just Node, and how it interprets source code.

Copy link
Contributor Author

@aduh95 aduh95 Jan 7, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK but using the ES module loader has more effect than on simply JS code (it no longer accepts .node and .json file as entry point, it no longer treats extensionless / unknown extension as JS files), which was the nuance I was trying to communicate here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add those other implications here, then. Most people reading these docs wouldn't know about those nuances just because the loader is mentioned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to address that in ce3edd0, PTAL.

via the `.mjs` file extension, the `package.json` [`"type"`][] field, or the
aduh95 marked this conversation as resolved.
Show resolved Hide resolved
`--input-type` flag. See
[Modules: Packages](packages.md#determining-module-system) for more
details.
[`--input-type`][] flag. Outside of those cases, Node.js will use the CommonJS
module loader. See [Determining module system][] for more details.
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

<!-- Anchors to make sure old links find a target -->

Expand Down Expand Up @@ -1425,6 +1425,7 @@ success!
[CommonJS]: modules.md
[Conditional exports]: packages.md#conditional-exports
[Core modules]: modules.md#core-modules
[Determining module system]: packages.md#determining-module-system
[Dynamic `import()`]: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports
[ECMAScript Top-Level `await` proposal]: https://github.com/tc39/proposal-top-level-await/
[ES Module Integration Proposal for WebAssembly]: https://github.com/webassembly/esm-integration
Expand All @@ -1437,6 +1438,7 @@ success!
[WHATWG JSON modules specification]: https://html.spec.whatwg.org/#creating-a-json-module-script
[`"exports"`]: packages.md#exports
[`"type"`]: packages.md#type
[`--input-type`]: cli.md#--input-typetype
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[`SharedArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer
[`TypedArray`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray
Expand Down
31 changes: 30 additions & 1 deletion doc/api/modules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,34 @@ module.exports = class Square {
};
```

The module system is implemented in the `require('module')` module.
The CommonJS module system is implemented in the [`module` core module][].

## Enabling

<!-- type=misc -->

Node.js has two module systems: CommonJS modules and [ECMAScript modules][].

By default, Node.js will treat the following as CommonJS modules:

* Files with a `.cjs` extension;

* Files with a `.js` extension when the nearest parent `package.json` file
contains a top-level field [`"type"`][] with a value of `"commonjs"`.

* Files with a `.js` extension when the nearest parent `package.json` file
doesn't contain a top-level field [`"type"`][].

* Files with an extension that is not `.mjs`, `.cjs`, `.json`, `.node`, or `.js`
(when the nearest parent `package.json` file contains a top-level field
[`"type"`][] with a value of `"module"`, those files will be recognized as
CommonJS modules only if they are being `require`d, not when used as the
command-line entry point of the program).

See [Determining module system][] for more details.

Calling `require()` always use the CommonJS loader, calling `import()` always
use the [ECMAScript modules][] loader.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Calling `require()` always use the CommonJS loader, calling `import()` always
use the [ECMAScript modules][] loader.
Calls to `require()` will always try to load the referenced resource as a
CommonJS module. `import()` expressions will load _either_ an
ES module or a CommonJS module, depending on how
the file being imported should be interpreted.
`import()` can be used to reference ES modules from
CommonJS code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think your suggestion is correct, require('./file.json') will first try to load a JSON file before trying to load it as CJS.
I mean, I get that using jargon and expect the reader to know what we mean by "loader" is not a low-bar, but this is a technical documentation, I'd rather use the technical terms than write a potentially mis-leading or incomplete documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If require is loading json, it's loading it as a CommonJS module. Does a CommonJS module need to be JavaScript?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, that was my understanding but maybe it's wrong and a JSON file can count as a CommonJS module. The lack of official spec makes the definition blurry so it's probably not worth debating it.


## Accessing the main module

Expand Down Expand Up @@ -1047,13 +1074,15 @@ This section was moved to
[ECMAScript Modules]: esm.md
[GLOBAL_FOLDERS]: #loading-from-the-global-folders
[`"main"`]: packages.md#main
[`"type"`]: packages.md#type
[`ERR_REQUIRE_ESM`]: errors.md#err_require_esm
[`Error`]: errors.md#class-error
[`__dirname`]: #__dirname
[`__filename`]: #__filename
[`import()`]: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports
[`module.children`]: #modulechildren
[`module.id`]: #moduleid
[`module` core module]: module.md
[`module` object]: #the-module-object
[`package.json`]: packages.md#nodejs-packagejson-field-definitions
[`path.dirname()`]: path.md#pathdirnamepath
Expand Down
18 changes: 10 additions & 8 deletions doc/api/packages.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,13 @@ along with a reference for the [`package.json`][] fields defined by Node.js.
## Determining module system

Node.js will treat the following as [ES modules][] when passed to `node` as the
initial input, or when referenced by `import` statements within ES module code:
initial input, or when referenced by `import` statements or `import()`
expressions:

* Files ending in `.mjs`.
* Files with a `.mjs` extension.
aduh95 marked this conversation as resolved.
Show resolved Hide resolved

* Files ending in `.js` when the nearest parent `package.json` file contains a
top-level [`"type"`][] field with a value of `"module"`.
* Files with a `.js` extension when the nearest parent `package.json` file
contains a top-level [`"type"`][] field with a value of `"module"`.

* Strings passed in as an argument to `--eval`, or piped to `node` via `STDIN`,
with the flag `--input-type=module`.
Expand All @@ -67,12 +68,13 @@ field, or string input without the flag `--input-type`. This behavior is to
preserve backward compatibility. However, now that Node.js supports both
CommonJS and ES modules, it is best to be explicit whenever possible. Node.js
will treat the following as CommonJS when passed to `node` as the initial input,
or when referenced by `import` statements within ES module code:
or when referenced by `import` statements, `import()` expressions, or
`require()` expressions:
GeoffreyBooth marked this conversation as resolved.
Show resolved Hide resolved

* Files ending in `.cjs`.
* Files with a `.cjs` extension.

* Files ending in `.js` when the nearest parent `package.json` file contains a
top-level field [`"type"`][] with a value of `"commonjs"`.
* Files with a `.js` extension when the nearest parent `package.json` file
contains a top-level field [`"type"`][] with a value of `"commonjs"`.

* Strings passed in as an argument to `--eval` or `--print`, or piped to `node`
via `STDIN`, with the flag `--input-type=commonjs`.
Expand Down
2 changes: 1 addition & 1 deletion doc/api/synopsis.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,6 @@ Now, open any preferred web browser and visit `http://127.0.0.1:3000`.
If the browser displays the string `Hello, World!`, that indicates
the server is working.

[Command-line options]: cli.md#command-line-options
[Command-line options]: cli.md#options
GeoffreyBooth marked this conversation as resolved.
Show resolved Hide resolved
[Installing Node.js via package manager]: https://nodejs.org/en/download/package-manager/
[web server]: http.md
2 changes: 1 addition & 1 deletion tools/doc/links-mapper.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"doc/api/synopsis.md": {
"command line options": "cli.html#command-line-options",
"command line options": "cli.html#options",
"web server": "http.html"
}
}