esm: import.meta.resolve with nodejs: builtins

Backport-PR-URL: #32610
PR-URL: #31032
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>

doc/api/cli.md

@@ -156,6 +156,13 @@ Enable experimental Source Map V3 support for stack traces.
 Currently, overriding `Error.prepareStackTrace` is ignored when the
 `--enable-source-maps` flag is set.
 
+### `--experimental-import-meta-resolve`
+<!-- YAML
+added: REPLACEME
+-->
+
+Enable experimental `import.meta.resolve()` support.
+
 ### `--experimental-json-modules`
 <!-- YAML
 added: v12.9.0
@@ -1085,6 +1092,7 @@ Node.js options that are allowed are:
 <!-- node-options-node start -->
 * `--enable-fips`
 * `--enable-source-maps`
+* `--experimental-import-meta-resolve`
 * `--experimental-json-modules`
 * `--experimental-loader`
 * `--experimental-modules`

doc/api/esm.md

@@ -837,6 +837,32 @@ const __filename = fileURLToPath(import.meta.url);
 const __dirname = dirname(__filename);
 ```
 
+### No `require.resolve`
+
+Former use cases relying on `require.resolve` to determine the resolved path
+of a module can be supported via `import.meta.resolve`, which is experimental
+and supported via the `--experimental-import-meta-resolve` flag:
+
+```js
+(async () => {
+  const dependencyAsset = await import.meta.resolve('component-lib/asset.css');
+})();
+```
+
+`import.meta.resolve` also accepts a second argument which is the parent module
+from which to resolve from:
+
+```js
+(async () => {
+  // Equivalent to import.meta.resolve('./dep')
+  await import.meta.resolve('./dep', import.meta.url);
+})();
+```
+
+This function is asynchronous since the ES module resolver in Node.js is
+asynchronous. With the introduction of [Top-Level Await][], these use cases
+will be easier as they won't require an async function wrapper.
+
 ### No `require.extensions`
 
 `require.extensions` is not used by `import`. The expectation is that loader
@@ -1405,13 +1431,14 @@ The resolver has the following properties:
 
 The algorithm to load an ES module specifier is given through the
 **ESM_RESOLVE** method below. It returns the resolved URL for a
-module specifier relative to a parentURL, in addition to the unique module
-format for that resolved URL given by the **ESM_FORMAT** routine.
+module specifier relative to a parentURL.
 
-The _"module"_ format is returned for an ECMAScript Module, while the
-_"commonjs"_ format is used to indicate loading through the legacy
-CommonJS loader. Additional formats such as _"addon"_ can be extended in future
-updates.
+The algorithm to determine the module format of a resolved URL is
+provided by **ESM_FORMAT**, which returns the unique module
+format for any file. The _"module"_ format is returned for an ECMAScript
+Module, while the _"commonjs"_ format is used to indicate loading through the
+legacy CommonJS loader. Additional formats such as _"addon"_ can be extended in
+future updates.
 
 In the following algorithms, all subroutine errors are propagated as errors
 of these top-level routines unless stated otherwise.
@@ -1440,11 +1467,13 @@ _defaultEnv_ is the conditional environment name priority array,
 > 1. If _resolvedURL_ contains any percent encodings of _"/"_ or _"\\"_ (_"%2f"_
 >    and _"%5C"_ respectively), then
 >    1. Throw an _Invalid Specifier_ error.
-> 1. If the file at _resolvedURL_ does not exist, then
+> 1. If _resolvedURL_ does not end with a trailing _"/"_ and the file at
+>    _resolvedURL_ does not exist, then
 >    1. Throw a _Module Not Found_ error.
 > 1. Set _resolvedURL_ to the real path of _resolvedURL_.
 > 1. Let _format_ be the result of **ESM_FORMAT**(_resolvedURL_).
 > 1. Load _resolvedURL_ as module format, _format_.
+> 1. Return _resolvedURL_.
 
 **PACKAGE_RESOLVE**(_packageSpecifier_, _parentURL_)
 
@@ -1472,7 +1501,7 @@ _defaultEnv_ is the conditional environment name priority array,
 > 1. If _selfUrl_ isn't empty, return _selfUrl_.
 > 1. If _packageSubpath_ is _undefined_ and _packageName_ is a Node.js builtin
 >    module, then
->    1. Return the string _"node:"_ concatenated with _packageSpecifier_.
+>    1. Return the string _"nodejs:"_ concatenated with _packageSpecifier_.
 > 1. While _parentURL_ is not the file system root,
 >    1. Let _packageURL_ be the URL resolution of _"node_modules/"_
 >       concatenated with _packageSpecifier_, relative to _parentURL_.
@@ -1481,6 +1510,8 @@ _defaultEnv_ is the conditional environment name priority array,
 >       1. Set _parentURL_ to the parent URL path of _parentURL_.
 >       1. Continue the next loop iteration.
 >    1. Let _pjson_ be the result of **READ_PACKAGE_JSON**(_packageURL_).
+>    1. If _packageSubpath_ is equal to _"./"_, then
+>       1. Return _packageURL_ + _"/"_.
 >    1. If _packageSubpath_ is _undefined__, then
 >       1. Return the result of **PACKAGE_MAIN_RESOLVE**(_packageURL_,
 >          _pjson_).
@@ -1502,6 +1533,8 @@ _defaultEnv_ is the conditional environment name priority array,
 > 1. If _pjson_ does not include an _"exports"_ property, then
 >    1. Return **undefined**.
 > 1. If _pjson.name_ is equal to _packageName_, then
+>    1. If _packageSubpath_ is equal to _"./"_, then
+>       1. Return _packageURL_ + _"/"_.
 >    1. If _packageSubpath_ is _undefined_, then
 >       1. Return the result of **PACKAGE_MAIN_RESOLVE**(_packageURL_, _pjson_).
 >    1. Otherwise,
@@ -1680,3 +1713,4 @@ success!
 [the official standard format]: https://tc39.github.io/ecma262/#sec-modules
 [transpiler loader example]: #esm_transpiler_loader
 [6.1.7 Array Index]: https://tc39.es/ecma262/#integer-index
+[Top-Level Await]: https://github.com/tc39/proposal-top-level-await

doc/node.1

@@ -113,6 +113,9 @@ Requires Node.js to be built with
 .It Fl -enable-source-maps
 Enable experimental Source Map V3 support for stack traces.
 .
+.It Fl -experimental-import-meta-resolve
+Enable experimental ES modules support for import.meta.resolve().
+.
 .It Fl -experimental-json-modules
 Enable experimental JSON interop support for the ES Module loader.
 .

lib/internal/modules/esm/get_format.js

@@ -1,6 +1,5 @@
 'use strict';
 
-const { NativeModule } = require('internal/bootstrap/loaders');
 const { extname } = require('path');
 const { getOptionValue } = require('internal/options');
 
@@ -39,7 +38,7 @@ if (experimentalJsonModules)
   extensionFormatMap['.json'] = legacyExtensionFormatMap['.json'] = 'json';
 
 function defaultGetFormat(url, context, defaultGetFormat) {
-  if (NativeModule.canBeRequiredByUsers(url)) {
+  if (url.startsWith('nodejs:')) {
     return { format: 'builtin' };
   }
   const parsed = new URL(url);
@@ -73,5 +72,6 @@ function defaultGetFormat(url, context, defaultGetFormat) {
     }
     return { format: format || null };
   }
+  return { format: null };
 }
 exports.defaultGetFormat = defaultGetFormat;

lib/internal/modules/esm/loader.js

@@ -94,7 +94,10 @@ class Loader {
       throw new ERR_INVALID_RETURN_PROPERTY_VALUE(
         'string', 'loader resolve', 'url', url);
     }
+    return url;
+  }
 
+  async getFormat(url) {
     const getFormatResponse = await this._getFormat(
       url, {}, defaultGetFormat);
     if (typeof getFormatResponse !== 'object') {
@@ -109,7 +112,7 @@ class Loader {
     }
 
     if (format === 'builtin') {
-      return { url: `node:${url}`, format };
+      return format;
     }
 
     if (this._resolve !== defaultResolve) {
@@ -132,7 +135,7 @@ class Loader {
       );
     }
 
-    return { url, format };
+    return format;
   }
 
   async eval(
@@ -185,7 +188,8 @@ class Loader {
   }
 
   async getModuleJob(specifier, parentURL) {
-    const { url, format } = await this.resolve(specifier, parentURL);
+    const url = await this.resolve(specifier, parentURL);
+    const format = await this.getFormat(url);
     let job = this.moduleMap.get(url);
     // CommonJS will set functions for lazy job evaluation.
     if (typeof job === 'function')

lib/internal/modules/esm/resolve.js

@@ -8,6 +8,7 @@ const internalFS = require('internal/fs/utils');
 const { NativeModule } = require('internal/bootstrap/loaders');
 const { realpathSync } = require('fs');
 const { getOptionValue } = require('internal/options');
+const { sep } = require('path');
 
 const preserveSymlinks = getOptionValue('--preserve-symlinks');
 const preserveSymlinksMain = getOptionValue('--preserve-symlinks-main');
@@ -29,11 +30,13 @@ function defaultResolve(specifier, { parentURL } = {}, defaultResolve) {
       };
     }
   } catch {}
+  if (parsed && parsed.protocol === 'nodejs:')
+    return { url: specifier };
   if (parsed && parsed.protocol !== 'file:' && parsed.protocol !== 'data:')
     throw new ERR_UNSUPPORTED_ESM_URL_SCHEME();
   if (NativeModule.canBeRequiredByUsers(specifier)) {
     return {
-      url: specifier
+      url: 'nodejs:' + specifier
     };
   }
   if (parentURL && parentURL.startsWith('data:')) {
@@ -58,11 +61,12 @@ function defaultResolve(specifier, { parentURL } = {}, defaultResolve) {
   let url = moduleWrapResolve(specifier, parentURL);
 
   if (isMain ? !preserveSymlinksMain : !preserveSymlinks) {
-    const real = realpathSync(fileURLToPath(url), {
+    const urlPath = fileURLToPath(url);
+    const real = realpathSync(urlPath, {
       [internalFS.realpathCacheKey]: realpathCache
     });
     const old = url;
-    url = pathToFileURL(real);
+    url = pathToFileURL(real + (urlPath.endsWith(sep) ? '/' : ''));
     url.search = old.search;
     url.hash = old.hash;
   }

lib/internal/modules/esm/translators.js

@@ -28,6 +28,9 @@ const { ERR_UNKNOWN_BUILTIN_MODULE } = require('internal/errors').codes;
 const { maybeCacheSourceMap } = require('internal/source_map/source_map_cache');
 const moduleWrap = internalBinding('module_wrap');
 const { ModuleWrap } = moduleWrap;
+const { getOptionValue } = require('internal/options');
+const experimentalImportMetaResolve =
+    getOptionValue('--experimental-import-meta-resolve');
 
 const debug = debuglog('esm');
 
@@ -42,16 +45,28 @@ function errPath(url) {
   return url;
 }
 
-function initializeImportMeta(meta, { url }) {
-  meta.url = url;
-}
-
 let esmLoader;
 async function importModuleDynamically(specifier, { url }) {
   if (!esmLoader) {
-    esmLoader = require('internal/process/esm_loader');
+    esmLoader = require('internal/process/esm_loader').ESMLoader;
   }
-  return esmLoader.ESMLoader.import(specifier, url);
+  return esmLoader.import(specifier, url);
+}
+
+function createImportMetaResolve(defaultParentUrl) {
+  return async function resolve(specifier, parentUrl = defaultParentUrl) {
+    if (!esmLoader) {
+      esmLoader = require('internal/process/esm_loader').ESMLoader;
+    }
+    return esmLoader.resolve(specifier, parentUrl);
+  };
+}
+
+function initializeImportMeta(meta, { url }) {
+  // Alphabetical
+  if (experimentalImportMetaResolve)
+    meta.resolve = createImportMetaResolve(url);
+  meta.url = url;
 }
 
 // Strategy for loading a standard JavaScript module
@@ -104,10 +119,10 @@ translators.set('commonjs', function commonjsStrategy(url, isMain) {
 // through normal resolution
 translators.set('builtin', async function builtinStrategy(url) {
   debug(`Translating BuiltinModule ${url}`);
-  // Slice 'node:' scheme
-  const id = url.slice(5);
+  // Slice 'nodejs:' scheme
+  const id = url.slice(7);
   const module = loadNativeModule(id, url, true);
-  if (!module) {
+  if (!url.startsWith('nodejs:') || !module) {
     throw new ERR_UNKNOWN_BUILTIN_MODULE(id);
   }
   debug(`Loading BuiltinModule ${url}`);

src/module_wrap.cc

@@ -812,6 +812,10 @@ Maybe<URL> FinalizeResolution(Environment* env,
     return Nothing<URL>();
   }
 
+  if (resolved.path().back() == '/') {
+    return Just(resolved);
+  }
+
   const std::string& path = resolved.ToFilePath();
   if (CheckDescriptorAtPath(path) != FILE) {
     std::string msg = "Cannot find module " +
@@ -1197,7 +1201,9 @@ Maybe<URL> ResolveSelf(Environment* env,
     }
     if (!found_pjson || pcfg->name != pkg_name) return Nothing<URL>();
     if (pcfg->exports.IsEmpty()) return Nothing<URL>();
-    if (!pkg_subpath.length()) {
+    if (pkg_subpath == "./") {
+      return Just(URL("./", pjson_url));
+    } else if (!pkg_subpath.length()) {
       return PackageMainResolve(env, pjson_url, *pcfg, base);
     } else {
       return PackageExportsResolve(env, pjson_url, pkg_subpath, *pcfg, base);
@@ -1241,8 +1247,7 @@ Maybe<URL> PackageResolve(Environment* env,
     return Nothing<URL>();
   }
   std::string pkg_subpath;
-  if ((sep_index == std::string::npos ||
-      sep_index == specifier.length() - 1)) {
+  if (sep_index == std::string::npos) {
     pkg_subpath = "";
   } else {
     pkg_subpath = "." + specifier.substr(sep_index);
@@ -1273,7 +1278,9 @@ Maybe<URL> PackageResolve(Environment* env,
     Maybe<const PackageConfig*> pcfg = GetPackageConfig(env, pjson_path, base);
     // Invalid package configuration error.
     if (pcfg.IsNothing()) return Nothing<URL>();
-    if (!pkg_subpath.length()) {
+    if (pkg_subpath == "./") {
+      return Just(URL("./", pjson_url));
+    } else if (!pkg_subpath.length()) {
       return PackageMainResolve(env, pjson_url, *pcfg.FromJust(), base);
     } else {
       if (!pcfg.FromJust()->exports.IsEmpty()) {

src/node_options.cc

@@ -116,6 +116,10 @@ void PerIsolateOptions::CheckOptions(std::vector<std::string>* errors) {
 }
 
 void EnvironmentOptions::CheckOptions(std::vector<std::string>* errors) {
+  if (experimental_import_meta_resolve && !experimental_modules) {
+    errors->push_back("--experimental-meta-resolve requires "
+                      "--experimental-modules be enabled");
+  }
   if (!userland_loader.empty() && !experimental_modules) {
     errors->push_back("--experimental-loader requires "
                       "--experimental-modules be enabled");
@@ -360,6 +364,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
             "experimental ES Module support for webassembly modules",
             &EnvironmentOptions::experimental_wasm_modules,
             kAllowedInEnvironment);
+  AddOption("--experimental-import-meta-resolve",
+            "experimental ES Module import.meta.resolve() support",
+            &EnvironmentOptions::experimental_import_meta_resolve,
+            kAllowedInEnvironment);
   AddOption("--experimental-policy",
             "use the specified file as a "
             "security policy",

src/node_options.h

@@ -106,6 +106,7 @@ class EnvironmentOptions : public Options {
   std::string experimental_specifier_resolution;
   std::string es_module_specifier_resolution;
   bool experimental_wasm_modules = false;
+  bool experimental_import_meta_resolve = false;
   std::string module_type;
   std::string experimental_policy;
   std::string experimental_policy_integrity;

test/es-module/test-esm-dynamic-import.js

@@ -54,11 +54,12 @@ function expectFsNamespace(result) {
   expectFsNamespace(import('fs'));
   expectFsNamespace(eval('import("fs")'));
   expectFsNamespace(eval('import("fs")'));
+  expectFsNamespace(import('nodejs:fs'));
 
+  expectModuleError(import('nodejs:unknown'),
+                    'ERR_UNKNOWN_BUILTIN_MODULE');
   expectModuleError(import('./not-an-existing-module.mjs'),
                     'ERR_MODULE_NOT_FOUND');
-  expectModuleError(import('node:fs'),
-                    'ERR_UNSUPPORTED_ESM_URL_SCHEME');
   expectModuleError(import('http://example.com/foo.js'),
                     'ERR_UNSUPPORTED_ESM_URL_SCHEME');
 })();