module: support pattern trailers

[guybedford]

This implements an extension to exports (and imports) patterns to support LHS matches consisting of trailing strings.

The use case for this came out of discussion with @ljharb, in order to support packages that want to provide "exports" support while enabling subpaths that can support both extensioned and unextensioned forms.

For example, with this PR, the following exports field can be used to expose a features folder with optional extensions:

{
  "exports": {
    "./features/*": "./features/*.js",
    "./features/*.js": "./features/*.js"
  }
}

While we would still encourage package authors to decide on a single pattern, often packages don't have a choice when backwards compatibility is required, so this can also allow for smoother interface transitions if deprecating one or the other pattern as well.

This was left out of the initial implementation to keep the initial implementation as simple as possible. Most of the work is just clearly defining pattern specificity and handling edge cases.

This PR also refines the following edge cases:

1. Trailing "/" exports that resolve to files will now give the trailing "/" deprecation warning.
2. Given the two exports keys "./x/" and "./x*", the second pattern will take precedence over the path for resolving ./x/y according to the specificity rules.
3. Multiple * can't be matched in patterns, so given "exports": { "./*/*.js": "./*.js" } this will no longer match import './*/x.js' and instead throw an unexported error. It is of course possible to resolve a * as part of the user specifier component, it just won't be supported as a string character in exports.

[guybedford]

@nodejs/modules

[ljharb]

What will happen to a package using pattern trailers, in a version of node that supports subpath patterns but not pattern trailers?

[guybedford]

@ljharb it would simply be ignored as was reserved for the forwards compatibility path here. In the above example, ./features/* would be matched.

[ljharb]

Great! It’s a shame this wasn’t in the initial version, but it’s a great improvement now.

[ljharb]

Before this lands, is there any remotely easy way i can test it locally against my package being imported by a dependency?

[guybedford]

@ljharb the best way is always to take the time to clone and run the build locally. Perhaps @MylesBorins or @targos would be able to trigger a special build but I'm not sure of the process there.

[targos]

Test build: https://ci-release.nodejs.org/job/iojs+release/7075/

I'll update this message when it's available to download.

[guybedford]

Turns out there was an issue with the feedback changes, just pushed up a fix.

@targos thanks for triggering this, is it possible to restart it?

[targos]

New build: https://ci-release.nodejs.org/job/iojs+release/7076/

[targos]

Download links: https://nodejs.org/download/test/v17.0.0-test24288e0c26/

[guybedford]

Ping for further reviewers here //cc @nodejs/modules.

[ljharb]

@targos unfortunately i can't easily install that with nvm because it's not present in the index.tab :-p thanks tho, i'll try to get it installed locally.

[bmeck]

LGTM, not performing the actual sort seems fine as well since the list of keys is likely not huge, but it could be beneficial if we think we will see large mappings.

[ljharb]

@guybedford it'd be great if i could test this before you land it; or at least before it's backported.

[MylesBorins]

I'll try and find time to review this over the next week. @ljharb if we make a test build you can install via nvm, I'll create a test build and post the one liner to install via nvm.

@guybedford How far were you thinking we could backport this? I'm guessing 14 is as far back as we could go.

[ljharb]

@MylesBorins the test build exists - #39635 (comment) - but it's not listed in https://nodejs.org/download/test/index.tab, so i can't in fact install it via nvm (happy to be wrong here tho)

[MylesBorins]

@ljharb I just checked the CI job for the build and there was a failure that resulted in the tabfile not getting updated. Just reran in and finger crossed it should work this time.

[ljharb]

@MylesBorins It's been updated (the timestamp on https://nodejs.org/download/test/ updated) but it still only contains v17.0.0-test20210702a107e9054a, and not v17.0.0-test24288e0c26 :-/

[guybedford]

How far were you thinking we could backport this? I'm guessing 14 is as far back as we could go.

@MylesBorins as far back as we can go yes, 14 could be fine though.

[ljharb]

I got it downloaded and installed by locally hardcoding nvm to accept v17.0.0-test24288e0c26.

Tested it with es-abstract imported by another module, and I still couldn't get all of v17.0.0-test24288e0c26, v16.6.1, v12.17, and v12.22 to be able to import the module with, or without, the extension.

The best results were without the extension - with an exports like:

"./2020/*": "./2020/*.js",
"./2020/*.js": "./2020/*.js",
"./2020/": "./2020/",

everything worked but v12.17, which had an error message about using the extension (which, when i use it, works in 17 and 12.17 but fails in latest 12 and 16).

I won't be able to use the feature if I can't maintain back compat; is there any iteration we might want to do before landing this?

[guybedford]

@ljharb thanks for testing it out. 12.17 doesn't support patterns or extensions searching for path exports I believe which would explain that case. I assume by "17" in your last reference you mean this PR, which yes is the only way to support the extensioned form mapping so your results sound as expected to me. As discussed in the issue on your repo, I don't think this feature can apply to your use case at this point as, as you have seen, there is no way to support both the extensioned and unextensioned import forms while supporting all versions of Node.js and using the exports field. I can't think of a variation of this feature that could get around the compatibility.

If we had shipped this feature from day one then yes it would have solved your use case though.

[ljharb]

I understand that I’m unable to support both forms; I’m asking if there’s a way to support one of them in all those node versions, even if it’s extensioned for ESM (as long as CJS is always extensionless). iow, i don’t mind if to support 12.17 means ESM must use extensions, as long as a) CJS can continue to omit them, and b) a bonus is if new code that doesn’t support 12.17 et al can use extensionless ESM.

[guybedford]

The best option is yes to use the extensioned form, and then use the exports field:

"./2020/*": "./2020/*",
"./2020/": "./2020/",

That should work in all versions of Node.

Because exports applies equally to ESM and CJS, there's no way to support extensioned or not for one or the other short of using conditions to just disable certain paths.

If you want to support unextensioned, then you would use the rule:

"./2020/*": "./2020/*.js",
"./2020/": "./2020/",

This PR doesn't apply to either case, as the use case for this PR is being able to support both.

The other benefit of the trailing *.js mapping is just that it specifies the exact file extensions so "./2020/*": "./2020/*" being replaced with "./2020/*.js": "./2020/*.js" for modern Node.js support ensures only .js files are exposed publicly, allowing files with other file extensions to maintain encapsulation despite using extensioned subpaths.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/39544/