no way to programmatically discover node:test

[ljharb]

Version

18.0.0

Platform

No response

Subsystem

No response

What steps will reproduce the bug?

require('module').builtinModules

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior?

Everything that's requireable that ships with node is listed (or, the listed items are requireable with node: added).

What do you see instead?

node:test is not listed.

Additional information

There needs to be a way to programmatically discover all builtin modules. require('module').builtinModules is supposed to be it.

This broke assumptions in my tests for https://npmjs.com/is-core-module - specifically, CIGTM should have failed prior to node 18 going out, because node:test wasn't part of is-core-module, but by making it effectively "secret", my tests didn't know about it.

[ljharb]

cc @nodejs/modules @cjihrig @nodejs/test_runner

[aduh95]

import("node:test").then(()=>true, ()=>false)?

[ljharb]

That's not synchronous, and requires that I hardcode the identifier in advance.

I should be able to write code that works for all future prefix-only core modules without ever needing to hardcode their names.

[jasnell]

@cjihrig ... Looking through the code it appears that not listing node:test in builtinModules was intentional ... though it's not clear why. I agree with @ljharb that it likely should be there. Adding it, however, does break a handful of tests so I wanted to check to see what needs to be considered there first.

[aduh95]

@ljharb there's a workaround, but it requires to use --expose-internals, sharing in case that unblocks you:

'use strict';

const { internalBinding } = require('internal/test/binding');
const {
  moduleCategories: { canBeRequired },
} = internalBinding('native_module');

console.log(canBeRequired.has('test')); // true

[ljharb]

@aduh95 thanks; good to know, but i don't think it suffices.

I think that the solutions here are either:

1. add node:test to builtinModules
2. make test requireable
3. add a new list of prefix-only core modules

My preference is the second one, the first is the simplest, and the third imo would be more of an argument for the second one because of the user confusion it furthers.

[devsnek]

I would prefer option 1, it seems acceptable as an opaque string.

[jasnell]

Option 2 has already been settled by the TSC vote. That makes options 1 and 3 the viable paths forward here. My preference would be for option 1. As far as I can tell, that shouldn't actually break anyone except a couple of our tests.

[cjihrig]

I'm also in favor of option 1 (only because option 2 is off the table).

[aduh95]

An option 4 would be to freeze and expose the canBeRequired set. I agree that option 1 and 3 are also viable paths, and option 1 is probably the simplest (although it might still be a breaking change?)

[ljharb]

I'm not sure why it would be - was it ever communicated that the API of this list is that nothing has a node prefix?

My tests were doing "builtinModule item", and "builtinModule item with a node prefix" - so i did have to change the logic to "only add the node prefix if it's not already there". That's a pretty minimal change tho, and arguably i shouldn't have hardcoded the assumption that things in that list don't have the prefix. So option 1 does seem like it'd be fine.

[ljharb]

another oddity i noticed is that most all the builtin modules are available as globals in the repl - except for ones like module that are shadowed by the CJS module - and test is not available there. Should I file a separate issue for that? It seems like there'll be a bunch of unaccounted-for edge cases with the "prefix-only" approach the TSC vote unfortunately settled on.

[jasnell]

The repl limitation is known already. It's not worth opening an issue, I think. Prefix only modules just won't be available as globals in the repl.

[ljharb]

That's probably fine for the test runner, but I suspect it will be pretty limiting if future modules are added prefix-only. I guess I'll wait until that happens to bring it up again tho.

[ljharb]

@aduh95 this issue is not fixed; isBuiltIn doesn’t do it because you can’t discover node:test with it. Please reopen.

[hemanth]

@ljharb do we see a need for module. builtinPrefixOnly ? I can work on cooking that if we decide to have one.

[ljharb]

@hemanth possibly, but that still wouldn't address this issue either. We likely need an API that provides an iterator over all of the names of builtin modules (fs, node:fs, node:test, etc), or, a Set instance containing them, or something similar.

[hemanth]

@ljharb trying to recall on why we didn't ended up populating builtinModules will all of the prefix, non-prefix and prefix-only modules to begin with.

[ljharb]

Probably because the assumption was that there'd never be a prefix-only module, but node:test broke that axiom :-(

[hemanth]

Hmm, need to verify if there are any side effects of populating builtinModules with all of it.

[hemanth]

Module.builtinModules = ObjectFreeze([...allBuiltins]); // in loader.js

All of the module tests pass.

Should I do a PR if there are no other concerns? //cc @nodejs/modules

[loynoir]

#47465

> nodeModule.isBuiltin('node:test')
true

---

After node: protocol introduced, and node:X and X is now kind of different, does it make sense to add .builtinModuleIds, and deprecate .builtinModules?

.builtinModuleIds

• it is something like ["X", "node:X", "X/promises", "node:X/promises"]

• in ESM, it refers to AST level import statement importId

• in CJS, it refers to AST level require expression arg[0]

• Compared with .builtinModules which use concept module, now a layer over raw AST concept, leads to different instinct between node dev and userland dev

[ljharb]

This issue also masks the existence of node:sea and node:sqlite.

[RedYetiDev]

This issue also masks the existence of node:sea and node:sqlite.

And it's likely that more will follow. For my 2 cents, the most logical approach IMO is to populate the builtinModules list with all modules, each labeled with the node: prefix (since having node:sea alongside fs (not node:fs) wouldn't make sense IMO).