Allow --experimental-network-imports import container http code

[loynoir]

What is the problem this feature will solve?

$ node --experimental-network-imports
Welcome to Node.js v20.4.0.
Type ".help" for more information.
> (node:455897) ExperimentalWarning: Network Imports is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

> await import('http://self-hosted-git-server/user/repo/raw/commit/929afbdef910c2a845b5de7f21f19010b4b0cc69/src/example.mjs')
Uncaught:
Error [ERR_NETWORK_IMPORT_DISALLOWED]: import of 'http://self-hosted-git-server/user/repo/raw/commit/929afbdef910c2a845b5de7f21f19010b4b0cc69/src/example.mjs' by undefined is not supported: http can only be used to load local resources (use https instead).
    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)
    at new NodeError (node:internal/errors:405:5) {
  code: 'ERR_NETWORK_IMPORT_DISALLOWED'
}
> 

What is the feature you are proposing to solve the problem?

Allow --experimental-network-imports import container http code

What alternatives have you considered?

iptables-workaround.sh

#!/bin/bash
sudo iptables -t nat \
    -A OUTPUT -s 127.0.0.1/32 -p tcp -m comment --comment x_workaround_node_http_import_container -m tcp --dport 11111 -j DNAT --to-destination 172.22.0.3:80
sudo iptables -t nat \
    -A POSTROUTING -o eth2 -m comment --comment x_workaround_node_http_import_container -j MASQUERADE

$ bash iptables-workaround.sh
$ node --experimental-network-imports
Welcome to Node.js v20.4.0.
Type ".help" for more information.
> (node:456666) ExperimentalWarning: Network Imports is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

> await import('http://127.0.0.1:11111/user/repo/raw/commit/929afbdef910c2a845b5de7f21f19010b4b0cc69/src/example.mjs')
Uncaught:
Error [ERR_NETWORK_IMPORT_DISALLOWED]: import of 'node:path' by http://127.0.0.1:11111/user/repo/raw/commit/929afbdef910c2a845b5de7f21f19010b4b0cc69/src/example.mjs is not supported: only relative and absolute specifiers are supported.
    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)
    at new NodeError (node:internal/errors:405:5)
    at checkIfDisallowedImport (node:internal/modules/esm/resolve:921:13)
    at defaultResolve (node:internal/modules/esm/resolve:1007:23)
    at DefaultModuleLoader.resolve (node:internal/modules/esm/loader:251:12)
    at DefaultModuleLoader.getModuleJob (node:internal/modules/esm/loader:140:32) {
  code: 'ERR_NETWORK_IMPORT_DISALLOWED'
}

Related

#48591

[mertcanaltin]

Is it a good solution if I update the request and get functions so that I can import the module? @loynoir

lib/http.js

function request(url, options, cb) {
  if (typeof url === 'string' && url.startsWith('http://self-hosted-git-server/')) {
    const parsedUrl = new URL(url);
    const protocol = parsedUrl.protocol.slice(0, -1);
    const host = parsedUrl.host;
    const path = parsedUrl.pathname;

    const requestOptions = Object.assign({}, options, {
      protocol: protocol,
      host: host,
      path: path
    });

    return new ClientRequest(requestOptions, cb);
  } else {
    return new ClientRequest(url, options, cb);
  }
}

[loynoir]

@mertcanaltin

I don't think it related to lib/http.js, but

node/lib/internal/modules/esm/fetch_module.js

Lines 248 to 259 in 7196946

	if (parsed.protocol === 'http:') {
	return PromisePrototypeThen(isLocalAddress(parsed.hostname), (is) => {
	if (is !== true) {
	throw new ERR_NETWORK_IMPORT_DISALLOWED(
	href,
	parentURL,
	'http can only be used to load local resources (use https instead).',
	);
	}
	return fetchWithRedirects(parsed);
	});
	}

node/lib/internal/modules/esm/fetch_module.js

Lines 212 to 227 in 7196946

	async function isLocalAddress(hostname) {
	try {
	if (
	StringPrototypeStartsWith(hostname, '[') &&
	StringPrototypeEndsWith(hostname, ']')
	) {
	hostname = StringPrototypeSlice(hostname, 1, -1);
	}
	const addr = await dnsLookup(hostname, { verbatim: true });
	const ipv = addr.family === 4 ? 'ipv4' : 'ipv6';
	return allowList.check(addr.address, ipv);
	} catch {
	// If it errored, the answer is no.
	}
	return false;
	}

node/lib/internal/modules/esm/fetch_module.js

Lines 202 to 205 in 7196946

	const allowList = new net.BlockList();
	allowList.addAddress('::1', 'ipv6');
	allowList.addRange('127.0.0.1', '127.255.255.255');

And docker container ip like 172.22.0.3 not in http allowList.

[ThaUnknown]

you can solve this using import hooks, it's not the safest thing out there, but it will work:

import { register } from 'node:module'

register('./importHooks.mjs', import.meta.url)

importHooks.mjs:

export async function resolve (specifier, context, nextResolve) {
  if (specifier?.startsWith?.('node:')) return nextResolve(specifier, { ...context, parentURL: import.meta.url })
  return nextResolve(specifier)
}

you could change the condition to say check parentURL or specific modules, but this will enable all node native imports for all remotes, keep in mind how unsafe this is!

[github-actions]

There has been no activity on this feature request for 5 months. To help maintain relevant open issues, please add the never-stale Mark issue so that it is never considered stale label or close this issue if it should be closed. If not, the issue will be automatically closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.

[github-actions]

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.