host vs hostname inconsistency

[sholladay]

Version: Node 8 and others

Problem

The semantics of the host option for server.listen() are incompatible with the conventions of other APIs, both in Node itself and in browsers, etc. This is a footgun that could be fixed by making the options for .listen() be consistent with other APIs.

Context

Most systems I interact with distinguish between a hostname and a host, where the latter includes a port number and the former does not.

In general, Node behaves this way, too (see os.hostname(), WHATWG URL, and the legacy url module APIs). However, the option object passed to server.listen(option) is weird. It only understands host (not hostname), but a port number is disallowed. This is confusing and leads to very surprising results in some cases.

Prior art

In browsers

location.href = 'https://localhost:3000';
console.log(location.host !== location.hostname);  // true
const whatwgParsed = new URL('https://localhost:3000');
console.log(whatwgParsed.host !== whatwgParsed.hostname);  // true

In Node.js

const url = require('url');
const target = 'https://localhost:3000';
const legacyParsed = url.parse(target);
console.log(legacyParsed.host !== legacyParsed.hostname);  // true
const whatwgParsed = new url.URL(target);
console.log(whatwgParsed.host !== whatwgParsed.hostname);  // true

Current behavior

The server.listen() API understands a host option, but it is incompatible with host from other APIs, including WHATWG URL and others. This is because .listen() refuses perfectly valid hosts such as localhost:3000. A reasonable expectation would be to try hostname instead, which is the more common name for the current behavior of the host option. Unfortunately, that doesn't work either, as .listen() does not understand hostname, and it will be silently ignored.

// Reports an error correctly, since the API lacks a default port, but the error message is somewhat vague
server.listen({ host : 'localhost' }, () => {});
Error: Invalid listen argument: { host: 'localhost' }

// Listens on 0.0.0.0 rather than localhost (works as documented, but very confusing and as a footgun is arguably unsafe)
server.listen({ hostname : 'localhost', port : 3000}, () => {});

// Reports an error even though the host is valid (works as documented, but confusing and incompatible with other specs and APIs)
server.listen({ host : 'localhost:3000' }, () => {});
Error: Invalid listen argument: { host: 'localhost:3000' }

// Succeeds, even though host usually takes precedence over separate hostname and port options and the host lacks a port (works as documented, but a bit strange)
server.listen({ host : 'localhost', port : 3000 }, () => {});

Expected behavior

APIs that accept a host should respect their port number to avoid confusion and inconsistency. Having separate options for hostname and port is also awesome, but in that case it should be named hostname instead of host. I would argue that Node should only support hostname instead of host (leaving parsing hosts to userland) but that would be a breaking change, so I'm not sure how feasible that is. At the very least, I think a new hostname option could be introduced with the correct behavior, and then host could be extended to respect port numbers, and the fact that host continues to be in the API would purely be for backwards compatibility reasons.

// Should report an intuitive error. The host is perfectly valid, but the API lacks a default port.
server.listen({ host : 'localhost' }, () => {});
Error: No port specified

// Should listen on localhost, whereas it currently listens on 0.0.0.0 instead because hostname is not understood.
server.listen({ hostname : 'localhost', port : 3000}, () => {});

// If host needs to be supported, then this should listen on localhost and port 3000, whereas it currently throws an error.
server.listen({ host : 'localhost:3000' }, () => {});

// Should probably throw an error, whereas it currently does not.
server.listen({ host : 'localhost', port : 3000 }, () => {});

[maclover7]

@sholladay Would it make sense as a first step to add some more documentation about this behavior?

[PatrickHeneise]

What's the expected behaviour?

[sholladay]

@maclover7 Perhaps calling out the inconsistency would be nice. I don't find the documentation to be bad, per se, though. It's the API design that should be improved.

@PatrickHeneise The expected behavior is that host and hostname have consistent definitions throughout Node's API surface. I expanded upon my examples above, so hopefully it is more clear now. Also, here is a demo of incompatibility with the whatwg/url spec. Does this help?

const { URL } = require('url');
const server = require('net').createServer();
const { host, hostname, port } = new URL('http://localhost:3000');
server.listen({ host }, () => {});  // errors
server.listen({ hostname, port }, () => {});  // listens on the wrong hostname

[bnoordhuis]

The { host : 'localhost' } error message would be good to fix but the other "expected behaviors" could potentially create (possibly severe) ecosystem fallout for not enough value to take the risk.

Perhaps a good middle ground is to follow what http.request() does and prefer .hostname over .host, that makes url.parse() and new url.URL() work transparently.

[sholladay]

@bnoordhuis nothing I am asking for here should be a breaking change, as far as I can tell.

• All host values that Node currently accepts could still be valid, no need for users to do anything.
• The host option, if you want to keep it around for backwards compat (as mentioned above), would be extended to also allow ports in it (not require them).
• A new, discrete hostname option could be introduced, which would be exactly the same as the current host option.

There seems to be a clear path towards whatwg URL compatibility without any breakage changes.

[bnoordhuis]

The host option, if you want to keep it around for backwards compat (as mentioned above), would be extended to also allow ports in it

Whether that can be done in a backwards compatible way depends on whether host+port strings can always be unambiguously distinguished from strings that just look like them. I'm thinking of strings like abba::1972 that are valid IPv6 addresses.

(I made ^ up on the spot. More realistic examples can be construed with a bit more effort.)

[sholladay]

From what I can tell playing around in the REPL, Node doesn't allow : in the host option today. I could be wrong, though. Do you have any actual examples?

http.createServer().listen({ host : 'abba::1972' })
Error: Invalid listen argument: { host: 'abba::1972' }

http.createServer().listen({ host : '::1' })
Error: Invalid listen argument: { host: '::1' }

[bnoordhuis]

That's the other thing you commented on, the mandatory .port property. =)

(Touches on another issue: what if .host and .port both contain port numbers? What if they don't agree?)

[sholladay]

Oh, right. True enough... the error message is misleading. At any rate, from what I've read, part of the purpose of the URL spec was to clean up the grammar. I imagine they have a good way to distinguish IPv6 and such from hosts with ports. But I confess I don't know the details of how that works. Since Node already has the URL parser implemented, perhaps it should just use that directly and not worry about re-implementing anything?

As for conflicting input about the port number, here again I would go for consistency with the existing APIs. I don't think WHATWG provides any functionality that is prone to that, but Node's url.format() does and it treats host as taking precedence.

url.format({ host : 'localhost:4000', port : 5000 });
// 'localhost:4000'

It could even do something like const { hostname, port } = new url.URL(url.format(...args)) if you really wanted to get fancy.

[tswaters]

Using url parser directly won't really work without the other bits to make a valid url... e.g.:

const myUrl = new url.URL(url.format({host: 'localhost:3000'}))
myUrl.protocol // "localhost:"
myUrl.pathname // "3000"

It would need a dummy protocol (or maybe just http) to pull out hostname/port/host properly. And, even then, it wouldn't work for the current case, passing both host and port:

const myUrl = new url.URL(url.format({protocol: 'http', host: 'localhost', port: 4000}))
myUrl.host     // "localhost"
myUrl.hostname // "localhost"
myUrl.port     // "" - port is lost when `host` provided over `hostname`

[sholladay]

I'm not necessarily asking for Node itself to directly use the WHATWG URL parser. That is an implementation detail that may or may not be appropriate (though it seems attractive, if it can be made to work).

Personally, I just want one of the following to correctly listen on localhost port 3000 without reporting an error.

const { hostname, port } = new URL('http://localhost:3000');
server.listen({ hostname, port }, () => {});

const { host } = new URL('http://localhost:3000');
server.listen({ host }, () => {});

[Trott]

Personally, I just want one of the following to correctly listen on localhost port 3000 without reporting an error.

const { hostname, port } = new URL('http://localhost:3000');
server.listen({ hostname, port }, () => {});

const { host } = new URL('http://localhost:3000');
server.listen({ host }, () => {});

@sholladay If I'm not mistaken, the first example now works as expected in Node.js 10 and 11. Is that sufficient to close this issue?

[sholladay]

@Trott I'm not able to get either to work as expected on Node 11 (haven't tried 10). The first example still seems to listen on any address instead of localhost. Here's the result of my test in the REPL on Node 11.0.0:

const { hostname, port } = new URL('http://localhost:3000');
const server = http.createServer();
server.listen({ hostname, port });
server.address();
// => { address: '::', family: 'IPv6', port: 3000 }

And for the other example:

const { host } = new URL('https://localhost:3000');
const server = http.createServer();
server.listen({ host });
// TypeError [ERR_INVALID_ARG_VALUE]: The argument 'options' must have the property "port" or "path". Received { host: 'localhost:3000' }

The behavior of that first example is unsafe in the sense that it could easily lead to people exposing their servers publicly without intending to. That said, Node's documentation explicitly states that the option for .listen() is host instead of hostname, so this would technically be user error. Still, I think Node should conform to one of those two examples and listen on localhost, as expected.

[sam-github]

@sholladay That's not what I see on 11.1.0, did you mistype your example?

Now using node v11.1.0 (npm v6.4.1)
> const { hostname, port } = new URL('http://localhost:3000');
undefined
> hostname
'localhost'
> port
'3000'
> const server = http.createServer();
undefined
> server.address();
null

I'm not sure what you expected in the above. You parse a URL, but don't pass the host or port from the URL to server.listen(), why should the server's address have any relationship to the URL?

const { host } = new URL('http://localhost:3000');
server.listen({ host }, () => {});

This example parses a URL, extracts the host (and discards the port), then listens on the host and default port (not 3000, which was never extracted from the url).

I'm not sure what the expectations are here, and I think some of the examples might have typos, which confuses things.

[sholladay]

@sam-github apologies, I missed an important line when copy/pasting from my terminal back to GitHub. Needless to say, the URL stuff is only relevant if you pass the data to .listen(). 😅

Please see my updated comment above. I'm trying to demonstrate that the APIs are asymmetrical and this leads to surprising, confusing, and arguably unsafe behavior.

[sam-github]

Interestingly, I misread your example, because it didn't occur to me that host would be anything other than the name of the host... Expectations are a tricky thing.

So, do you expect host to always have a name and a port, or just sometimes? That is, is it an optional component?

Node itself mostly uses host to be a name, and port to be a number. The URL API is an exception, because it was externally specified. This is something I personally find completely expected, but expectations depend on background.

There have been periodic suggestions that in the various listen/connect APIs that if the host arg has a :port at the end that it be parsed out. I think this would be accepted if someone did the work (just my impression). If host had a port in it, and the port was also set, the behaviour would have to be specifed (maybe port wins?). This overloading of host has the interesting property that its a subtle change that could allow a port to be smuggled into the API by an attacker if the application doesn't validate the input sufficiently, but I don't think that would block the change.

Also, I just checked, and I think a host of "localhost:3000", since its not a valid DNS name, is interpreted as a named socket.... That might make it hard to change the meaning of host (or at least make it semver-major).

Those are some of my thoughts. I can understand your critique, but its not clear to me what you think should be done. You want host removed? hostname supported as an alias for host? host changed to allow a port in it? Something else?

[sholladay]

I wrote a section about my expectations in the original comment at the top of the thread, which I've edited a few times for clarity. Might help to re-read that. But in short: I like almost everything about the existing .listen() APi and its semantics, except for the fact that the option is currently named host. My expectation is that it should be renamed to hostname. The rest of my comment is essentially my musings on how to do that in a backwards compatible manner, avoiding a breaking change, without confusing people. I claim that we should probably keep an option named host to avoid breaking things, but in doing so, I feel it is important to match WHATWG URL (and others) to avoid confusion. Those other APIs consider host to have a slightly different meaning than hostname. And I believe Node can match those APIs in a backwards compatible manner. Though, @bnoordhuis had some concerns that are a little outside of my area of expertise.

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[github-actions]

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.