Resolving NS records against manual configured servers fails (ENODATA)

[f3lang]

• Version: v14.4.0
• Platform: Linux wolfgang-ThinkPad-P51 5.4.0-31-generic net: give better error messages #35-Ubuntu SMP Thu May 7 20:20:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: dns

What steps will reproduce the bug?

First, you need a DNS zone with NS records delegating a subdomain to another DNS zone.
I set up an example on our domain:
root domain: ventx.de
delegated subdomain: delegation-test.ventx.de
The zone ventx.de contains a NS record "delegation-test.ventx.de" with this content:

ns-1926.awsdns-48.co.uk.
ns-670.awsdns-19.net.
ns-282.awsdns-35.com.
ns-1527.awsdns-62.org.

This can be tested with dig delegation-test.ventx.de NS @ns-675.awsdns-20.net, which results in the very same output, so the zone is setup correctly.
For the following description I'll use the IP address of ns-675.awsdns-20.net, which is 205.251.194.163

When you try to replicate this query with nodejs, an ENODATA error is thrown:

Error: queryNs ENODATA delegation-test.ventx.de
    at QueryReqWrap.onresolve [as oncomplete] (dns.js:203:19) {
  errno: undefined,
  code: 'ENODATA',
  syscall: 'queryNs',
  hostname: 'delegation-test.ventx.de'
} undefined

This is the output of the following code-snippet:

const {Resolver} = require('dns');
const resolver = new Resolver();
resolver.setServers(['205.251.194.163']);

resolver.resolve('delegation-test.ventx.de', 'NS', (err, resolve) => {
	console.log(err, resolve);
});

I debugged this also with wireshark and was able to see, that the actual query to the nameserver
is correct and also the response is coming back with the correct recordSet:

As you can see, the response has the status "No error", however nodejs throws an ENODATA error.

How often does it reproduce? Is there a required condition?

I was able to reproduce the bug everytime without any other condition

What is the expected behavior?

null [
  'ns-670.awsdns-19.net',
  'ns-282.awsdns-35.com',
  'ns-1926.awsdns-48.co.uk',
  'ns-1527.awsdns-62.org'
]

that's how the output of the snippet looks like, when you to not manually set a server

What do you see instead?

Error: queryNs ENODATA delegation-test.ventx.de
    at QueryReqWrap.onresolve [as oncomplete] (dns.js:203:19) {
  errno: undefined,
  code: 'ENODATA',
  syscall: 'queryNs',
  hostname: 'delegation-test.ventx.de'
} undefined

Additional information

I was also able to reproduce the issue with the node:latest docker container:

 $ node -v
 v14.4.0
 $ uname -a
 Linux runner-6ab0c1ba-project-16-concurrent-0 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 GNU/Linux
 $ node dns_fail.js
 Error: queryNs ENODATA delegation-test.ventx.de
     at QueryReqWrap.onresolve [as oncomplete] (dns.js:203:19) {
   errno: undefined,
   code: 'ENODATA',
   syscall: 'queryNs',
   hostname: 'delegation-test.ventx.de'
 } undefined

[bnoordhuis]

I can confirm what you're seeing but I don't think this is a Node.js bug. It might be a c-ares bug (the library Node.js uses to perform the NS query) but my hunch is that it's a problem with the DNS server.

Here is the response I get back:

(gdb) b qcallback
Breakpoint 1 at 0x555556c68d70: file ../deps/cares/src/ares_query.c, line 148.
(gdb) r
Starting program: /home/bnoordhuis/src/master/out/Release/node tmp/bug33858.js
# <elided>
Thread 1 "node" hit Breakpoint 1, qcallback (arg=0x5555596e16a0, status=0, timeouts=0, abuf=0x7fffffff9910 <incomplete sequence \375\216\201>, alen=182)
    at ../deps/cares/src/ares_query.c:148
148     {
(gdb) p {char[182]} abuf
$1 = "\375\216\201\000\000\001\000\000\000\004\000\000\017delegation-test\005ventx\002de\000\000\002\000\001\300\f\000\002\000\001\000\000\001,\000\027\ans-1527\tawsdns-62\003org\000\300\f\000\002\000\001\000\000\001,\000\031\ans-1926\tawsdns-48\002co\002uk\000\300\f\000\002\000\001\000\000\001,\000\026\006ns-282\tawsdns-35\003com\000\300\f\000\002\000\001\000\000\001,\000\026\006ns-670\tawsdns-19\003net"

Note how the answer count field is zero, even though there are clearly answers in the response packet:

(gdb) p/x {char[2]} (abuf+6)
$2 = {0x0, 0x0}

The server acknowledges the query though because the question count field is one:

(gdb) p/x {char[2]} (abuf+4)
$3 = {0x0, 0x1}

I didn't, ah, dig too deeply into what dig(1) does differently but on my system (and yours probably too since you use ubuntu too) it also consults systemd's 127.0.0.53 resolver:

$ strace -e connect,recvfrom,sendmmsg -s 256 dig delegation-test.ventx.de NS @ns-675.awsdns-20.net
connect(7, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(6, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
connect(6, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, 16) = 0
sendmmsg(6, [{msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="W\226\1\0\0\1\0\0\0\0\0\1\6ns-675\tawsdns-20\3net\0\0\1\0\1\0\0)\4\260\0\0\0\0\0\0", iov_len=49}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, msg_len=49}, {msg_hdr={msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\301\224\1\0\0\1\0\0\0\0\0\1\6ns-675\tawsdns-20\3net\0\0\34\0\1\0\0)\4\260\0\0\0\0\0\0", iov_len=49}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, msg_len=49}], 2, MSG_NOSIGNAL) = 2
recvfrom(6, "W\226\201\200\0\1\0\1\0\0\0\1\6ns-675\tawsdns-20\3net\0\0\1\0\1\300\f\0\1\0\1\0\0\27\332\0\4\315\373\302\243\0\0)\377\326\0\0\0\0\0\0", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, [28->16]) = 65
recvfrom(6, "\301\224\201\200\0\1\0\1\0\0\0\1\6ns-675\tawsdns-20\3net\0\0\34\0\1\300\f\0\34\0\1\0\0\27\332\0\20&\0\220\0S\2\243\0\0\0\0\0\0\0\0\1\0\0)\377\326\0\0\0\0\0\0", 65536, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, [28->16]) = 77
connect(6, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("205.251.194.163")}, 16) = 0
connect(6, {sa_family=AF_INET6, sin6_port=htons(0), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "2600:9000:5302:a300::1", &sin6_addr), sin6_scope_id=0}, 28) = -1 ENETUNREACH (Network is unreachable)
# <elided>

I'm going to close this as not-our-bug but let me know if you have reason to believe it's a Node.js issue after all. You could try filing a bug report with c-ares.

[iatpatelishan]

I was trying to do the same thing and found the same issue with dns implementation of nodejs. I dont know why it's closed as 'not-our-bug' if the nodejs official library is not returning the correct result (doesnt matter if it uses other library underneath which might have bug).

[stjosh]

Stumbled over this issue as well and did a bit of research. It seems to me that when for NS records at the nameservers of the parent zone, Route53 does include the NS records in the DNS response's AUTHORITY section, not in the ANSWER section. This is also the case in @f3lang's description above. The NS records are then included in the response's ANSWER section for queries targeted at the nameservers of the sub-zone (i.e., those contained in the NS records of the parent zone).

So my interpretation is that Node only returns the NS reccords if the response was contained in the ANSWER section and seems to ignore anything contained in the AUTHORITY section.

However, I'm not sure who is now misbehaving - would have to dig deeper in the RFCs for that. There are multiple possibilities:

• Route53 misbehaves because it does not respond with the NS records contained in the ANSWER section.
• c-ares misbehaves because it ignores the nameservers returned in the AUTHORITY section.
• Node misbehaves because it ignores what c-ares returned in its AUTHORITY section.
• The end-user misbehaves by expecting node to return things contained in the AUTHORITY section.

A quick (and honestly, quite superficial) look into the c-ares source code gives me the impression that c-ares replies with ARES_NODATA if the answer count is zero, which is exactly what we are seeing with Route53's answers containing nothing in the ANSWER section.

Personally, I think this is an issue specific to how Route53 (correctly?) handles replying to NS queries and the expectations of the end-user. Getting back the contents of the AUTHORITY section in Node would be nice, but I guess there are limited use-cases for this and it would make the API unnecessarily complex for an edge-case.

A workaround for the initial issue seems simple: Just manually set the nameservers of the sub zone when querying, not the parent zone. Or don't set the nameservers manually at all (however, I guess there's a reason why you do set them manually).

[bradh352]

FYI, dig and the c-ares adig return the same results. There's no answer section in either, only authority. You need to use the right apis in c-ares to extract the authority section which most people have no desire to retrieve.

dig delegation-test.ventx.de NS @ns-675.awsdns-20.net

; <<>> DiG 9.10.6 <<>> delegation-test.ventx.de NS @ns-675.awsdns-20.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23856
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;delegation-test.ventx.de.	IN	NS

;; AUTHORITY SECTION:
delegation-test.ventx.de. 300	IN	NS	ns-1527.awsdns-62.org.
delegation-test.ventx.de. 300	IN	NS	ns-1926.awsdns-48.co.uk.
delegation-test.ventx.de. 300	IN	NS	ns-282.awsdns-35.com.
delegation-test.ventx.de. 300	IN	NS	ns-670.awsdns-19.net.

;; Query time: 24 msec
;; SERVER: 2600:9000:5302:a300::1#53(2600:9000:5302:a300::1)
;; WHEN: Tue Oct 12 22:10:06 EDT 2021
;; MSG SIZE  rcvd: 193

adig -s ns-675.awsdns-20.net -t NS delegation-test.ventx.de
DNS server returned answer with no data
id: 41776
flags: qr rd 
opcode: QUERY
rcode: NOERROR
Questions:
	delegation-test.ventx.de.		NS
Answers:
NS records:
	delegation-test.ventx.de.	300	NS	ns-1527.awsdns-62.org.
	delegation-test.ventx.de.	300	NS	ns-1926.awsdns-48.co.uk.
	delegation-test.ventx.de.	300	NS	ns-282.awsdns-35.com.
	delegation-test.ventx.de.	300	NS	ns-670.awsdns-19.net.
Additional records:

[mjsztainbok]

If you query an authorative server, you are going to get an AUTHORITY section and not an ANSWER section. That is what is happening here. The DNS query is against the authoratative server in Route 53 so it is getting back an AUTHORITY seciton and not an ANSWER section. This would also happen if you do a query against another domain with one of the servers from its NS records outside of Route 53.