-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace deps/cares with pure-javascript alternative #1013
Comments
I'm in favor of moving to (something like) node-dns.
I'm guessing 'no'. The dependency chain is a few layers deep and some things seem to be more convenient than optimal.
node-dns doesn't support fallback to TCP, like c-ares does. Probably not an issue in everyday use for most.
I'd say that's the best argument for moving away from c-ares. |
+1 |
1 similar comment
+1 |
I will provide a better write up on if I think as it exists today if node-dns should be included. I do want to quickly clarify that it does do both tcp and udp lookups.
|
I'd definitely be in favor of a javascript implementation that exposes more than I have to caution you of dropping in |
Perf implications? |
For hysterical^Whistorical context, allow me to describe why I wrote node-dns/native-dns. At the time I was fed up with a DNS server I was having to maintain that was written in twisted. Having recently fell in love with Node.js I embarked on writing a DNS application server in JavaScript. I started by re-using someone else's module, until I found a bug and realized that (despite what people were distributing it as on npm) the original source of the module was actually unlicensed. So I went about the work to write my own DNS parser and server, and it was evident the same code could be reusable as a client stack as well. At this point I decided I wanted to it to be a drop in replacement for the That's the lens you should look at native-dns with (now maintained by @taoeffect [though I owe him some code reviews]). A module that initially wanted to enable application server style workflows (like the Now to ask the question if I, a maintainer of Node.js, would accept the module I wrote in core? As it stands today, no. There are far too many places native-dns overreaches in scope for what I would consider to be included in core:
Now on to the pieces of the API for native-dns that I think are worthwhile.
Things that fall out of this implementation that I find neat.
Random thoughts I have on this:
Many of the things I like about native-dns could probably find ways of being achieved with other implementations, or even the current c-ares implementation. It's just a matter of understanding what it is you're trying to solve for by doing the work. If you decide you want core to have a pure JavaScript implementation, I would advocate for working from an existing code base that can be morphed into the thing you're looking for. I do believe that native-dns with @taoeffect is a good place to coordinate that effort. That being said, I also know @indutny has a DNS parser that he wrote, and there are a myriad of others that can be found in our ecosystem. Food for thought. |
In general, replacing cares by an all-js implementation is is an idea worth exploring IMO. My preference would be to keep only GetAddrInfo/GetNameInfo (the APIs that net and http rely on) in core, and move the more advanced dns methods to userland. What we need is a transition path.
c-ares afaik doesn't do any caching. I've always seen this as problematic; using node-dns might be a way to resolve this long-standing issue. |
I think the desire for caching mostly stems from use of If we contain the conversation to only If caching were to be considered in scope, there's almost certainly no way for the opinion of core to get it right for what the users needs are. You could do traditional TTL based caching, but the consumer may need more nuanced semantics. All in all, caching is effectively wrapping // error handling, typos, etc all reserved -- this is a paraphrase.
var cache = {};
function lookup(name, cb) {
if (name in cache)
setImmediate(cb, cache[name]);
else
dns.lookup(name, function store(err, ip) {
cache[name] = ip;
cb(err, ip);
});
}
function myconnect(port, name, cb) {
lookup(name, function(err, ip) {
net.connect(port, ip, cb);
});
} Where you can handle your own invalidation, on whatever semantics you want to achieve. native-dns grew a concept about caching because it's related to wanting to provide the semantics of holding records that need to be resolved on the server side, which in some ways behaves similarly to what you might need for forwarder caching. But all in all, out of scope for what a pure resolver needs to function. Splitting DNS resolution out of core is an interesting idea, and indeed fits more in line with the needs of core -- but in my opinion that ship has sailed, the API was included and for many people expected to be there and it (currently) is not too much of a burden to continue to maintain. However, shifting to something that core would be seeking to maintain, that burden could indeed feel too great and decide to not do that at all. |
But c-ares doesn't even expose the TTL that's reported by the DNS server, so implementing sensible custom caching is almost impossible.
That's true, we can't remove the dns module and break people's code. We need a creative solution for the core-to-userland problem however. |
Exposing the TTL in the api would be nice. That is something I've needed in the past. |
On the topic of a DNS implementation, have any of you had a chance to evaluate/look at/consider getdns-node? It's node bindings for getdns. Re: node-dns, it needs a good amount of simplification and cleanup IMO. I especially want to see fast packet relay that avoids parsing packets unless necessary (see issues tjfontaine/node-dns#67 and tjfontaine/node-dns#69). If the C++ from getdns can be integrated into node, then it might be a reasonable alternative (I still need to evaluate it more though to be sure). While I don't like C++, the advantage is that it seems to be a fairly mature library that's being actively maintained by another group. |
(Note to email subscribers: I edited the comment I just posted. So visit GitHub to see the changes.) |
@taoeffect I think a C++ implementation is out of question for io.js at this point. There's a huge risk of it ending up in a stale state like c-ares is right now, where only a couple people are able/willing to dive into it to fix issues like #894. Having a clean and well-tested DNS implementation in JS would probably encourage a lot more people to maintain it, I'm just afraid that it may not yet be written. |
Hoping that there is a pool of people who know DNS and DNSSEC well enough to maintain a pure JS implementation, but don't know C, seems optimistic. Also, a pure-js DNS library would be used by nothing but iojs, so would be entirely supported by iojs, and be unlikely to have DNSSEC support. getdns looks like an interesting alternative to c-ares, and it is built to work off an event loop (it even has libuv support), so it might be a more modern and better supported alternative to a pure-js implementation, though I don't know how much pain C-ARES is actually causing. It seems pretty stable. |
Just adding this to the discussion here: Setting per-query options like |
Throwing my 2 cents here :-)
AFAIS, it does report the TTL, but just for A and AAAA. Not ideal, but it's something. https://github.com/piscisaureus/cares/blob/master/src/ares_parse_a_reply.c#L49 I've been following getdns for a while and indeed it does have a nicer API, more features (DNSSEC for one) and it's actively developed. c-ares OTOH: http://c-ares.haxx.se/mail/c-ares-archive-2014-05/0000.shtml FWIW, one thing that could benefit some people would be the ability to create multiple resolvers (aka ares_channel things) since all options apply to those:
In addition, lookup / lookupService could be deprecated in favor of getaddrinfo / getnameinfo, since they use the system resolver, and that's what they really are inside. Of course, any instance of Resolver wouldn't have those, they'd be available just in the global dns module. The Resolver API could be designed in such a way that replacing the guts with getdns or something else doesn't change the user facing API, hopefully. |
Yeah, I'd prefer not to introduce the complexity of having to instantiate a resolver like in your example. I'd prefer adding an option argument to TTLs would be a great addition, but I think it's not worth adding the feature when it won't work for every rrtype. |
FWIW I have a pure js dns resolver in the works that is more or less done now (still missing tests and backwards-compatible error objects though). It handles configuration file parsing, Look for a PR to add this resolver in the near future.... |
@mscdex awesome, very eager to review that ;) |
+1 |
@nodejs/collaborators While working on error compatibility, I saw a comment in dns.js by @bnoordhuis that mentioned wanting to return the actual DNS error code/name instead of some of c-ares' custom error codes/names. For example, if you query a nameserver for a non-existent domain, you will get back My question is should I be making that switch to real DNS errors (e.g. |
Would it be possible to stick with what c-ares does in the short term, to reduce the perceived changes and then possibly transition to "real" DNS over a few major version bumps? |
@cjihrig as with the http parser, I think behind a flag and phased in was discussed, maybe that could be done ontop of what you suggested? |
Sounds good to me 👍 |
+1 for real DNS errors. I'd prefer the dns implementation landing in a I don't see much use in a transition period for errors, either we break them or we don't. |
Regarding flags, I currently have it so that the old DNS functionality is behind a flag ( I think I agree with @silverwind about the errors though. If they need the old errors they could just use the old DNS functionality? I dunno... The exposure of TTL and other data is a separate issue and I'm not sure of the best way to make that information (optionally) available. |
Yeah, that's a bit tricky given the current API. We could possibly return an array of TTLs in a third callback parameter, or go the route of adding something like |
Very positive towards exposing the real DNS error. |
Most of the official error codes aren't all-caps like our errors, should we still caps them like |
@silverwind to avoid case comparison issues wouldn't the error code (rcode) be more beneficial to use [as a user]? That way we can at least say we follow rfc. |
@silverwind The error field in a DNS packet is just a number field, so technically you could use any string name you want. However our DNS module currently uses all caps for its error names. I would probably keep it that same style like |
+1 to continuing to push in this direction. Great to see additional work moving forward |
ref: #1843 |
+1 to get this out asap. This is a big barrier for us using io.js currently. |
@snsparrish shouldn't be more of a barrier than any other node version though? |
@snsparrish can you clarify the barrier for us? This is basically about improving on very long-term Node technical debt, not fixing anything specific to io.js. Or are you talking about a barrier to Node in general? |
We are currently using node-dns on older versions of node.js but that module doesn't appear to function in io.js 2.2.1. We need the ability to query multiple nameservers without the crash mentioned in #894. But as I understand it, even if setServers didn't crash, the resolver "stored in a per-app global state" keeps me from being able to query different nameservers at the same time in the same app. We can't upgrade to io.js from our current outdated node.js versions without that ability. |
This is correct, right now you must do such queries to multiple servers serially because of the crashing issue. I plan on introducing a async-friendly option to |
Applying |
Given the lack of further activity and discussion, and the lack of any response to adding the stalled label, going to go ahead and close this. It's possible that we'd still want to pursue this but there's no reason to keep this particular issue open. |
There is an alternate library |
Uh, yes it does. |
Tat function just does some prints, that's not usable. The TTL is only On Oct 15, 2016 20:43, "Cal Leeming" notifications@github.com wrote:
|
Ohh, damn they should make that more clear in the README. You might have just saved me a lot of headache, thank you. |
It has been 2 or 3 years and I am now experiencing a horrible time compiling/cross compiling node.js for my android device, with bunches of |
This has been brought up a time or two, so I just thought I'd get a discussion going whether we actually want to do this.
We've somewhat moved away from upstream c-ares. We're not abi compatible any longer which made us remove supporting building against a shared library. The way I see it; we could avoid complexity and c++ by replacing c-ares with a pure-javascript alternative.
The most likely contender for this spot would be node-dns by @tjfontaine which is meant to be a drop-in replacement. It has additional functionality such as a simple dns server which we may (or likely not) be interested in adopting. It has a few dependencies of it's own I'm unsure we'd want to have maintained externally. There's also been a few issues mentioned that needs to be resolved [todo: be more specific].
Before putting too much work in this, are there counter-arguments against such a transition -- or should we perhaps turn it around; why move at all? My main arguments would be that the benefits of c++ doesn't give us much here, as well as not having to stay close to c-ares (for good or bad). It would probably also allow more people to chip in in terms of maintenance.
The text was updated successfully, but these errors were encountered: