* Resolver overhaul concepts @ 2014-05-04 12:42 Rich Felker 2014-05-04 16:07 ` Laurent Bercot 0 siblings, 1 reply; 10+ messages in thread From: Rich Felker @ 2014-05-04 12:42 UTC (permalink / raw) To: musl Since resolver overhaul is up on the roadmap for 1.1.2, targetted for sometime this month, I'd like to get a discussion of the proposed design/behavior changes going. The current behavior: 1. Determine a single matching port/protocol (if the protocol is unspecified, only tcp is matched). 2. Special-case passive/local address requests (no host arg). 3. Try host as ip literal. 4. Parse hosts file and return a single (first matching) result from it. (This will fail to produce multiple results, even in AF_UNSPEC case where both v4 and v6 addresses are in the hosts file.) 5. DNS query (in parallel with multiple nameservers and possibly v4/v6, but only a single name lookup) with results going into DNS packet buffers. 6. Count results with a DNS packet pre-parse phase and allocate space. Then parse the packet(s) and fill in the results. (Note: only in the case where this step is reached can multiple results even be returned.) The new behavior: 1. From the beginning, have moderately large fixed-size automatic (stack-based) buffers to store both service and address results into. 2. Find all matching services and store them into the service list. 3. If host is null or an ip literal, store a single address, as appropriate, in the address list. 4. If address list is empty, try hosts file, inserting each matching record into the address list. (A record only matches if the address family matches, possibly with AF_V4MAPPED applied, or if the request is for AF_UNSPEC.) 5. Transform hostname for IDN, if necessary. 6. If the address list is empty, perform DNS queries, but with a new DNS query backend that's more flexible (to be described later) and store results directly to the address list. 7. Possibly filter, transform, or sort address results. (e.g. applying AI_V4MAPPED). 8. Allocate space for cross-product of service list and address list, and if successful, copy the results into the allocated space. The concepts of the new DNS query backend are not really solid yet. One idea is that it should support the "search"/"domain" functionality of resolv.conf to allow querying multiple seach suffixes in parallel and returning as soon as there's a (possibly zero-length) initial run of negative results followed immediately by a positive result. The cleanest way to implement this kind of thing may be using a callback function for writing each packet and for reading the responses; otherwise, storing all the queries and responses as full DNS packets would take an unwantedly-large amount of space. Rich ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 12:42 Resolver overhaul concepts Rich Felker @ 2014-05-04 16:07 ` Laurent Bercot 2014-05-04 16:24 ` Rich Felker 0 siblings, 1 reply; 10+ messages in thread From: Laurent Bercot @ 2014-05-04 16:07 UTC (permalink / raw) To: musl I believe the very first thing to address is what exactly you call a resolver. getaddrinfo() is a horrible interface, and one of the reasons why is that it is loosely designed. Not much is standardized, and it's up to you to decide exactly what to do with it; it's important to be clear about what is implemented, and to document it, because not all applications have the same expectations, and it's very easy to get confused when the resolution path is unexpected. glibc's getaddrinfo() is the entry point to the NSS layer, which can basically implement *any* kind of "name resolution". AFAICT, it's not a goal of musl to reimplement the whole NSS spaghetti monster, but some applications will depend on /etc/nsswitch.conf or something similar; even without supporting /etc/nsswitch.conf, it would be nice to provide a mechanism to selectively enable/disable at least /etc/hosts lookup and DNS lookup. The current resolution policy is hardcoded as "/etc/hosts, then DNS, and nothing else", which is a very sensible default, but probably shouldn't be the only alternative - or if it is, it should be made abundantly clear. > The concepts of the new DNS query backend are not really solid yet. > One idea is that it should support the "search"/"domain" functionality > of resolv.conf to allow querying multiple seach suffixes in parallel > and returning as soon as there's a (possibly zero-length) initial run > of negative results followed immediately by a positive result. The > cleanest way to implement this kind of thing may be using a callback > function for writing each packet and for reading the responses; > otherwise, storing all the queries and responses as full DNS packets > would take an unwantedly-large amount of space. This is the approach I used in s6-dns (src/libs6dns/s6dns_resolveq.c) and it has worked fine for me so far. I don't think the amount of space is a concern here: the typical search line is very short - 3 to 4 suffixes at most. You will have to store the queries anyway to check the responses against them. Another question that comes to mind is the timeout and retry policy. This is network, it's going to suck; this is DNS, it's going to suck even more. getaddrinfo() doesn't allow the user to specify a timeout (yay for unboundedly synchronous network-facing interfaces), so it's up to musl to decide what to do: do you resend a query after a soft timeout ? do you have a hard timeout after which you report failure ? or do you block indefinitely ? Doing network communications the right way (especially with an old and ugly protocol) is complex. It should be way outside the scope of a libc. glibc people have it easy: the DNS part of NSS directly ties into libresolv, so they have a full-fledged resolver to use. I say we should do the same and tie musl to libs6dns. :P -- Laurent, forhttp://skarnet.org/software/s6-dns/getaddrinfo.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 16:07 ` Laurent Bercot @ 2014-05-04 16:24 ` Rich Felker 2014-05-04 17:56 ` Laurent Bercot 2014-05-10 1:04 ` Andy Lutomirski 0 siblings, 2 replies; 10+ messages in thread From: Rich Felker @ 2014-05-04 16:24 UTC (permalink / raw) To: musl On Sun, May 04, 2014 at 05:07:33PM +0100, Laurent Bercot wrote: > > I believe the very first thing to address is what exactly you call > a resolver. There are some legacy dn_*/res_* interfaces in demand which are presently supported only poorly or not at all. Part of the side goal of the resolver overhaul is to provide them cleanly without code duplication. But for the most part, "resolver" means "getaddrinfo" since it is the only standard, non-deprecated interface to name resolution. > getaddrinfo() is a horrible interface, and one of the reasons why > is that it is loosely designed. Not much is standardized, and it's up > to you to decide exactly what to do with it; it's important to be > clear about what is implemented, and to document it, because not all > applications have the same expectations, and it's very easy to get > confused when the resolution path is unexpected. It's standardized by POSIX, and the POSIX text is sufficient to tell you how to use it for all portable usages. Most of the confusion/mess comes from non-conforming implemnentations, particularly in the area of returning wrong error codes. > glibc's getaddrinfo() is the entry point to the NSS layer, which > can basically implement *any* kind of "name resolution". AFAICT, > it's not a goal of musl to reimplement the whole NSS spaghetti > monster, but some applications will depend on /etc/nsswitch.conf > or something similar; even without supporting /etc/nsswitch.conf, > it would be nice to provide a mechanism to selectively enable/disable > at least /etc/hosts lookup and DNS lookup. The current resolution The policy for supporting something like nss has always been that musl implements a perfectly reasonable public protocol for providing any back-end you want: the DNS protocol. You can run a local daemon speaking DNS and serving names from any backend you like, and this is the correct way to achieve it (rather than linking random buggy, likely-not-namespace-clean libraries into the application's address space). In order to make this the most useful, though, musl should support nameservers on non-default ports (is there a standard syntax for this, or can we support one without breaking anything?), and it would also be nice to be able to override resolv.conf on a per-process basis (e.g. via the environment). > policy is hardcoded as "/etc/hosts, then DNS, and nothing else", > which is a very sensible default, but probably shouldn't be the only > alternative - or if it is, it should be made abundantly clear. There was a legacy file, /etc/host.conf, that allowed the order to be changed, but changing the order seems rather useless to me. On the other hand suppressing /etc/hosts could be useful in some instances. > >The concepts of the new DNS query backend are not really solid yet. > >One idea is that it should support the "search"/"domain" functionality > >of resolv.conf to allow querying multiple seach suffixes in parallel > >and returning as soon as there's a (possibly zero-length) initial run > >of negative results followed immediately by a positive result. The > >cleanest way to implement this kind of thing may be using a callback > >function for writing each packet and for reading the responses; > >otherwise, storing all the queries and responses as full DNS packets > >would take an unwantedly-large amount of space. > > This is the approach I used in s6-dns (src/libs6dns/s6dns_resolveq.c) > and it has worked fine for me so far. > I don't think the amount of space is a concern here: the typical > search line is very short - 3 to 4 suffixes at most. You will have > to store the queries anyway to check the responses against them. 4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes 4k to store the responses and up to 2k to store the queries. That's not too bad, but along with the address lists, file buffers, and other stuff getaddrinfo has around, it's getting the stack usage up to the point where getaddrinfo would probably be the biggest stack user in musl, which in turn increases the minimum stack size you need for some usage cases (think: getaddrinfo_a, which makes one thread per query and would like to be able to set the thread stack to one page with no guard page). > Another question that comes to mind is the timeout and retry policy. > This is network, it's going to suck; this is DNS, it's going to suck > even more. getaddrinfo() doesn't allow the user to specify a timeout > (yay for unboundedly synchronous network-facing interfaces), so it's For asynchronous use, you call it from its own thread (or use the getaddrinfo_a extension, which we don't yet provide but which is easy to provide on your own and which I may add to musl since it's convenient and ultra-light). > up to musl to decide what to do: do you resend a query after a soft > timeout ? do you have a hard timeout after which you report failure ? > or do you block indefinitely ? There is presently a hard-coded failure timeout of 5 seconds and a retry time of 1 second. It would be nice to honor settings from resolv.conf to tweak these. > Doing network communications the right way (especially with an old > and ugly protocol) is complex. It should be way outside the scope of > a libc. glibc people have it easy: the DNS part of NSS directly ties > into libresolv, so they have a full-fledged resolver to use. I say > we should do the same and tie musl to libs6dns. :P Using a full-fledged DNS library to provide getaddrinfo is akin to using GMP to provide printf... Rich ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 16:24 ` Rich Felker @ 2014-05-04 17:56 ` Laurent Bercot 2014-05-04 19:04 ` Rich Felker 2014-05-10 1:04 ` Andy Lutomirski 1 sibling, 1 reply; 10+ messages in thread From: Laurent Bercot @ 2014-05-04 17:56 UTC (permalink / raw) To: musl On 04/05/2014 17:24, Rich Felker wrote: > The policy for supporting something like nss has always been that musl > implements a perfectly reasonable public protocol for providing any > back-end you want: the DNS protocol. You can run a local daemon > speaking DNS and serving names from any backend you like, and this is > the correct way to achieve it (rather than linking random buggy, > likely-not-namespace-clean libraries into the application's address > space). That makes sense. > In order to make this the most useful, though, musl should > support nameservers on non-default ports (is there a standard syntax > for this, or can we support one without breaking anything?) I'm not aware of any standardized way of running DNS server/caches on anything else than the default port; but I don't see why it should be necessary. Anyone can run a translator daemon on localhost:53. > and it > would also be nice to be able to override resolv.conf on a per-process > basis (e.g. via the environment). djbdns and s6-dns do this. It makes sense for every resolver to do it too; the problem for a libc is, again, namespace pollution. I suggest having a compile-time option (yes...) that enables musl-specific extensions, among which some environment variables in the MUSL_* namespace. You'll have to accept that, or something similar, at some point. > There was a legacy file, /etc/host.conf, that allowed the order to be > changed, but changing the order seems rather useless to me. On the > other hand suppressing /etc/hosts could be useful in some instances. /etc/host.conf is actually used by libresolv itself. So with glibc, name resolution goes getaddrinfo() -> NSS -> /etc/nsswitch.conf -> /etc/hosts or DNS. If DNS: -> libresolv -> /etc/host.conf -> /etc/hosts or real DNS. That's the magic of glibc configuration, and of compatibility layers upon compatibility layers: /etc/hosts can actually be checked twice ! musl should not check /etc/host.conf itself: that is libresolv internals. A libc-level switching mechanism would be /etc/nsswitch.conf if anything, but parsing /etc/nsswitch.conf is too complex if it's simply about setting 2 boolean flags, so I suggest doing otherwise. > 4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes > 4k to store the responses and up to 2k to store the queries. That's > not too bad, but along with the address lists, file buffers, and other > stuff getaddrinfo has around, it's getting the stack usage up to the > point where getaddrinfo would probably be the biggest stack user in > musl IIRC, the 512 byte limit is only true for UDP responses, and when you get a truncated UDP response you have to retry with TCP, and there the maximum length is much more than 512 bytes. Do you just extract the response from truncated queries ? Anyway getaddrinfo() is authorized to use heap memory, and apart from not handling TCP at all I don't see how you can avoid it. This would, paradoxically, save memory most of the time, because the typical query is short, as well as the typical response; and you're already using heap memory at some point in the current getaddrinfo(), so I don't understand the math of putting everything in the stack. > For asynchronous use, you call it from its own thread (or use the > getaddrinfo_a extension, which we don't yet provide but which is easy > to provide on your own and which I may add to musl since it's > convenient and ultra-light). Making a new thread just to work around a lack of asynchronous interfaces is ugly. (Remember Netscape Navigator's "dns_helper" subprocesses ?) That's the very reason getaddrinfo_a() exists. > There is presently a hard-coded failure timeout of 5 seconds and a > retry time of 1 second. It would be nice to honor settings from > resolv.conf to tweak these. And the RES_TIMEOUT and RES_DFLRETRY environment variables, then, if you're going for libresolv compatibility. > Using a full-fledged DNS library to provide getaddrinfo is akin to > using GMP to provide printf... How so ? All the complex machinery of parsing the DNS protocol, parsing /etc/resolv.conf, talking from/to the network (in an asynchronous manner when "search" is implemented) with a retry policy, both UDP and TCP, and so on, has to be present already. I'm interested in learning the ninja coding techniques that allow you to write getaddrinfo without all that ! :) -- Laurent ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 17:56 ` Laurent Bercot @ 2014-05-04 19:04 ` Rich Felker 2014-05-04 21:32 ` Laurent Bercot 0 siblings, 1 reply; 10+ messages in thread From: Rich Felker @ 2014-05-04 19:04 UTC (permalink / raw) To: musl On Sun, May 04, 2014 at 06:56:59PM +0100, Laurent Bercot wrote: > >In order to make this the most useful, though, musl should > >support nameservers on non-default ports (is there a standard syntax > >for this, or can we support one without breaking anything?) > > I'm not aware of any standardized way of running DNS server/caches > on anything else than the default port; but I don't see why it > should be necessary. Anyone can run a translator daemon on localhost:53. Requiring port 53 is not very prohibitive relative to resolv.conf and nsswitch.conf which are impossible to override without root, but it's slightly worse: it might be a problem if you also need a public DNS on the same machine. Of course if we want to make it possible to override the config on a per-process basis, requiring port 53 is a fairly serious limitation. Note that per-process override would be very nice, if nothing else, as a means of testing; the test framework could setup a custom resolv.conf and generate malicious packets, packets that don't answer the query, etc. to test that libc handles them right. > >and it > >would also be nice to be able to override resolv.conf on a per-process > >basis (e.g. via the environment). > > djbdns and s6-dns do this. It makes sense for every resolver to do it > too; the problem for a libc is, again, namespace pollution. I suggest > having a compile-time option (yes...) that enables musl-specific > extensions, among which some environment variables in the MUSL_* > namespace. You'll have to accept that, or something similar, at some > point. The aim is to use existing mechanisms when available as this facilitates dropping programs into existing, already-configured systems. However it may be necessary at some point to add further options. This is an important topic to discuss at some point, maybe soon. > >There was a legacy file, /etc/host.conf, that allowed the order to be > >changed, but changing the order seems rather useless to me. On the > >other hand suppressing /etc/hosts could be useful in some instances. > > /etc/host.conf is actually used by libresolv itself. So with glibc, > name resolution goes getaddrinfo() -> NSS -> /etc/nsswitch.conf -> > /etc/hosts or DNS. If DNS: -> libresolv -> /etc/host.conf -> > /etc/hosts or real DNS. That's the magic of glibc configuration, and > of compatibility layers upon compatibility layers: /etc/hosts can > actually be checked twice ! > musl should not check /etc/host.conf itself: that is libresolv > internals. Well it's also (historically) a public libc configuration interface. Since musl does not use libresolv (note: glibc doesn't really either, except a seriously-forked version of parts of it) it would need to check this file itself if we wanted to provide the same configuration opportunity. If we do want a way to turn off hosts processing and there's a traditional way to do it via host.conf, I think supporting the traditional way is better than invending a new one. > A libc-level switching mechanism would be /etc/nsswitch.conf > if anything, but parsing /etc/nsswitch.conf is too complex if it's > simply about setting 2 boolean flags, so I suggest doing otherwise. Yes, I don't really want anything to do with nss in musl anyway. :-) > >4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes > >4k to store the responses and up to 2k to store the queries. That's > >not too bad, but along with the address lists, file buffers, and other > >stuff getaddrinfo has around, it's getting the stack usage up to the > >point where getaddrinfo would probably be the biggest stack user in > >musl > > IIRC, the 512 byte limit is only true for UDP responses, and when you > get a truncated UDP response you have to retry with TCP, and there the > maximum length is much more than 512 bytes. Do you just extract the > response from truncated queries ? Yes, tcp is not supported at all. I don't see any reason one would need tcp for a non-recursive resolver. In principle a response just needs a few more bytes than the request, plus 4 bytes per address (or 16 for AAAA), and the request size is bounded just above 256 bytes (the max hostname length). > Anyway getaddrinfo() is authorized > to use heap memory, and apart from not handling TCP at all I don't see > how you can avoid it. This would, paradoxically, save memory most of > the time, because the typical query is short, as well as the typical > response; and you're already using heap memory at some point in the > current getaddrinfo(), so I don't understand the math of putting > everything in the stack. More complexity, more failure cases, and then it also depends on free as opposed to just malloc. Also it avoids additional fragmentation. If you have lots of threads making DNS queries and frequently allocating and freeing small blocks, it's conceivable that the allocation timing ends up breaking up contiguous space that another thread wants (e.g. the other thread has called malloc then calls realloc after the address just past its first allocation is taken). From a standpoint of not making a fragmented mess of the heap, it's best not to make unnecessary use of allocated storage. > >For asynchronous use, you call it from its own thread (or use the > >getaddrinfo_a extension, which we don't yet provide but which is easy > >to provide on your own and which I may add to musl since it's > >convenient and ultra-light). > > Making a new thread just to work around a lack of asynchronous > interfaces is ugly. (Remember Netscape Navigator's "dns_helper" > subprocesses ?) That's the very reason getaddrinfo_a() exists. A round trip network query (even to localhost) takes several times as long as creating a thread (and for tcp, typically takes hundreds of times the resources of thread creation since the kernel allocates massively bloated send/recv buffers). With Netscape's forked dns_helper, there are other costs like error handling complexity when the helper is wrongly killed, etc., but most of those don't apply to threads. > >There is presently a hard-coded failure timeout of 5 seconds and a > >retry time of 1 second. It would be nice to honor settings from > >resolv.conf to tweak these. > > And the RES_TIMEOUT and RES_DFLRETRY environment variables, then, if > you're going for libresolv compatibility. I'm not sure if they have the same semantics, but it's doubtful that anyone cares if they're exactly the same, so we could probably reuse them. resolv.conf also has a mechanism for setting these. > >Using a full-fledged DNS library to provide getaddrinfo is akin to > >using GMP to provide printf... > > How so ? All the complex machinery of parsing the DNS protocol, > parsing /etc/resolv.conf, talking from/to the network (in an asynchronous > manner when "search" is implemented) with a retry policy, both UDP and TCP, > and so on, has to be present already. I'm interested in learning the > ninja coding techniques that allow you to write getaddrinfo without all > that ! :) See my printf analogy. printf needs decimal bignums, but only needs two operations on them: << and >>. A general bignum implementation that can do arbitrary operations on them is a lot more complex and costly than an implementation that just does two operations in-place. Similarly, getaddrinfo needs DNS, but it only needs generation of fixed-form queries, minimal data extraction from result packets, and some degree of validation. FYI the current code is ~4k binary and the overhaul is not expected to increase that much. I really doubt you could achieve that with general DNS library code. Rich ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 19:04 ` Rich Felker @ 2014-05-04 21:32 ` Laurent Bercot 0 siblings, 0 replies; 10+ messages in thread From: Laurent Bercot @ 2014-05-04 21:32 UTC (permalink / raw) To: musl > Requiring port 53 is not very prohibitive relative to resolv.conf and > nsswitch.conf which are impossible to override without root, but it's > slightly worse: it might be a problem if you also need a public DNS on > the same machine. You mean running a resolver and a server on the same machine ? I've been doing that for years with tinydns on the outside and dnscache on 127.0.0.1: there's no reason why a user couldn't do the same with a custom resolution daemon. Of course, providing custom resolution *and* DNS data to the outside world then requires two different public IP addresses, but that's nothing new: using the same port for resolution and data service is a fundamental flaw in the DNS protocol in the first place (and the main reason why mainstream DNS software is so hopelessly monolithic), and a custom resolution daemon won't be in a different position from, say, dnscache. > Of course if we want to make it possible to override the config on a > per-process basis, requiring port 53 is a fairly serious limitation. I don't feel the same way. Name resolution is name resolution; if a resolver, no matter how it resolves, reads and understands client DNS queries, then it makes sense for it to listen on port 53 somewhere (if we forget that DNS data servers also listen on the same port). I understand the desire for flexibility, and I can imagine cases where this would be useful, but I don't see a blatant need for it. Especially with IPv6 around the corner (only two or three decades now), where address space is cheap. > Yes, tcp is not supported at all. I don't see any reason one would > need tcp for a non-recursive resolver. In principle a response just > needs a few more bytes than the request, plus 4 bytes per address (or > 16 for AAAA), and the request size is bounded just above 256 bytes > (the max hostname length). Plus the authority and additional sections, and... oh wait, you only have to handle A and AAAA, which shouldn't have any of those. OK, now I understand how you don't need a full DNS engine. :) Still, I wouldn't bet it's going to remain that way. Having 6 or 7 A fields isn't uncommon nowadays (google.com has 6 in most places). Now imagine 6 or 7 AAAA fields instead: it will begin to seriously flirt with the limit. It won't happen in the near future, but it will happen. > More complexity, more failure cases, and then it also depends on free > as opposed to just malloc. > > Also it avoids additional fragmentation. Don't get me wrong, I'm a huge advocate of using the stack whenever possible. s6-dns actually started when I studied djbdns's client library and went "ewww, there are way too many mallocs in there - I can do better." It's just that for generic DNS responses, there's no way around malloc - but if you don't need to support TCP, then you can store all responses in the stack indeed, and that's a lot of savings. > A round trip network query (even to localhost) takes several times as > long as creating a thread (and for tcp, typically takes hundreds of > times the resources of thread creation since the kernel allocates > massively bloated send/recv buffers). I was thinking ease of use from a programmer's point of view. Creating a thread to perform a simple operation, then join the thread, is not elegant. In Go, this is absolutely the right way of doing things (because goroutines are even lighter than threads - the runtime has its own multiplexer), but in C, something like getaddrinfo_a is easier and more idiomatic. > Similarly, getaddrinfo needs DNS, but it only needs generation of > fixed-form queries, minimal data extraction from result packets, and > some degree of validation. Being able to guarantee that all queries and responses fit into the stack is simply huge. > FYI the current code is ~4k binary and the overhaul is not expected to > increase that much. I really doubt you could achieve that with general > DNS library code. Indeed. My s6-dnsip4 static binary (linked against musl, for x86_64) is just short of 40k, including roughly 25k of DNS code, and I've made it as lean as I could without sacrificing readability. -- Laurent ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Resolver overhaul concepts 2014-05-04 16:24 ` Rich Felker 2014-05-04 17:56 ` Laurent Bercot @ 2014-05-10 1:04 ` Andy Lutomirski 2014-05-10 2:36 ` Rich Felker 1 sibling, 1 reply; 10+ messages in thread From: Andy Lutomirski @ 2014-05-10 1:04 UTC (permalink / raw) To: musl On 05/04/2014 09:24 AM, Rich Felker wrote: > On Sun, May 04, 2014 at 05:07:33PM +0100, Laurent Bercot wrote: >> >> I believe the very first thing to address is what exactly you call >> a resolver. > > There are some legacy dn_*/res_* interfaces in demand which are > presently supported only poorly or not at all. Part of the side goal > of the resolver overhaul is to provide them cleanly without code > duplication. But for the most part, "resolver" means "getaddrinfo" > since it is the only standard, non-deprecated interface to name > resolution. > >> getaddrinfo() is a horrible interface, and one of the reasons why >> is that it is loosely designed. Not much is standardized, and it's up >> to you to decide exactly what to do with it; it's important to be >> clear about what is implemented, and to document it, because not all >> applications have the same expectations, and it's very easy to get >> confused when the resolution path is unexpected. > > It's standardized by POSIX, and the POSIX text is sufficient to tell > you how to use it for all portable usages. Most of the confusion/mess > comes from non-conforming implemnentations, particularly in the area > of returning wrong error codes. > >> glibc's getaddrinfo() is the entry point to the NSS layer, which >> can basically implement *any* kind of "name resolution". AFAICT, >> it's not a goal of musl to reimplement the whole NSS spaghetti >> monster, but some applications will depend on /etc/nsswitch.conf >> or something similar; even without supporting /etc/nsswitch.conf, >> it would be nice to provide a mechanism to selectively enable/disable >> at least /etc/hosts lookup and DNS lookup. The current resolution > > The policy for supporting something like nss has always been that musl > implements a perfectly reasonable public protocol for providing any > back-end you want: the DNS protocol. You can run a local daemon > speaking DNS and serving names from any backend you like, and this is > the correct way to achieve it (rather than linking random buggy, > likely-not-namespace-clean libraries into the application's address > space). In order to make this the most useful, though, musl should > support nameservers on non-default ports (is there a standard syntax > for this, or can we support one without breaking anything?), and it > would also be nice to be able to override resolv.conf on a per-process > basis (e.g. via the environment). How about 'nameserver /path/to/unix/socket'? If glibc supported that, too, it might solve a lot of problems involving systemwide resolvers and containers. This might have to be 'unixsocknameserver /path/to/unix/socket' or something for better interoperability. --Andy ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Re: Resolver overhaul concepts 2014-05-10 1:04 ` Andy Lutomirski @ 2014-05-10 2:36 ` Rich Felker 2014-05-10 9:26 ` Laurent Bercot 0 siblings, 1 reply; 10+ messages in thread From: Rich Felker @ 2014-05-10 2:36 UTC (permalink / raw) To: musl On Fri, May 09, 2014 at 06:04:08PM -0700, Andy Lutomirski wrote: > > The policy for supporting something like nss has always been that musl > > implements a perfectly reasonable public protocol for providing any > > back-end you want: the DNS protocol. You can run a local daemon > > speaking DNS and serving names from any backend you like, and this is > > the correct way to achieve it (rather than linking random buggy, > > likely-not-namespace-clean libraries into the application's address > > space). In order to make this the most useful, though, musl should > > support nameservers on non-default ports (is there a standard syntax > > for this, or can we support one without breaking anything?), and it > > would also be nice to be able to override resolv.conf on a per-process > > basis (e.g. via the environment). > > How about 'nameserver /path/to/unix/socket'? If glibc supported that, > too, it might solve a lot of problems involving systemwide resolvers and > containers. > > This might have to be 'unixsocknameserver /path/to/unix/socket' or > something for better interoperability. I don't see any convincing reason to support this. It greatly complicates the lookup code (having to have 2+ sockets instead of just one, having to support different address/protocol families, ...) and doesn't let you do anything you can't already do with AF_INET[6] and udp. There's also no precedent, which rather defeats the principle of not inventing new mechanisms for something where there's already a fully-general option available. Rich ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Re: Resolver overhaul concepts 2014-05-10 2:36 ` Rich Felker @ 2014-05-10 9:26 ` Laurent Bercot 2014-05-10 17:41 ` Rich Felker 0 siblings, 1 reply; 10+ messages in thread From: Laurent Bercot @ 2014-05-10 9:26 UTC (permalink / raw) To: musl On 10/05/2014 03:36, Rich Felker wrote: > doesn't let you do anything you can't already do with AF_INET[6] and > udp. Nit: it does. (User authentication, fd passing.) But I agree that this is not needed for name resolution and would gratuitously add muslisms. -- Laurent ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Re: Resolver overhaul concepts 2014-05-10 9:26 ` Laurent Bercot @ 2014-05-10 17:41 ` Rich Felker 0 siblings, 0 replies; 10+ messages in thread From: Rich Felker @ 2014-05-10 17:41 UTC (permalink / raw) To: musl On Sat, May 10, 2014 at 10:26:46AM +0100, Laurent Bercot wrote: > On 10/05/2014 03:36, Rich Felker wrote: > >doesn't let you do anything you can't already do with AF_INET[6] and > >udp. > > Nit: it does. (User authentication, fd passing.) But I agree that > this is not needed for name resolution and would gratuitously add > muslisms. I meant in the context of being a resolver back-end, not anything else. For this purpose there is no use in fd passing, and if you really want authentication, iptables (or whatever its successor is) can tag local packets by the originating uid/pid/etc. and provide access controls roughly equivalent to what AF_UNIX could provide (however it seems unlikely that this would be useful either). Rich ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-05-10 17:41 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-05-04 12:42 Resolver overhaul concepts Rich Felker 2014-05-04 16:07 ` Laurent Bercot 2014-05-04 16:24 ` Rich Felker 2014-05-04 17:56 ` Laurent Bercot 2014-05-04 19:04 ` Rich Felker 2014-05-04 21:32 ` Laurent Bercot 2014-05-10 1:04 ` Andy Lutomirski 2014-05-10 2:36 ` Rich Felker 2014-05-10 9:26 ` Laurent Bercot 2014-05-10 17:41 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).