Resolver overhaul concepts

Resolver overhaul concepts

[Rich Felker]

Since resolver overhaul is up on the roadmap for 1.1.2, targetted for
sometime this month, I'd like to get a discussion of the proposed
design/behavior changes going.


The current behavior:

1. Determine a single matching port/protocol (if the protocol is
unspecified, only tcp is matched).

2. Special-case passive/local address requests (no host arg).

3. Try host as ip literal.

4. Parse hosts file and return a single (first matching) result from
it. (This will fail to produce multiple results, even in AF_UNSPEC
case where both v4 and v6 addresses are in the hosts file.)

5. DNS query (in parallel with multiple nameservers and possibly
v4/v6, but only a single name lookup) with results going into DNS
packet buffers.

6. Count results with a DNS packet pre-parse phase and allocate space.
Then parse the packet(s) and fill in the results. (Note: only in the
case where this step is reached can multiple results even be
returned.)


The new behavior:

1. From the beginning, have moderately large fixed-size automatic
(stack-based) buffers to store both service and address results into.

2. Find all matching services and store them into the service list.

3. If host is null or an ip literal, store a single address, as
appropriate, in the address list.

4. If address list is empty, try hosts file, inserting each matching
record into the address list. (A record only matches if the address
family matches, possibly with AF_V4MAPPED applied, or if the request
is for AF_UNSPEC.)

5. Transform hostname for IDN, if necessary.

6. If the address list is empty, perform DNS queries, but with a new
DNS query backend that's more flexible (to be described later) and
store results directly to the address list.

7. Possibly filter, transform, or sort address results. (e.g. applying
AI_V4MAPPED).

8. Allocate space for cross-product of service list and address list,
and if successful, copy the results into the allocated space.


The concepts of the new DNS query backend are not really solid yet.
One idea is that it should support the "search"/"domain" functionality
of resolv.conf to allow querying multiple seach suffixes in parallel
and returning as soon as there's a (possibly zero-length) initial run
of negative results followed immediately by a positive result. The
cleanest way to implement this kind of thing may be using a callback
function for writing each packet and for reading the responses;
otherwise, storing all the queries and responses as full DNS packets
would take an unwantedly-large amount of space.

Rich

Re: Resolver overhaul concepts

[Laurent Bercot]

  I believe the very first thing to address is what exactly you call
a resolver.
  getaddrinfo() is a horrible interface, and one of the reasons why
is that it is loosely designed. Not much is standardized, and it's up
to you to decide exactly what to do with it; it's important to be
clear about what is implemented, and to document it, because not all
applications have the same expectations, and it's very easy to get
confused when the resolution path is unexpected.

  glibc's getaddrinfo() is the entry point to the NSS layer, which
can basically implement *any* kind of "name resolution". AFAICT,
it's not a goal of musl to reimplement the whole NSS spaghetti
monster, but some applications will depend on /etc/nsswitch.conf
or something similar; even without supporting /etc/nsswitch.conf,
it would be nice to provide a mechanism to selectively enable/disable
at least /etc/hosts lookup and DNS lookup. The current resolution
policy is hardcoded as "/etc/hosts, then DNS, and nothing else",
which is a very sensible default, but probably shouldn't be the only
alternative - or if it is, it should be made abundantly clear.


> The concepts of the new DNS query backend are not really solid yet.
> One idea is that it should support the "search"/"domain" functionality
> of resolv.conf to allow querying multiple seach suffixes in parallel
> and returning as soon as there's a (possibly zero-length) initial run
> of negative results followed immediately by a positive result. The
> cleanest way to implement this kind of thing may be using a callback
> function for writing each packet and for reading the responses;
> otherwise, storing all the queries and responses as full DNS packets
> would take an unwantedly-large amount of space.

  This is the approach I used in s6-dns (src/libs6dns/s6dns_resolveq.c)
and it has worked fine for me so far.
  I don't think the amount of space is a concern here: the typical
search line is very short - 3 to 4 suffixes at most. You will have
to store the queries anyway to check the responses against them.

  Another question that comes to mind is the timeout and retry policy.
This is network, it's going to suck; this is DNS, it's going to suck
even more. getaddrinfo() doesn't allow the user to specify a timeout
(yay for unboundedly synchronous network-facing interfaces), so it's
up to musl to decide what to do: do you resend a query after a soft
timeout ? do you have a hard timeout after which you report failure ?
or do you block indefinitely ?

  Doing network communications the right way (especially with an old
and ugly protocol) is complex. It should be way outside the scope of
a libc. glibc people have it easy: the DNS part of NSS directly ties
into libresolv, so they have a full-fledged resolver to use. I say
we should do the same and tie musl to libs6dns. :P

-- 
  Laurent, forhttp://skarnet.org/software/s6-dns/getaddrinfo.html

Re: Resolver overhaul concepts

[Rich Felker]

On Sun, May 04, 2014 at 05:07:33PM +0100, Laurent Bercot wrote:
> 
>  I believe the very first thing to address is what exactly you call
> a resolver.

There are some legacy dn_*/res_* interfaces in demand which are
presently supported only poorly or not at all. Part of the side goal
of the resolver overhaul is to provide them cleanly without code
duplication. But for the most part, "resolver" means "getaddrinfo"
since it is the only standard, non-deprecated interface to name
resolution.

>  getaddrinfo() is a horrible interface, and one of the reasons why
> is that it is loosely designed. Not much is standardized, and it's up
> to you to decide exactly what to do with it; it's important to be
> clear about what is implemented, and to document it, because not all
> applications have the same expectations, and it's very easy to get
> confused when the resolution path is unexpected.

It's standardized by POSIX, and the POSIX text is sufficient to tell
you how to use it for all portable usages. Most of the confusion/mess
comes from non-conforming implemnentations, particularly in the area
of returning wrong error codes.

>  glibc's getaddrinfo() is the entry point to the NSS layer, which
> can basically implement *any* kind of "name resolution". AFAICT,
> it's not a goal of musl to reimplement the whole NSS spaghetti
> monster, but some applications will depend on /etc/nsswitch.conf
> or something similar; even without supporting /etc/nsswitch.conf,
> it would be nice to provide a mechanism to selectively enable/disable
> at least /etc/hosts lookup and DNS lookup. The current resolution

The policy for supporting something like nss has always been that musl
implements a perfectly reasonable public protocol for providing any
back-end you want: the DNS protocol. You can run a local daemon
speaking DNS and serving names from any backend you like, and this is
the correct way to achieve it (rather than linking random buggy,
likely-not-namespace-clean libraries into the application's address
space). In order to make this the most useful, though, musl should
support nameservers on non-default ports (is there a standard syntax
for this, or can we support one without breaking anything?), and it
would also be nice to be able to override resolv.conf on a per-process
basis (e.g. via the environment).

> policy is hardcoded as "/etc/hosts, then DNS, and nothing else",
> which is a very sensible default, but probably shouldn't be the only
> alternative - or if it is, it should be made abundantly clear.

There was a legacy file, /etc/host.conf, that allowed the order to be
changed, but changing the order seems rather useless to me. On the
other hand suppressing /etc/hosts could be useful in some instances.

> >The concepts of the new DNS query backend are not really solid yet.
> >One idea is that it should support the "search"/"domain" functionality
> >of resolv.conf to allow querying multiple seach suffixes in parallel
> >and returning as soon as there's a (possibly zero-length) initial run
> >of negative results followed immediately by a positive result. The
> >cleanest way to implement this kind of thing may be using a callback
> >function for writing each packet and for reading the responses;
> >otherwise, storing all the queries and responses as full DNS packets
> >would take an unwantedly-large amount of space.
> 
>  This is the approach I used in s6-dns (src/libs6dns/s6dns_resolveq.c)
> and it has worked fine for me so far.
>  I don't think the amount of space is a concern here: the typical
> search line is very short - 3 to 4 suffixes at most. You will have
> to store the queries anyway to check the responses against them.

4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes
4k to store the responses and up to 2k to store the queries. That's
not too bad, but along with the address lists, file buffers, and other
stuff getaddrinfo has around, it's getting the stack usage up to the
point where getaddrinfo would probably be the biggest stack user in
musl, which in turn increases the minimum stack size you need for some
usage cases (think: getaddrinfo_a, which makes one thread per query
and would like to be able to set the thread stack to one page with no
guard page).

>  Another question that comes to mind is the timeout and retry policy.
> This is network, it's going to suck; this is DNS, it's going to suck
> even more. getaddrinfo() doesn't allow the user to specify a timeout
> (yay for unboundedly synchronous network-facing interfaces), so it's

For asynchronous use, you call it from its own thread (or use the
getaddrinfo_a extension, which we don't yet provide but which is easy
to provide on your own and which I may add to musl since it's
convenient and ultra-light).

> up to musl to decide what to do: do you resend a query after a soft
> timeout ? do you have a hard timeout after which you report failure ?
> or do you block indefinitely ?

There is presently a hard-coded failure timeout of 5 seconds and a
retry time of 1 second. It would be nice to honor settings from
resolv.conf to tweak these.

>  Doing network communications the right way (especially with an old
> and ugly protocol) is complex. It should be way outside the scope of
> a libc. glibc people have it easy: the DNS part of NSS directly ties
> into libresolv, so they have a full-fledged resolver to use. I say
> we should do the same and tie musl to libs6dns. :P

Using a full-fledged DNS library to provide getaddrinfo is akin to
using GMP to provide printf...

Rich

Re: Resolver overhaul concepts

[Laurent Bercot]

On 04/05/2014 17:24, Rich Felker wrote:
> The policy for supporting something like nss has always been that musl
> implements a perfectly reasonable public protocol for providing any
> back-end you want: the DNS protocol. You can run a local daemon
> speaking DNS and serving names from any backend you like, and this is
> the correct way to achieve it (rather than linking random buggy,
> likely-not-namespace-clean libraries into the application's address
> space).

  That makes sense.


> In order to make this the most useful, though, musl should
> support nameservers on non-default ports (is there a standard syntax
> for this, or can we support one without breaking anything?)

  I'm not aware of any standardized way of running DNS server/caches
on anything else than the default port; but I don't see why it
should be necessary. Anyone can run a translator daemon on localhost:53.


> and it
> would also be nice to be able to override resolv.conf on a per-process
> basis (e.g. via the environment).

  djbdns and s6-dns do this. It makes sense for every resolver to do it
too; the problem for a libc is, again, namespace pollution. I suggest
having a compile-time option (yes...) that enables musl-specific
extensions, among which some environment variables in the MUSL_*
namespace. You'll have to accept that, or something similar, at some
point.


> There was a legacy file, /etc/host.conf, that allowed the order to be
> changed, but changing the order seems rather useless to me. On the
> other hand suppressing /etc/hosts could be useful in some instances.

  /etc/host.conf is actually used by libresolv itself. So with glibc,
name resolution goes getaddrinfo() -> NSS -> /etc/nsswitch.conf ->
/etc/hosts or DNS. If DNS: -> libresolv -> /etc/host.conf ->
/etc/hosts or real DNS. That's the magic of glibc configuration, and
of compatibility layers upon compatibility layers: /etc/hosts can
actually be checked twice !
  musl should not check /etc/host.conf itself: that is libresolv
internals. A libc-level switching mechanism would be /etc/nsswitch.conf
if anything, but parsing /etc/nsswitch.conf is too complex if it's
simply about setting 2 boolean flags, so I suggest doing otherwise.


> 4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes
> 4k to store the responses and up to 2k to store the queries.  That's
> not too bad, but along with the address lists, file buffers, and other
> stuff getaddrinfo has around, it's getting the stack usage up to the
> point where getaddrinfo would probably be the biggest stack user in
> musl

  IIRC, the 512 byte limit is only true for UDP responses, and when you
get a truncated UDP response you have to retry with TCP, and there the
maximum length is much more than 512 bytes. Do you just extract the
response from truncated queries ? Anyway getaddrinfo() is authorized
to use heap memory, and apart from not handling TCP at all I don't see
how you can avoid it. This would, paradoxically, save memory most of
the time, because the typical query is short, as well as the typical
response; and you're already using heap memory at some point in the
current getaddrinfo(), so I don't understand the math of putting
everything in the stack.


> For asynchronous use, you call it from its own thread (or use the
> getaddrinfo_a extension, which we don't yet provide but which is easy
> to provide on your own and which I may add to musl since it's
> convenient and ultra-light).

  Making a new thread just to work around a lack of asynchronous
interfaces is ugly. (Remember Netscape Navigator's "dns_helper"
subprocesses ?) That's the very reason getaddrinfo_a() exists.


> There is presently a hard-coded failure timeout of 5 seconds and a
> retry time of 1 second. It would be nice to honor settings from
> resolv.conf to tweak these.

  And the RES_TIMEOUT and RES_DFLRETRY environment variables, then, if
you're going for libresolv compatibility.


> Using a full-fledged DNS library to provide getaddrinfo is akin to
> using GMP to provide printf...

  How so ? All the complex machinery of parsing the DNS protocol,
parsing /etc/resolv.conf, talking from/to the network (in an asynchronous
manner when "search" is implemented) with a retry policy, both UDP and TCP,
and so on, has to be present already. I'm interested in learning the
ninja coding techniques that allow you to write getaddrinfo without all
that ! :)

-- 
  Laurent

Re: Resolver overhaul concepts

[Rich Felker]

On Sun, May 04, 2014 at 06:56:59PM +0100, Laurent Bercot wrote:
> >In order to make this the most useful, though, musl should
> >support nameservers on non-default ports (is there a standard syntax
> >for this, or can we support one without breaking anything?)
> 
>  I'm not aware of any standardized way of running DNS server/caches
> on anything else than the default port; but I don't see why it
> should be necessary. Anyone can run a translator daemon on localhost:53.

Requiring port 53 is not very prohibitive relative to resolv.conf and
nsswitch.conf which are impossible to override without root, but it's
slightly worse: it might be a problem if you also need a public DNS on
the same machine.

Of course if we want to make it possible to override the config on a
per-process basis, requiring port 53 is a fairly serious limitation.
Note that per-process override would be very nice, if nothing else, as
a means of testing; the test framework could setup a custom
resolv.conf and generate malicious packets, packets that don't answer
the query, etc. to test that libc handles them right.

> >and it
> >would also be nice to be able to override resolv.conf on a per-process
> >basis (e.g. via the environment).
> 
>  djbdns and s6-dns do this. It makes sense for every resolver to do it
> too; the problem for a libc is, again, namespace pollution. I suggest
> having a compile-time option (yes...) that enables musl-specific
> extensions, among which some environment variables in the MUSL_*
> namespace. You'll have to accept that, or something similar, at some
> point.

The aim is to use existing mechanisms when available as this
facilitates dropping programs into existing, already-configured
systems. However it may be necessary at some point to add further
options. This is an important topic to discuss at some point, maybe
soon.

> >There was a legacy file, /etc/host.conf, that allowed the order to be
> >changed, but changing the order seems rather useless to me. On the
> >other hand suppressing /etc/hosts could be useful in some instances.
> 
>  /etc/host.conf is actually used by libresolv itself. So with glibc,
> name resolution goes getaddrinfo() -> NSS -> /etc/nsswitch.conf ->
> /etc/hosts or DNS. If DNS: -> libresolv -> /etc/host.conf ->
> /etc/hosts or real DNS. That's the magic of glibc configuration, and
> of compatibility layers upon compatibility layers: /etc/hosts can
> actually be checked twice !
>  musl should not check /etc/host.conf itself: that is libresolv
> internals.

Well it's also (historically) a public libc configuration interface.
Since musl does not use libresolv (note: glibc doesn't really either,
except a seriously-forked version of parts of it) it would need to
check this file itself if we wanted to provide the same configuration
opportunity.

If we do want a way to turn off hosts processing and there's a
traditional way to do it via host.conf, I think supporting the
traditional way is better than invending a new one.

> A libc-level switching mechanism would be /etc/nsswitch.conf
> if anything, but parsing /etc/nsswitch.conf is too complex if it's
> simply about setting 2 boolean flags, so I suggest doing otherwise.

Yes, I don't really want anything to do with nss in musl anyway. :-)

> >4 suffixes times 2 RR's (A and AAAA) makes for 8 queries, which takes
> >4k to store the responses and up to 2k to store the queries.  That's
> >not too bad, but along with the address lists, file buffers, and other
> >stuff getaddrinfo has around, it's getting the stack usage up to the
> >point where getaddrinfo would probably be the biggest stack user in
> >musl
> 
>  IIRC, the 512 byte limit is only true for UDP responses, and when you
> get a truncated UDP response you have to retry with TCP, and there the
> maximum length is much more than 512 bytes. Do you just extract the
> response from truncated queries ?

Yes, tcp is not supported at all. I don't see any reason one would
need tcp for a non-recursive resolver. In principle a response just
needs a few more bytes than the request, plus 4 bytes per address (or
16 for AAAA), and the request size is bounded just above 256 bytes
(the max hostname length).

> Anyway getaddrinfo() is authorized
> to use heap memory, and apart from not handling TCP at all I don't see
> how you can avoid it. This would, paradoxically, save memory most of
> the time, because the typical query is short, as well as the typical
> response; and you're already using heap memory at some point in the
> current getaddrinfo(), so I don't understand the math of putting
> everything in the stack.

More complexity, more failure cases, and then it also depends on free
as opposed to just malloc.

Also it avoids additional fragmentation. If you have lots of threads
making DNS queries and frequently allocating and freeing small blocks,
it's conceivable that the allocation timing ends up breaking up
contiguous space that another thread wants (e.g. the other thread has
called malloc then calls realloc after the address just past its first
allocation is taken). From a standpoint of not making a fragmented
mess of the heap, it's best not to make unnecessary use of allocated
storage.

> >For asynchronous use, you call it from its own thread (or use the
> >getaddrinfo_a extension, which we don't yet provide but which is easy
> >to provide on your own and which I may add to musl since it's
> >convenient and ultra-light).
> 
>  Making a new thread just to work around a lack of asynchronous
> interfaces is ugly. (Remember Netscape Navigator's "dns_helper"
> subprocesses ?) That's the very reason getaddrinfo_a() exists.

A round trip network query (even to localhost) takes several times as
long as creating a thread (and for tcp, typically takes hundreds of
times the resources of thread creation since the kernel allocates
massively bloated send/recv buffers).

With Netscape's forked dns_helper, there are other costs like error
handling complexity when the helper is wrongly killed, etc., but most
of those don't apply to threads.

> >There is presently a hard-coded failure timeout of 5 seconds and a
> >retry time of 1 second. It would be nice to honor settings from
> >resolv.conf to tweak these.
> 
>  And the RES_TIMEOUT and RES_DFLRETRY environment variables, then, if
> you're going for libresolv compatibility.

I'm not sure if they have the same semantics, but it's doubtful that
anyone cares if they're exactly the same, so we could probably reuse
them. resolv.conf also has a mechanism for setting these.

> >Using a full-fledged DNS library to provide getaddrinfo is akin to
> >using GMP to provide printf...
> 
>  How so ? All the complex machinery of parsing the DNS protocol,
> parsing /etc/resolv.conf, talking from/to the network (in an asynchronous
> manner when "search" is implemented) with a retry policy, both UDP and TCP,
> and so on, has to be present already. I'm interested in learning the
> ninja coding techniques that allow you to write getaddrinfo without all
> that ! :)

See my printf analogy. printf needs decimal bignums, but only needs
two operations on them: << and >>. A general bignum implementation
that can do arbitrary operations on them is a lot more complex and
costly than an implementation that just does two operations in-place.

Similarly, getaddrinfo needs DNS, but it only needs generation of
fixed-form queries, minimal data extraction from result packets, and
some degree of validation.

FYI the current code is ~4k binary and the overhaul is not expected to
increase that much. I really doubt you could achieve that with general
DNS library code.

Rich

Re: Resolver overhaul concepts

[Laurent Bercot]

> Requiring port 53 is not very prohibitive relative to resolv.conf and
> nsswitch.conf which are impossible to override without root, but it's
> slightly worse: it might be a problem if you also need a public DNS on
> the same machine.

  You mean running a resolver and a server on the same machine ? I've
been doing that for years with tinydns on the outside and dnscache on
127.0.0.1: there's no reason why a user couldn't do the same with a
custom resolution daemon.
  Of course, providing custom resolution *and* DNS data to the outside
world then requires two different public IP addresses, but that's
nothing new: using the same port for resolution and data service is a
fundamental flaw in the DNS protocol in the first place (and the main
reason why mainstream DNS software is so hopelessly monolithic), and
a custom resolution daemon won't be in a different position from, say,
dnscache.


> Of course if we want to make it possible to override the config on a
> per-process basis, requiring port 53 is a fairly serious limitation.

  I don't feel the same way. Name resolution is name resolution; if a
resolver, no matter how it resolves, reads and understands client DNS
queries, then it makes sense for it to listen on port 53 somewhere
(if we forget that DNS data servers also listen on the same port).
I understand the desire for flexibility, and I can imagine cases where
this would be useful, but I don't see a blatant need for it.
Especially with IPv6 around the corner (only two or three decades now),
where address space is cheap.

  
> Yes, tcp is not supported at all. I don't see any reason one would
> need tcp for a non-recursive resolver. In principle a response just
> needs a few more bytes than the request, plus 4 bytes per address (or
> 16 for AAAA), and the request size is bounded just above 256 bytes
> (the max hostname length).

  Plus the authority and additional sections, and... oh wait, you only
have to handle A and AAAA, which shouldn't have any of those. OK, now
I understand how you don't need a full DNS engine. :)

  Still, I wouldn't bet it's going to remain that way. Having 6 or 7 A
fields isn't uncommon nowadays (google.com has 6 in most places).
Now imagine 6 or 7 AAAA fields instead: it will begin to seriously
flirt with the limit. It won't happen in the near future, but it will
happen.


> More complexity, more failure cases, and then it also depends on free
> as opposed to just malloc.
>
> Also it avoids additional fragmentation.

  Don't get me wrong, I'm a huge advocate of using the stack whenever
possible. s6-dns actually started when I studied djbdns's client
library and went "ewww, there are way too many mallocs in there -
I can do better."
  It's just that for generic DNS responses, there's no way around
malloc - but if you don't need to support TCP, then you can store
all responses in the stack indeed, and that's a lot of savings.


> A round trip network query (even to localhost) takes several times as
> long as creating a thread (and for tcp, typically takes hundreds of
> times the resources of thread creation since the kernel allocates
> massively bloated send/recv buffers).

  I was thinking ease of use from a programmer's point of view.
Creating a thread to perform a simple operation, then join the thread,
is not elegant. In Go, this is absolutely the right way of doing
things (because goroutines are even lighter than threads - the runtime
has its own multiplexer), but in C, something like getaddrinfo_a is
easier and more idiomatic.


> Similarly, getaddrinfo needs DNS, but it only needs generation of
> fixed-form queries, minimal data extraction from result packets, and
> some degree of validation.

  Being able to guarantee that all queries and responses fit into the
stack is simply huge.


> FYI the current code is ~4k binary and the overhaul is not expected to
> increase that much. I really doubt you could achieve that with general
> DNS library code.

  Indeed. My s6-dnsip4 static binary (linked against musl, for x86_64) is
just short of 40k, including roughly 25k of DNS code, and I've made it as
lean as I could without sacrificing readability.

-- 
  Laurent

Re: Resolver overhaul concepts

[Andy Lutomirski]

On 05/04/2014 09:24 AM, Rich Felker wrote:
> On Sun, May 04, 2014 at 05:07:33PM +0100, Laurent Bercot wrote:
>>
>>  I believe the very first thing to address is what exactly you call
>> a resolver.
> 
> There are some legacy dn_*/res_* interfaces in demand which are
> presently supported only poorly or not at all. Part of the side goal
> of the resolver overhaul is to provide them cleanly without code
> duplication. But for the most part, "resolver" means "getaddrinfo"
> since it is the only standard, non-deprecated interface to name
> resolution.
> 
>>  getaddrinfo() is a horrible interface, and one of the reasons why
>> is that it is loosely designed. Not much is standardized, and it's up
>> to you to decide exactly what to do with it; it's important to be
>> clear about what is implemented, and to document it, because not all
>> applications have the same expectations, and it's very easy to get
>> confused when the resolution path is unexpected.
> 
> It's standardized by POSIX, and the POSIX text is sufficient to tell
> you how to use it for all portable usages. Most of the confusion/mess
> comes from non-conforming implemnentations, particularly in the area
> of returning wrong error codes.
> 
>>  glibc's getaddrinfo() is the entry point to the NSS layer, which
>> can basically implement *any* kind of "name resolution". AFAICT,
>> it's not a goal of musl to reimplement the whole NSS spaghetti
>> monster, but some applications will depend on /etc/nsswitch.conf
>> or something similar; even without supporting /etc/nsswitch.conf,
>> it would be nice to provide a mechanism to selectively enable/disable
>> at least /etc/hosts lookup and DNS lookup. The current resolution
> 
> The policy for supporting something like nss has always been that musl
> implements a perfectly reasonable public protocol for providing any
> back-end you want: the DNS protocol. You can run a local daemon
> speaking DNS and serving names from any backend you like, and this is
> the correct way to achieve it (rather than linking random buggy,
> likely-not-namespace-clean libraries into the application's address
> space). In order to make this the most useful, though, musl should
> support nameservers on non-default ports (is there a standard syntax
> for this, or can we support one without breaking anything?), and it
> would also be nice to be able to override resolv.conf on a per-process
> basis (e.g. via the environment).

How about 'nameserver /path/to/unix/socket'?  If glibc supported that,
too, it might solve a lot of problems involving systemwide resolvers and
containers.

This might have to be 'unixsocknameserver /path/to/unix/socket' or
something for better interoperability.

--Andy

Re: Re: Resolver overhaul concepts

[Rich Felker]

On Fri, May 09, 2014 at 06:04:08PM -0700, Andy Lutomirski wrote:
> > The policy for supporting something like nss has always been that musl
> > implements a perfectly reasonable public protocol for providing any
> > back-end you want: the DNS protocol. You can run a local daemon
> > speaking DNS and serving names from any backend you like, and this is
> > the correct way to achieve it (rather than linking random buggy,
> > likely-not-namespace-clean libraries into the application's address
> > space). In order to make this the most useful, though, musl should
> > support nameservers on non-default ports (is there a standard syntax
> > for this, or can we support one without breaking anything?), and it
> > would also be nice to be able to override resolv.conf on a per-process
> > basis (e.g. via the environment).
> 
> How about 'nameserver /path/to/unix/socket'?  If glibc supported that,
> too, it might solve a lot of problems involving systemwide resolvers and
> containers.
> 
> This might have to be 'unixsocknameserver /path/to/unix/socket' or
> something for better interoperability.

I don't see any convincing reason to support this. It greatly
complicates the lookup code (having to have 2+ sockets instead of just
one, having to support different address/protocol families, ...) and
doesn't let you do anything you can't already do with AF_INET[6] and
udp. There's also no precedent, which rather defeats the principle of
not inventing new mechanisms for something where there's already a
fully-general option available.

Rich

Re: Re: Resolver overhaul concepts

[Laurent Bercot]

On 10/05/2014 03:36, Rich Felker wrote:
> doesn't let you do anything you can't already do with AF_INET[6] and
> udp.

  Nit: it does. (User authentication, fd passing.) But I agree that
this is not needed for name resolution and would gratuitously add
muslisms.

-- 
  Laurent

Re: Re: Resolver overhaul concepts

[Rich Felker]

On Sat, May 10, 2014 at 10:26:46AM +0100, Laurent Bercot wrote:
> On 10/05/2014 03:36, Rich Felker wrote:
> >doesn't let you do anything you can't already do with AF_INET[6] and
> >udp.
> 
>  Nit: it does. (User authentication, fd passing.) But I agree that
> this is not needed for name resolution and would gratuitously add
> muslisms.

I meant in the context of being a resolver back-end, not anything
else. For this purpose there is no use in fd passing, and if you
really want authentication, iptables (or whatever its successor is)
can tag local packets by the originating uid/pid/etc. and provide
access controls roughly equivalent to what AF_UNIX could provide
(however it seems unlikely that this would be useful either).

Rich