* Re: Re: Would love to see reconsideration for domain and search @ 2015-10-23 15:30 Jan Broer 2015-10-23 16:01 ` Tim Hockin 0 siblings, 1 reply; 44+ messages in thread From: Jan Broer @ 2015-10-23 15:30 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 2066 bytes --] A very interesting discussion going on here. > You can't query all nameservers and just take > the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of > the higher-priority (listed first in resolv.conf) nameservers have > timed out or SERVFAIL'ed. You can issue queries in parallel, but you > must process responses in order, which is what you describe below. GNU libc resolver gives priority to the primary nameserver because it assumes that this is the one with the lowest RTT / highest availability. This priorization was never understood as meaning that the primary's servers records are more "authorative" than those of secondary nameservers. > My view has always been that multiple > nameservers in resolv.conf are for redundancy, not for serving > conflicting records. I agree. >I have to disagree. Some non-forwarding DNS servers use SERVFAIL to >indicate "I am not serving for that domain" specifically to make the >client move to their next nameserver. if ns1 returns SERVFAIL, try >ns2. If ns1 times out, try ns2. Otherwise what good is ns2? This kind of result unioning you want from musl-libs isn't even supported by GNU libc resolver. Yes it works for you - but this is because your DNS server is breaking DNS RFC specs: SERVFAIL is not a technically legal response for "I am not serving for that domain". It's just WRONG. When an authorative server (which the Kubernetes DNS server is) doesn't serve a specific zone it has to respond with NXDOMAIN. Of course if your server would follow the rules than that type of result unioning doesn't work because GNU libc resolver does not failover to secondary servers when the response is NXDOMAIN. It's just a weak argument to essentially say: "The GNU libc resolver does exactly what i want when i point it to my broken DNS server, why can't musl-libc?". The bottom line: Leave nameserver querying logic as it is: Check! Implement search paths: Check! Consider ndots option in resolv.conf: Check! query name <= ndots: search first query name > ndots : never search LGTM! Jan [-- Attachment #2: Type: text/html, Size: 2621 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 15:30 Re: Would love to see reconsideration for domain and search Jan Broer @ 2015-10-23 16:01 ` Tim Hockin 2015-10-23 17:40 ` Jan Broer 0 siblings, 1 reply; 44+ messages in thread From: Tim Hockin @ 2015-10-23 16:01 UTC (permalink / raw) To: musl On Fri, Oct 23, 2015 at 8:30 AM, Jan Broer <jasiu.79@gmail.com> wrote: > Leave nameserver querying logic as it is: Check! > Implement search paths: Check! > Consider ndots option in resolv.conf: Check! > > query name <= ndots: search first > query name > ndots : never search I think that is where we have arrived (except it's < and >= in your expressions). Kubernetes will have to change some to satisfy the assumed fungibility of nameservers, but it's the direction things are almost certainly moving anyway. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 16:01 ` Tim Hockin @ 2015-10-23 17:40 ` Jan Broer 2015-10-23 18:02 ` Tim Hockin ` (2 more replies) 0 siblings, 3 replies; 44+ messages in thread From: Jan Broer @ 2015-10-23 17:40 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 605 bytes --] >I think that is where we have arrived (except it's < and >= in your >expressions). Kubernetes will have to change some to satisfy the >assumed fungibility of nameservers, but it's the direction things are >almost certainly moving anyway. Hi Tim, i guess i skipped these last message - sorry. If you could clear this up for me: When there are multiple nameservers, does the musl resolver return the first response it receives regardless of it's status? So even when it is something like SERVFAIL or REFUSED it will not wait for the responses from the other nameservers that might be positive? Thanks! [-- Attachment #2: Type: text/html, Size: 763 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 17:40 ` Jan Broer @ 2015-10-23 18:02 ` Tim Hockin 2015-10-23 18:03 ` Jan Broer 2015-10-23 18:29 ` Rich Felker 2 siblings, 0 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-23 18:02 UTC (permalink / raw) To: musl I don't know the answer to that - my knowledge of the details of musl are pretty limited and mostly stop at "search doesn't work". On Fri, Oct 23, 2015 at 10:40 AM, Jan Broer <jasiu.79@gmail.com> wrote: >>I think that is where we have arrived (except it's < and >= in your >>expressions). Kubernetes will have to change some to satisfy the >>assumed fungibility of nameservers, but it's the direction things are >>almost certainly moving anyway. > > Hi Tim, i guess i skipped these last message - sorry. > > If you could clear this up for me: > When there are multiple nameservers, does the musl resolver return the first > response it receives regardless of it's status? > So even when it is something like SERVFAIL or REFUSED it will not wait for > the responses from the other nameservers that might be positive? > > Thanks! ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 17:40 ` Jan Broer 2015-10-23 18:02 ` Tim Hockin @ 2015-10-23 18:03 ` Jan Broer 2015-10-23 18:29 ` Rich Felker 2 siblings, 0 replies; 44+ messages in thread From: Jan Broer @ 2015-10-23 18:03 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 941 bytes --] > On Thu, Oct 22, 2015 at 10:37:53PM -0700, Tim Hockin wrote: > > I saw from a different thread that musl doesn't or didn't do TCP > > fallbacks - is that still the case? I know we need that for things > > like large multi-SRV sets (which I do not expect libc to support), and > > we have some people who have large A sets (which I do expect libc to > > support). >Indeed. The only way you can overflow the UDP size limit with the >records the stub resolver uses is with a max-length CNAME pointing to >a max or near-max length record with little or no overlap to allow for >compression. Most DNS servers are actually well able to send records greater than 512 bytes over UDP. This is part of the EDNS0 DNS extension. I think it's worth discussing to implement EDNS0 in the musl resolver. Codewise you just have to set the UDP receive buffer to the desired size and include an OPT RR in every query indicating the supported max UDP size. [-- Attachment #2: Type: text/html, Size: 1184 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 17:40 ` Jan Broer 2015-10-23 18:02 ` Tim Hockin 2015-10-23 18:03 ` Jan Broer @ 2015-10-23 18:29 ` Rich Felker 2015-10-23 21:27 ` Jan Broer 2 siblings, 1 reply; 44+ messages in thread From: Rich Felker @ 2015-10-23 18:29 UTC (permalink / raw) To: musl On Fri, Oct 23, 2015 at 07:40:45PM +0200, Jan Broer wrote: > >I think that is where we have arrived (except it's < and >= in your > >expressions). Kubernetes will have to change some to satisfy the > >assumed fungibility of nameservers, but it's the direction things are > >almost certainly moving anyway. > > Hi Tim, i guess i skipped these last message - sorry. > > If you could clear this up for me: > When there are multiple nameservers, does the musl resolver return the > first response it receives regardless of it's status? > So even when it is something like SERVFAIL or REFUSED it will not wait for > the responses from the other nameservers that might be positive? No. musl basically ignores and response except success or nxdomain. Actually servfail is treated specially and results in a limited number of immediate retries because there are a lot of broken nameservers that return servfail but immediately succeed on a second try, and doing the immediate retry drastically speeds up queries in such environments. In any case, only success or nxdomain are treated as results reportable to the caller. Otherwise musl waits for one of these and returns a transient failure (not does-not-exist) if it times out. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Re: Would love to see reconsideration for domain and search 2015-10-23 18:29 ` Rich Felker @ 2015-10-23 21:27 ` Jan Broer 0 siblings, 0 replies; 44+ messages in thread From: Jan Broer @ 2015-10-23 21:27 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 252 bytes --] > > In any case, only success or nxdomain are treated as results > reportable to the caller. Otherwise musl waits for one of these and > returns a transient failure (not does-not-exist) if it times out. > > Rich > Thank you that makes a lot of sense. [-- Attachment #2: Type: text/html, Size: 548 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Would love to see reconsideration for domain and search @ 2015-10-22 21:24 Tim Hockin 2015-10-22 21:56 ` Rich Felker 2015-10-23 5:26 ` Kurt H Maier 0 siblings, 2 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-22 21:24 UTC (permalink / raw) To: musl Hi all, I saw this thread on the web archive but am not sure how to respond to the thread directly as a new joinee of the ML. I hope this finds its way... I am one of the developers of Kubernetes and I own the DNS portion, in particular. I desperately want to use Alpine Linux (based on musl) but for now I have to warn people NOT to use it because of this issue. On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: >> I'm writing the wonderful musl project today to open discussion >> about the future possibility of DNS search and domain keyword >> support. We've been using musl libc (by way of Alpine Linux) for >> new development of applications as containers that discover each >> other through DNS and other software defined networking. >> >> In particular, we are starting to use applications like SkyDNS, >> Consul, and Kubernetes, all of which rely on local name >> resolution in some way using search paths. Many users of the >> Alpine Linux container image have also expressed their desire for >> this feature at >> https://github.com/gliderlabs/docker-alpine/issues/8. >> >> On the functional differences between glibc page, the domain and >> search keyword "Support may be added in the future if there is >> demand". So please consider this request an addition to whatever >> demand for the feature already exists. >> >> Thank you for your time and great work on the musl libc project! > > I think this is a reasonable request. I'll look into it more. > > One property I do not want to break is deterministic results, so > when a search is performed, if any step of the search ends with > an error rather than a positive or negative result, the whole > lookup needs to stop and report the error rather than falling > back. Falling back is not safe and creates a situation where DoS > can be used to control which results are returned. I understand your point, though the world at large tends to disagree. Everyone has a primary and secondary `nameserver` record (or should). If the first one times out, try the second. Most resolver libs seem to accept a SERVFAIL response or a timeout as a signal to try the next server, and I would encourage you to do the same. Stopping on positive response or NXDOMAIN seems to be commonly accepted with a caveat. You can't query all nameservers and just take the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of the higher-priority (listed first in resolv.conf) nameservers have timed out or SERVFAIL'ed. You can issue queries in parallel, but you must process responses in order, which is what you describe below. > While it would be possible to parallelize the search while > serializing the results (i.e. waiting to accept a result from the > second query until the first query finished with a negative > result), I think the consensus during the last round of > discussion of this topic was that the complexity cost is too > great and the benefit too small. Ideally, the first query should > always succeed, anyway. The real world is not ideal. Not all nameservers are identically scoped - you MUST respect the ordering in resolv.conf - to do otherwise is semantically broken. If implementation simplicity means literally doing queries in serial, then that is what you should do. Similarly, you can't just search all search domains in parallel and take the first response. The ordering is meaningful. > I also have a few questions: > > 1. Do you need multiple search items, or just a single domain? > Any setup with multiple searches necessarily has suboptimal > performance because ndots is not sufficient to make the right > initial choice of query. If you do need this functionality, a > unioning proxy dns server may be a better option than resolv.conf > domain search; it would give much better performance. We use multiple search paths and ndots > 1. I'm not sure what you mean by "unioning" here. Search path ordering is as meaningful as nameserver ordering. You can't avoid making the query for each search suffix in the worst case, and it has the same restriction as nameserver - the search order must be respected. There does seem to be some different implementations that search for the "naked" query first vs last, though. I think the semantically correct (but pessimal performance) is to search for that last. > 2. For your intended applications, is there a need to support > ndots>1? Such configurations are generally not friendly to > applications that expect to be able to resolve normal internet > domain lookups, and performance for such lookups will be very bad > (because the search domains first have to fail). DNS is a very lightweight protocol. We have not measured any practical detriment for having 6 search domains and ndots=5. In the normal case it fails very quickly. That aside, it should be my business if I want to (mis)configure my system that way :) > 3. The glibc behavior is just to swap the order of search when > the query string has >=ndots dots in it, but would it be > acceptable never to try the search domains at all in this case? > That would yield much better performance for nxdomain results and > avoid unexpected positive results due to weird subdomains > existing in your search domain (e.g. a wildcard for > *.us.example.com would cause *.us to wrongly resolve for > non-existant .us domains). I think that would be correct. If I have 3 dots and ndots=2, search paths should be ignored. Tim ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 21:24 Tim Hockin @ 2015-10-22 21:56 ` Rich Felker 2015-10-22 22:36 ` Tim Hockin 2015-10-23 5:26 ` Kurt H Maier 1 sibling, 1 reply; 44+ messages in thread From: Rich Felker @ 2015-10-22 21:56 UTC (permalink / raw) To: Tim Hockin; +Cc: musl On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > Hi all, > > I saw this thread on the web archive but am not sure how to respond to > the thread directly as a new joinee of the ML. I hope this finds its > way... No problem; just starting a new thread like this and quoting the old one is fine. > I am one of the developers of Kubernetes and I own the DNS portion, in > particular. I desperately want to use Alpine Linux (based on musl) > but for now I have to warn people NOT to use it because of this issue. > > On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: > > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: > >> I'm writing the wonderful musl project today to open discussion > >> about the future possibility of DNS search and domain keyword > >> support. We've been using musl libc (by way of Alpine Linux) for > >> new development of applications as containers that discover each > >> other through DNS and other software defined networking. > >> > >> In particular, we are starting to use applications like SkyDNS, > >> Consul, and Kubernetes, all of which rely on local name > >> resolution in some way using search paths. Many users of the > >> Alpine Linux container image have also expressed their desire for > >> this feature at > >> https://github.com/gliderlabs/docker-alpine/issues/8. > >> > >> On the functional differences between glibc page, the domain and > >> search keyword "Support may be added in the future if there is > >> demand". So please consider this request an addition to whatever > >> demand for the feature already exists. > >> > >> Thank you for your time and great work on the musl libc project! > > > > I think this is a reasonable request. I'll look into it more. > > > > One property I do not want to break is deterministic results, so > > when a search is performed, if any step of the search ends with > > an error rather than a positive or negative result, the whole > > lookup needs to stop and report the error rather than falling > > back. Falling back is not safe and creates a situation where DoS > > can be used to control which results are returned. > > I understand your point, though the world at large tends to disagree. > Everyone has a primary and secondary `nameserver` record (or should). > If the first one times out, try the second. Most resolver libs seem > to accept a SERVFAIL response or a timeout as a signal to try the next > server, and I would encourage you to do the same. musl intentionally does not do this because it yields abysmal performance. If the first nameserver is overloaded or the packet is lost, you suffer several-second lookup latency. > Stopping on positive response or NXDOMAIN seems to be commonly > accepted with a caveat. You can't query all nameservers and just take > the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of > the higher-priority (listed first in resolv.conf) nameservers have > timed out or SERVFAIL'ed. You can issue queries in parallel, but you > must process responses in order, which is what you describe below. Timeout or servfail is not sufficient to accept an nxdomain from a lower-priority server. To preserve consistency of results under transient failure, you actually have to wait for the nxdomain from the higher-priority server. Either way, this very much pessimizes usage cases like running "netstat" with huge numbers of connections where many of the ip addresses fail to reverse. Being able to return immediately as soon as any one of the nameservers responds with nxdomain makes the difference between a <1s netstat run and a 5-10s netstat run. Thus, if we add extensions to support the kind of result unioning you want across multiple nameservers, I think they should be configurable and off-by-default. A simple option in resolv.conf could turn them on. And there could be options for requiring nxdomain from all servers (true union) or just for highest-priority when accepting negative results. > > While it would be possible to parallelize the search while > > serializing the results (i.e. waiting to accept a result from the > > second query until the first query finished with a negative > > result), I think the consensus during the last round of > > discussion of this topic was that the complexity cost is too > > great and the benefit too small. Ideally, the first query should > > always succeed, anyway. > > The real world is not ideal. Not all nameservers are identically > scoped - you MUST respect the ordering in resolv.conf - to do > otherwise is semantically broken. If implementation simplicity means > literally doing queries in serial, then that is what you should do. Even legacy resolvers had the option to rotate the nameservers for load-balancing, so I think it's a stretch to say the ordering is supposed to be semantic. My view has always been that multiple nameservers in resolv.conf are for redundancy, not for serving conflicting records. > Similarly, you can't just search all search domains in parallel and > take the first response. The ordering is meaningful. Indeed, search domains are like that, because they inherently produce ambiguity/overlapping namespaces with different definitions. This is why myself and others who weighed in on the original question of supporting them were against, but left the option open to revisit the topic if users who need them show up. > > I also have a few questions: > > > > 1. Do you need multiple search items, or just a single domain? > > Any setup with multiple searches necessarily has suboptimal > > performance because ndots is not sufficient to make the right > > initial choice of query. If you do need this functionality, a > > unioning proxy dns server may be a better option than resolv.conf > > domain search; it would give much better performance. > > We use multiple search paths and ndots > 1. I'm not sure what you > mean by "unioning" here. Search path ordering is as meaningful as > nameserver ordering. You can't avoid making the query for each search > suffix in the worst case, and it has the same restriction as > nameserver - the search order must be respected. > > There does seem to be some different implementations that search for > the "naked" query first vs last, though. I think the semantically > correct (but pessimal performance) is to search for that last. The traditional behavior is to do the naked query first if the query string has at least 'ndots' dots, and to do the search domains first otherwise. Also I believe a final dot always suppresses search. My point was that with ndots=1 (default) and only a single search domain, the _expected_ result is that the first query succeed. But if you have ndots>1 or multiple search domains, you expect a portion of your queries to fall back at least once. This adds significant latency. In such a situation, you can avoid the additional latency (except on the first query of a given record) by running a local caching nameserver that does the search and unioning for you, rather than having the stub resolver in libc do it. Then subsequent queries succeed immediately using the cache. The reason I asked about usage case (ndots=1 vs ndots>1, single vs multiple search) is that, in the multi-fallback case, it might make more sense (from a performance and clean design standpoint) to implement this with a caching nameserver on localhost rather than in musl. > > 2. For your intended applications, is there a need to support > > ndots>1? Such configurations are generally not friendly to > > applications that expect to be able to resolve normal internet > > domain lookups, and performance for such lookups will be very bad > > (because the search domains first have to fail). > > DNS is a very lightweight protocol. We have not measured any > practical detriment for having 6 search domains and ndots=5. In the > normal case it fails very quickly. That aside, it should be my > business if I want to (mis)configure my system that way :) I suspect we have different definitions of quick... :) > > 3. The glibc behavior is just to swap the order of search when > > the query string has >=ndots dots in it, but would it be > > acceptable never to try the search domains at all in this case? > > That would yield much better performance for nxdomain results and > > avoid unexpected positive results due to weird subdomains > > existing in your search domain (e.g. a wildcard for > > *.us.example.com would cause *.us to wrongly resolve for > > non-existant .us domains). > > I think that would be correct. If I have 3 dots and ndots=2, search > paths should be ignored. Glad we agree on this. I hope you feel like this conversation is productive. I don't want to rule out anything/"say no" right away, but rather try to get a better understanding of your requirements first and figure out what makes the most sense to do on musl's side. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 21:56 ` Rich Felker @ 2015-10-22 22:36 ` Tim Hockin 2015-10-22 23:00 ` Josiah Worcester 0 siblings, 1 reply; 44+ messages in thread From: Tim Hockin @ 2015-10-22 22:36 UTC (permalink / raw) To: Rich Felker; +Cc: musl On Thu, Oct 22, 2015 at 2:56 PM, Rich Felker <dalias@libc.org> wrote: > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: >> Hi all, >> >> I saw this thread on the web archive but am not sure how to respond to >> the thread directly as a new joinee of the ML. I hope this finds its >> way... > > No problem; just starting a new thread like this and quoting the old > one is fine. > >> I am one of the developers of Kubernetes and I own the DNS portion, in >> particular. I desperately want to use Alpine Linux (based on musl) >> but for now I have to warn people NOT to use it because of this issue. >> >> On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: >> > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: >> >> I'm writing the wonderful musl project today to open discussion >> >> about the future possibility of DNS search and domain keyword >> >> support. We've been using musl libc (by way of Alpine Linux) for >> >> new development of applications as containers that discover each >> >> other through DNS and other software defined networking. >> >> >> >> In particular, we are starting to use applications like SkyDNS, >> >> Consul, and Kubernetes, all of which rely on local name >> >> resolution in some way using search paths. Many users of the >> >> Alpine Linux container image have also expressed their desire for >> >> this feature at >> >> https://github.com/gliderlabs/docker-alpine/issues/8. >> >> >> >> On the functional differences between glibc page, the domain and >> >> search keyword "Support may be added in the future if there is >> >> demand". So please consider this request an addition to whatever >> >> demand for the feature already exists. >> >> >> >> Thank you for your time and great work on the musl libc project! >> > >> > I think this is a reasonable request. I'll look into it more. >> > >> > One property I do not want to break is deterministic results, so >> > when a search is performed, if any step of the search ends with >> > an error rather than a positive or negative result, the whole >> > lookup needs to stop and report the error rather than falling >> > back. Falling back is not safe and creates a situation where DoS >> > can be used to control which results are returned. >> >> I understand your point, though the world at large tends to disagree. >> Everyone has a primary and secondary `nameserver` record (or should). >> If the first one times out, try the second. Most resolver libs seem >> to accept a SERVFAIL response or a timeout as a signal to try the next >> server, and I would encourage you to do the same. > > musl intentionally does not do this because it yields abysmal > performance. If the first nameserver is overloaded or the packet is > lost, you suffer several-second lookup latency. But at least it works eventually. You're faced with a choice. Wait 2 seconds for ns1 to timeout and then fail in a way that most apps don't handle well or wait for 2 seconds and then (usually) get a fast response from ns2. It seems better in every way to eventually succeed, though I agree it's a bit less visible. >> Stopping on positive response or NXDOMAIN seems to be commonly >> accepted with a caveat. You can't query all nameservers and just take >> the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of >> the higher-priority (listed first in resolv.conf) nameservers have >> timed out or SERVFAIL'ed. You can issue queries in parallel, but you >> must process responses in order, which is what you describe below. > > Timeout or servfail is not sufficient to accept an nxdomain from a > lower-priority server. To preserve consistency of results under > transient failure, you actually have to wait for the nxdomain from the I have to disagree. Some non-forwarding DNS servers use SERVFAIL to indicate "I am not serving for that domain" specifically to make the client move to their next nameserver. if ns1 returns SERVFAIL, try ns2. If ns1 times out, try ns2. Otherwise what good is ns2? > higher-priority server. Either way, this very much pessimizes usage > cases like running "netstat" with huge numbers of connections where > many of the ip addresses fail to reverse. Being able to return > immediately as soon as any one of the nameservers responds with > nxdomain makes the difference between a <1s netstat run and a 5-10s > netstat run. Sure it's faster but it's WRONG. Returning a random number would be faster, too, but it is equally wrong. This is why netstat (and myriad other tools) has a `-n` flag. > Thus, if we add extensions to support the kind of result unioning you > want across multiple nameservers, I think they should be configurable > and off-by-default. A simple option in resolv.conf could turn them on. > And there could be options for requiring nxdomain from all servers > (true union) or just for highest-priority when accepting negative > results. I can't agree with this. It's reasonable to make options for these, but I think the non-standard behaviors should be off by default. Consider this from the point of view of a system like Docker or Kubernetes, which generate resolv.conf for you - they have no idea what libc your apps are using, so it's unreasonable to ask them to turn off libc-specific flags. However, the end user knows, and it is perfectly sane to ask them to explicitly opt-in to non-standard optimized behaviors. >> > While it would be possible to parallelize the search while >> > serializing the results (i.e. waiting to accept a result from the >> > second query until the first query finished with a negative >> > result), I think the consensus during the last round of >> > discussion of this topic was that the complexity cost is too >> > great and the benefit too small. Ideally, the first query should >> > always succeed, anyway. >> >> The real world is not ideal. Not all nameservers are identically >> scoped - you MUST respect the ordering in resolv.conf - to do >> otherwise is semantically broken. If implementation simplicity means >> literally doing queries in serial, then that is what you should do. > > Even legacy resolvers had the option to rotate the nameservers for > load-balancing, so I think it's a stretch to say the ordering is > supposed to be semantic. My view has always been that multiple > nameservers in resolv.conf are for redundancy, not for serving > conflicting records. You argued above that you should not try a secondary server in case of timeout or SERVFAIL. Obviously you would not try it on success nor NXDOMAIN. When do you see a secondary being used at all? As for rotate, note that it is an option and OFF by default in every mainstream resolver implementation. But this point is sort of academic for us - we're moving to a forwarding nameserver so really there is only the primary nameserver. We just need you to ask the first nameserver first. >> Similarly, you can't just search all search domains in parallel and >> take the first response. The ordering is meaningful. > > Indeed, search domains are like that, because they inherently produce > ambiguity/overlapping namespaces with different definitions. This is > why myself and others who weighed in on the original question of > supporting them were against, but left the option open to revisit the > topic if users who need them show up. Yeah, I scanned the related threads. I understand the issue in theory, but in practice these are things configured by admins. If there's a conflict or ambiguity, you should solve that, not jettison powerful functionality. >> > I also have a few questions: >> > >> > 1. Do you need multiple search items, or just a single domain? >> > Any setup with multiple searches necessarily has suboptimal >> > performance because ndots is not sufficient to make the right >> > initial choice of query. If you do need this functionality, a >> > unioning proxy dns server may be a better option than resolv.conf >> > domain search; it would give much better performance. >> >> We use multiple search paths and ndots > 1. I'm not sure what you >> mean by "unioning" here. Search path ordering is as meaningful as >> nameserver ordering. You can't avoid making the query for each search >> suffix in the worst case, and it has the same restriction as >> nameserver - the search order must be respected. >> >> There does seem to be some different implementations that search for >> the "naked" query first vs last, though. I think the semantically >> correct (but pessimal performance) is to search for that last. > > The traditional behavior is to do the naked query first if the query > string has at least 'ndots' dots, and to do the search domains first > otherwise. Also I believe a final dot always suppresses search. > > My point was that with ndots=1 (default) and only a single search > domain, the _expected_ result is that the first query succeed. But if > you have ndots>1 or multiple search domains, you expect a portion of > your queries to fall back at least once. This adds significant > latency. It adds latency, but the magnitude is very much determined by the installation. In our case it is negligible and well worth the cost. I fear you're optimizing without data - it should be the site-admin's problem to configure things in an acceptable way. libc doesn't get to decide what "acceptable" means. > In such a situation, you can avoid the additional latency (except on > the first query of a given record) by running a local caching > nameserver that does the search and unioning for you, rather than > having the stub resolver in libc do it. Then subsequent queries > succeed immediately using the cache. The reason I asked about usage > case (ndots=1 vs ndots>1, single vs multiple search) is that, in the > multi-fallback case, it might make more sense (from a performance and > clean design standpoint) to implement this with a caching nameserver > on localhost rather than in musl. We might be moving to a per-machine local DNS agent, which would cache as you describe. HOWEVER, there's a pretty important piece that I guess I left out. Docker and Kubernetes and similar systems run many containers per machine. Each container has a potentially different search path. I might run 100 or more containers on a single machine - I can't run 100 DNS caches, and I can't put that back on users. So from our perspective the search paths MUST come from the containers themselves, even if we run a machine local cache to mitigate latency and SPOF. >> > 2. For your intended applications, is there a need to support >> > ndots>1? Such configurations are generally not friendly to >> > applications that expect to be able to resolve normal internet >> > domain lookups, and performance for such lookups will be very bad >> > (because the search domains first have to fail). >> >> DNS is a very lightweight protocol. We have not measured any >> practical detriment for having 6 search domains and ndots=5. In the >> normal case it fails very quickly. That aside, it should be my >> business if I want to (mis)configure my system that way :) > > I suspect we have different definitions of quick... :) Quick is situational. In a cloud-based mostly-webapp stack, 50ms to do a name lookup ain't so bad, given the relative infrequency of that operation. Also most names actually DO resolve on the first or second search path. >> > 3. The glibc behavior is just to swap the order of search when >> > the query string has >=ndots dots in it, but would it be >> > acceptable never to try the search domains at all in this case? >> > That would yield much better performance for nxdomain results and >> > avoid unexpected positive results due to weird subdomains >> > existing in your search domain (e.g. a wildcard for >> > *.us.example.com would cause *.us to wrongly resolve for >> > non-existant .us domains). >> >> I think that would be correct. If I have 3 dots and ndots=2, search >> paths should be ignored. > > Glad we agree on this. > > I hope you feel like this conversation is productive. I don't want to > rule out anything/"say no" right away, but rather try to get a better > understanding of your requirements first and figure out what makes the > most sense to do on musl's side. Absolutely. I'm happy to engage. Obviously our use case is a bit outside of what musl was really aiming for, but it offers a really nice base for very efficient containers. A lot of people want to use Alpine and it breaks my heart to tell them they can't. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 22:36 ` Tim Hockin @ 2015-10-22 23:00 ` Josiah Worcester 2015-10-22 23:37 ` Tim Hockin 2015-10-23 8:12 ` u-uy74 0 siblings, 2 replies; 44+ messages in thread From: Josiah Worcester @ 2015-10-22 23:00 UTC (permalink / raw) To: musl, Rich Felker [-- Attachment #1: Type: text/plain, Size: 14778 bytes --] On Thu, Oct 22, 2015 at 3:37 PM Tim Hockin <thockin@google.com> wrote: > On Thu, Oct 22, 2015 at 2:56 PM, Rich Felker <dalias@libc.org> wrote: > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > >> Hi all, > >> > >> I saw this thread on the web archive but am not sure how to respond to > >> the thread directly as a new joinee of the ML. I hope this finds its > >> way... > > > > No problem; just starting a new thread like this and quoting the old > > one is fine. > > > >> I am one of the developers of Kubernetes and I own the DNS portion, in > >> particular. I desperately want to use Alpine Linux (based on musl) > >> but for now I have to warn people NOT to use it because of this issue. > >> > >> On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: > >> > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: > >> >> I'm writing the wonderful musl project today to open discussion > >> >> about the future possibility of DNS search and domain keyword > >> >> support. We've been using musl libc (by way of Alpine Linux) for > >> >> new development of applications as containers that discover each > >> >> other through DNS and other software defined networking. > >> >> > >> >> In particular, we are starting to use applications like SkyDNS, > >> >> Consul, and Kubernetes, all of which rely on local name > >> >> resolution in some way using search paths. Many users of the > >> >> Alpine Linux container image have also expressed their desire for > >> >> this feature at > >> >> https://github.com/gliderlabs/docker-alpine/issues/8. > >> >> > >> >> On the functional differences between glibc page, the domain and > >> >> search keyword "Support may be added in the future if there is > >> >> demand". So please consider this request an addition to whatever > >> >> demand for the feature already exists. > >> >> > >> >> Thank you for your time and great work on the musl libc project! > >> > > >> > I think this is a reasonable request. I'll look into it more. > >> > > >> > One property I do not want to break is deterministic results, so > >> > when a search is performed, if any step of the search ends with > >> > an error rather than a positive or negative result, the whole > >> > lookup needs to stop and report the error rather than falling > >> > back. Falling back is not safe and creates a situation where DoS > >> > can be used to control which results are returned. > >> > >> I understand your point, though the world at large tends to disagree. > >> Everyone has a primary and secondary `nameserver` record (or should). > >> If the first one times out, try the second. Most resolver libs seem > >> to accept a SERVFAIL response or a timeout as a signal to try the next > >> server, and I would encourage you to do the same. > > > > musl intentionally does not do this because it yields abysmal > > performance. If the first nameserver is overloaded or the packet is > > lost, you suffer several-second lookup latency. > > But at least it works eventually. You're faced with a choice. Wait 2 > seconds for ns1 to timeout and then fail in a way that most apps don't > handle well or wait for 2 seconds and then (usually) get a fast > response from ns2. > > It seems better in every way to eventually succeed, though I agree > it's a bit less visible. > With musl's current design, you get a request to ns1 and ns2, and the first authoritative response wins. So, if ns1 fails then all is well and performance isn't even notably impacted. What you are describing appears to be how you would *have* to implement it if you decide against considering all servers equal, but instead try and serve the union of their responses (that is, wait for timeout and then fail). Consider what would happen if ns1 and ns2 have different responses, but ns1 for whatever reason times out (potentially an attacker). Then you get the results for ns2, even though ns1 is intended to override it. > >> Stopping on positive response or NXDOMAIN seems to be commonly > >> accepted with a caveat. You can't query all nameservers and just take > >> the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of > >> the higher-priority (listed first in resolv.conf) nameservers have > >> timed out or SERVFAIL'ed. You can issue queries in parallel, but you > >> must process responses in order, which is what you describe below. > > > > Timeout or servfail is not sufficient to accept an nxdomain from a > > lower-priority server. To preserve consistency of results under > > transient failure, you actually have to wait for the nxdomain from the > > I have to disagree. Some non-forwarding DNS servers use SERVFAIL to > indicate "I am not serving for that domain" specifically to make the > client move to their next nameserver. if ns1 returns SERVFAIL, try > ns2. If ns1 times out, try ns2. Otherwise what good is ns2? > Note that this means that any condition where ns1 can't be accessed changes what DNS resolves to. If you wish to prevent unexpected behavior when ns1 can't be accessed, you simply have to get a response from ns1 first, and only ever query ns2 when the response from ns1 indicates you may. > > > higher-priority server. Either way, this very much pessimizes usage > > cases like running "netstat" with huge numbers of connections where > > many of the ip addresses fail to reverse. Being able to return > > immediately as soon as any one of the nameservers responds with > > nxdomain makes the difference between a <1s netstat run and a 5-10s > > netstat run. > > Sure it's faster but it's WRONG. Returning a random number would be > faster, too, but it is equally wrong. This is why netstat (and myriad > other tools) has a `-n` flag. Again, musl's design assumes that all nameservers are hosting the same space, and thus a single nxdomain is authoritative. If this is the case, then this is perfectly correct. If it's not the case, then it's wrong (and you need to wait for literally everything to nxdomain, and if there's a timeout from a single server you *need* to report that). > > Thus, if we add extensions to support the kind of result unioning you > > want across multiple nameservers, I think they should be configurable > > and off-by-default. A simple option in resolv.conf could turn them on. > > And there could be options for requiring nxdomain from all servers > > (true union) or just for highest-priority when accepting negative > > results. > > I can't agree with this. It's reasonable to make options for these, > but I think the non-standard behaviors should be off by default. > Consider this from the point of view of a system like Docker or > Kubernetes, which generate resolv.conf for you - they have no idea > what libc your apps are using, so it's unreasonable to ask them to > turn off libc-specific flags. However, the end user knows, and it is > perfectly sane to ask them to explicitly opt-in to non-standard > optimized behaviors. > > >> > While it would be possible to parallelize the search while > >> > serializing the results (i.e. waiting to accept a result from the > >> > second query until the first query finished with a negative > >> > result), I think the consensus during the last round of > >> > discussion of this topic was that the complexity cost is too > >> > great and the benefit too small. Ideally, the first query should > >> > always succeed, anyway. > >> > >> The real world is not ideal. Not all nameservers are identically > >> scoped - you MUST respect the ordering in resolv.conf - to do > >> otherwise is semantically broken. If implementation simplicity means > >> literally doing queries in serial, then that is what you should do. > > > > Even legacy resolvers had the option to rotate the nameservers for > > load-balancing, so I think it's a stretch to say the ordering is > > supposed to be semantic. My view has always been that multiple > > nameservers in resolv.conf are for redundancy, not for serving > > conflicting records. > > You argued above that you should not try a secondary server in case of > timeout or SERVFAIL. Obviously you would not try it on success nor > NXDOMAIN. When do you see a secondary being used at all? > If I understand correctly, what you are expecting is this: Try resolving from ns1. If that domain does not exist on ns1, try ns2. If that domain does not exist on ns2, report failure. The only way to do this in a consistent fashion is actually to fall through to the secondary if and only if you get an nxdomain from ns1. > > As for rotate, note that it is an option and OFF by default in every > mainstream resolver implementation. > > But this point is sort of academic for us - we're moving to a > forwarding nameserver so really there is only the primary nameserver. > We just need you to ask the first nameserver first. > > >> Similarly, you can't just search all search domains in parallel and > >> take the first response. The ordering is meaningful. > > > > Indeed, search domains are like that, because they inherently produce > > ambiguity/overlapping namespaces with different definitions. This is > > why myself and others who weighed in on the original question of > > supporting them were against, but left the option open to revisit the > > topic if users who need them show up. > > Yeah, I scanned the related threads. I understand the issue in > theory, but in practice these are things configured by admins. If > there's a conflict or ambiguity, you should solve that, not jettison > powerful functionality. > > >> > I also have a few questions: > >> > > >> > 1. Do you need multiple search items, or just a single domain? > >> > Any setup with multiple searches necessarily has suboptimal > >> > performance because ndots is not sufficient to make the right > >> > initial choice of query. If you do need this functionality, a > >> > unioning proxy dns server may be a better option than resolv.conf > >> > domain search; it would give much better performance. > >> > >> We use multiple search paths and ndots > 1. I'm not sure what you > >> mean by "unioning" here. Search path ordering is as meaningful as > >> nameserver ordering. You can't avoid making the query for each search > >> suffix in the worst case, and it has the same restriction as > >> nameserver - the search order must be respected. > >> > >> There does seem to be some different implementations that search for > >> the "naked" query first vs last, though. I think the semantically > >> correct (but pessimal performance) is to search for that last. > > > > The traditional behavior is to do the naked query first if the query > > string has at least 'ndots' dots, and to do the search domains first > > otherwise. Also I believe a final dot always suppresses search. > > > > My point was that with ndots=1 (default) and only a single search > > domain, the _expected_ result is that the first query succeed. But if > > you have ndots>1 or multiple search domains, you expect a portion of > > your queries to fall back at least once. This adds significant > > latency. > > It adds latency, but the magnitude is very much determined by the > installation. In our case it is negligible and well worth the cost. > I fear you're optimizing without data - it should be the site-admin's > problem to configure things in an acceptable way. libc doesn't get to > decide what "acceptable" means. > > > In such a situation, you can avoid the additional latency (except on > > the first query of a given record) by running a local caching > > nameserver that does the search and unioning for you, rather than > > having the stub resolver in libc do it. Then subsequent queries > > succeed immediately using the cache. The reason I asked about usage > > case (ndots=1 vs ndots>1, single vs multiple search) is that, in the > > multi-fallback case, it might make more sense (from a performance and > > clean design standpoint) to implement this with a caching nameserver > > on localhost rather than in musl. > > We might be moving to a per-machine local DNS agent, which would cache > as you describe. HOWEVER, there's a pretty important piece that I > guess I left out. Docker and Kubernetes and similar systems run many > containers per machine. Each container has a potentially different > search path. I might run 100 or more containers on a single machine - > I can't run 100 DNS caches, and I can't put that back on users. > Why not? 100 DNS caches shouldn't be particularly expensive. > > So from our perspective the search paths MUST come from the containers > themselves, even if we run a machine local cache to mitigate latency > and SPOF. > > >> > 2. For your intended applications, is there a need to support > >> > ndots>1? Such configurations are generally not friendly to > >> > applications that expect to be able to resolve normal internet > >> > domain lookups, and performance for such lookups will be very bad > >> > (because the search domains first have to fail). > >> > >> DNS is a very lightweight protocol. We have not measured any > >> practical detriment for having 6 search domains and ndots=5. In the > >> normal case it fails very quickly. That aside, it should be my > >> business if I want to (mis)configure my system that way :) > > > > I suspect we have different definitions of quick... :) > > Quick is situational. In a cloud-based mostly-webapp stack, 50ms to > do a name lookup ain't so bad, given the relative infrequency of that > operation. Also most names actually DO resolve on the first or second > search path. > > >> > 3. The glibc behavior is just to swap the order of search when > >> > the query string has >=ndots dots in it, but would it be > >> > acceptable never to try the search domains at all in this case? > >> > That would yield much better performance for nxdomain results and > >> > avoid unexpected positive results due to weird subdomains > >> > existing in your search domain (e.g. a wildcard for > >> > *.us.example.com would cause *.us to wrongly resolve for > >> > non-existant .us domains). > >> > >> I think that would be correct. If I have 3 dots and ndots=2, search > >> paths should be ignored. > > > > Glad we agree on this. > > > > I hope you feel like this conversation is productive. I don't want to > > rule out anything/"say no" right away, but rather try to get a better > > understanding of your requirements first and figure out what makes the > > most sense to do on musl's side. > > Absolutely. I'm happy to engage. Obviously our use case is a bit > outside of what musl was really aiming for, but it offers a really > nice base for very efficient containers. A lot of people want to use > Alpine and it breaks my heart to tell them they can't. > [-- Attachment #2: Type: text/html, Size: 18192 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 23:00 ` Josiah Worcester @ 2015-10-22 23:37 ` Tim Hockin 2015-10-23 4:27 ` Rich Felker 2015-10-23 8:12 ` u-uy74 1 sibling, 1 reply; 44+ messages in thread From: Tim Hockin @ 2015-10-22 23:37 UTC (permalink / raw) To: musl; +Cc: Rich Felker On Thu, Oct 22, 2015 at 4:00 PM, Josiah Worcester <josiahw@gmail.com> wrote: > On Thu, Oct 22, 2015 at 3:37 PM Tim Hockin <thockin@google.com> wrote: >> >> On Thu, Oct 22, 2015 at 2:56 PM, Rich Felker <dalias@libc.org> wrote: >> > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: >> >> Hi all, >> >> >> >> I saw this thread on the web archive but am not sure how to respond to >> >> the thread directly as a new joinee of the ML. I hope this finds its >> >> way... >> > >> > No problem; just starting a new thread like this and quoting the old >> > one is fine. >> > >> >> I am one of the developers of Kubernetes and I own the DNS portion, in >> >> particular. I desperately want to use Alpine Linux (based on musl) >> >> but for now I have to warn people NOT to use it because of this issue. >> >> >> >> On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: >> >> > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: >> >> >> I'm writing the wonderful musl project today to open discussion >> >> >> about the future possibility of DNS search and domain keyword >> >> >> support. We've been using musl libc (by way of Alpine Linux) for >> >> >> new development of applications as containers that discover each >> >> >> other through DNS and other software defined networking. >> >> >> >> >> >> In particular, we are starting to use applications like SkyDNS, >> >> >> Consul, and Kubernetes, all of which rely on local name >> >> >> resolution in some way using search paths. Many users of the >> >> >> Alpine Linux container image have also expressed their desire for >> >> >> this feature at >> >> >> https://github.com/gliderlabs/docker-alpine/issues/8. >> >> >> >> >> >> On the functional differences between glibc page, the domain and >> >> >> search keyword "Support may be added in the future if there is >> >> >> demand". So please consider this request an addition to whatever >> >> >> demand for the feature already exists. >> >> >> >> >> >> Thank you for your time and great work on the musl libc project! >> >> > >> >> > I think this is a reasonable request. I'll look into it more. >> >> > >> >> > One property I do not want to break is deterministic results, so >> >> > when a search is performed, if any step of the search ends with >> >> > an error rather than a positive or negative result, the whole >> >> > lookup needs to stop and report the error rather than falling >> >> > back. Falling back is not safe and creates a situation where DoS >> >> > can be used to control which results are returned. >> >> >> >> I understand your point, though the world at large tends to disagree. >> >> Everyone has a primary and secondary `nameserver` record (or should). >> >> If the first one times out, try the second. Most resolver libs seem >> >> to accept a SERVFAIL response or a timeout as a signal to try the next >> >> server, and I would encourage you to do the same. >> > >> > musl intentionally does not do this because it yields abysmal >> > performance. If the first nameserver is overloaded or the packet is >> > lost, you suffer several-second lookup latency. >> >> But at least it works eventually. You're faced with a choice. Wait 2 >> seconds for ns1 to timeout and then fail in a way that most apps don't >> handle well or wait for 2 seconds and then (usually) get a fast >> response from ns2. >> >> It seems better in every way to eventually succeed, though I agree >> it's a bit less visible. > > > With musl's current design, you get a request to ns1 and ns2, and the first > authoritative response wins. So, if ns1 fails then all is well and > performance isn't even notably impacted. What you are describing appears to > be how you would *have* to implement it if you decide against considering > all servers equal, but instead try and serve the union of their responses > (that is, wait for timeout and then fail). The authoritative-ness is a dimension I had not considered. I could believe that the first authoritative answer wins, but what if you only get a non-authoritative answer? from ns1 and ns2 never responds? > Consider what would happen if ns1 and ns2 have different responses, but ns1 > for whatever reason times out (potentially an attacker). Then you get the > results for ns2, even though ns1 is intended to override it. I agree in theory. And yet this is how most resolvers work today. Are they all broken? >> >> Stopping on positive response or NXDOMAIN seems to be commonly >> >> accepted with a caveat. You can't query all nameservers and just take >> >> the first NXDOMAIN to respond. You can only accept NXDOMAIN if all of >> >> the higher-priority (listed first in resolv.conf) nameservers have >> >> timed out or SERVFAIL'ed. You can issue queries in parallel, but you >> >> must process responses in order, which is what you describe below. >> > >> > Timeout or servfail is not sufficient to accept an nxdomain from a >> > lower-priority server. To preserve consistency of results under >> > transient failure, you actually have to wait for the nxdomain from the >> >> I have to disagree. Some non-forwarding DNS servers use SERVFAIL to >> indicate "I am not serving for that domain" specifically to make the >> client move to their next nameserver. if ns1 returns SERVFAIL, try >> ns2. If ns1 times out, try ns2. Otherwise what good is ns2? > > > Note that this means that any condition where ns1 can't be accessed changes > what DNS resolves to. If you wish to prevent unexpected behavior when ns1 > can't be accessed, you simply have to get a response from ns1 first, and > only ever query ns2 when the response from ns1 indicates you may. If you get a response from ns1, why would you ever go to ns2? >> > higher-priority server. Either way, this very much pessimizes usage >> > cases like running "netstat" with huge numbers of connections where >> > many of the ip addresses fail to reverse. Being able to return >> > immediately as soon as any one of the nameservers responds with >> > nxdomain makes the difference between a <1s netstat run and a 5-10s >> > netstat run. >> >> Sure it's faster but it's WRONG. Returning a random number would be >> faster, too, but it is equally wrong. This is why netstat (and myriad >> other tools) has a `-n` flag. > > > Again, musl's design assumes that all nameservers are hosting the same > space, and thus a single nxdomain is authoritative. If this is the case, > then this is perfectly correct. If it's not the case, then it's wrong (and Agree > you need to wait for literally everything to nxdomain, and if there's a > timeout from a single server you *need* to report that). I don't buy the "everything" part. If we assume that nameserver ordering is priority ordering, the first NXDOMAIN is sufficient. But I can see how this is maybe not the normal case, and we're probably moving to a model more consistent with musl's view (but we still need search :) >> > Thus, if we add extensions to support the kind of result unioning you >> > want across multiple nameservers, I think they should be configurable >> > and off-by-default. A simple option in resolv.conf could turn them on. >> > And there could be options for requiring nxdomain from all servers >> > (true union) or just for highest-priority when accepting negative >> > results. >> >> I can't agree with this. It's reasonable to make options for these, >> but I think the non-standard behaviors should be off by default. >> Consider this from the point of view of a system like Docker or >> Kubernetes, which generate resolv.conf for you - they have no idea >> what libc your apps are using, so it's unreasonable to ask them to >> turn off libc-specific flags. However, the end user knows, and it is >> perfectly sane to ask them to explicitly opt-in to non-standard >> optimized behaviors. >> >> >> > While it would be possible to parallelize the search while >> >> > serializing the results (i.e. waiting to accept a result from the >> >> > second query until the first query finished with a negative >> >> > result), I think the consensus during the last round of >> >> > discussion of this topic was that the complexity cost is too >> >> > great and the benefit too small. Ideally, the first query should >> >> > always succeed, anyway. >> >> >> >> The real world is not ideal. Not all nameservers are identically >> >> scoped - you MUST respect the ordering in resolv.conf - to do >> >> otherwise is semantically broken. If implementation simplicity means >> >> literally doing queries in serial, then that is what you should do. >> > >> > Even legacy resolvers had the option to rotate the nameservers for >> > load-balancing, so I think it's a stretch to say the ordering is >> > supposed to be semantic. My view has always been that multiple >> > nameservers in resolv.conf are for redundancy, not for serving >> > conflicting records. >> >> You argued above that you should not try a secondary server in case of >> timeout or SERVFAIL. Obviously you would not try it on success nor >> NXDOMAIN. When do you see a secondary being used at all? > > > If I understand correctly, what you are expecting is this: > Try resolving from ns1. > If that domain does not exist on ns1, try ns2. > If that domain does not exist on ns2, report failure. > > The only way to do this in a consistent fashion is actually to fall through > to the secondary if and only if you get an nxdomain from ns1. not nxdomain - servfail. If you get nxdomain the server is saying "I know that this domain does not exist". This seems to work on glibc and busybox and every other resolver we have tried except musl. But again, we're not even using this functionality and probably moving to a more "standard" model. >> As for rotate, note that it is an option and OFF by default in every >> mainstream resolver implementation. >> >> But this point is sort of academic for us - we're moving to a >> forwarding nameserver so really there is only the primary nameserver. >> We just need you to ask the first nameserver first. >> >> >> Similarly, you can't just search all search domains in parallel and >> >> take the first response. The ordering is meaningful. >> > >> > Indeed, search domains are like that, because they inherently produce >> > ambiguity/overlapping namespaces with different definitions. This is >> > why myself and others who weighed in on the original question of >> > supporting them were against, but left the option open to revisit the >> > topic if users who need them show up. >> >> Yeah, I scanned the related threads. I understand the issue in >> theory, but in practice these are things configured by admins. If >> there's a conflict or ambiguity, you should solve that, not jettison >> powerful functionality. >> >> >> > I also have a few questions: >> >> > >> >> > 1. Do you need multiple search items, or just a single domain? >> >> > Any setup with multiple searches necessarily has suboptimal >> >> > performance because ndots is not sufficient to make the right >> >> > initial choice of query. If you do need this functionality, a >> >> > unioning proxy dns server may be a better option than resolv.conf >> >> > domain search; it would give much better performance. >> >> >> >> We use multiple search paths and ndots > 1. I'm not sure what you >> >> mean by "unioning" here. Search path ordering is as meaningful as >> >> nameserver ordering. You can't avoid making the query for each search >> >> suffix in the worst case, and it has the same restriction as >> >> nameserver - the search order must be respected. >> >> >> >> There does seem to be some different implementations that search for >> >> the "naked" query first vs last, though. I think the semantically >> >> correct (but pessimal performance) is to search for that last. >> > >> > The traditional behavior is to do the naked query first if the query >> > string has at least 'ndots' dots, and to do the search domains first >> > otherwise. Also I believe a final dot always suppresses search. >> > >> > My point was that with ndots=1 (default) and only a single search >> > domain, the _expected_ result is that the first query succeed. But if >> > you have ndots>1 or multiple search domains, you expect a portion of >> > your queries to fall back at least once. This adds significant >> > latency. >> >> It adds latency, but the magnitude is very much determined by the >> installation. In our case it is negligible and well worth the cost. >> I fear you're optimizing without data - it should be the site-admin's >> problem to configure things in an acceptable way. libc doesn't get to >> decide what "acceptable" means. >> >> > In such a situation, you can avoid the additional latency (except on >> > the first query of a given record) by running a local caching >> > nameserver that does the search and unioning for you, rather than >> > having the stub resolver in libc do it. Then subsequent queries >> > succeed immediately using the cache. The reason I asked about usage >> > case (ndots=1 vs ndots>1, single vs multiple search) is that, in the >> > multi-fallback case, it might make more sense (from a performance and >> > clean design standpoint) to implement this with a caching nameserver >> > on localhost rather than in musl. >> >> We might be moving to a per-machine local DNS agent, which would cache >> as you describe. HOWEVER, there's a pretty important piece that I >> guess I left out. Docker and Kubernetes and similar systems run many >> containers per machine. Each container has a potentially different >> search path. I might run 100 or more containers on a single machine - >> I can't run 100 DNS caches, and I can't put that back on users. > > > Why not? 100 DNS caches shouldn't be particularly expensive. I only have one port 53 to use on the host, and I can't force this to run inside user containers. I guess I could write a split-horizon proxy that expands search paths and caches per-client. That's exceedingly silly given that ONLY musl doesn't work :) It's also somewhat wrong since I will return an address for a name the client did not actually ask for. >> So from our perspective the search paths MUST come from the containers >> themselves, even if we run a machine local cache to mitigate latency >> and SPOF. >> >> >> > 2. For your intended applications, is there a need to support >> >> > ndots>1? Such configurations are generally not friendly to >> >> > applications that expect to be able to resolve normal internet >> >> > domain lookups, and performance for such lookups will be very bad >> >> > (because the search domains first have to fail). >> >> >> >> DNS is a very lightweight protocol. We have not measured any >> >> practical detriment for having 6 search domains and ndots=5. In the >> >> normal case it fails very quickly. That aside, it should be my >> >> business if I want to (mis)configure my system that way :) >> > >> > I suspect we have different definitions of quick... :) >> >> Quick is situational. In a cloud-based mostly-webapp stack, 50ms to >> do a name lookup ain't so bad, given the relative infrequency of that >> operation. Also most names actually DO resolve on the first or second >> search path. >> >> >> > 3. The glibc behavior is just to swap the order of search when >> >> > the query string has >=ndots dots in it, but would it be >> >> > acceptable never to try the search domains at all in this case? >> >> > That would yield much better performance for nxdomain results and >> >> > avoid unexpected positive results due to weird subdomains >> >> > existing in your search domain (e.g. a wildcard for >> >> > *.us.example.com would cause *.us to wrongly resolve for >> >> > non-existant .us domains). >> >> >> >> I think that would be correct. If I have 3 dots and ndots=2, search >> >> paths should be ignored. >> > >> > Glad we agree on this. >> > >> > I hope you feel like this conversation is productive. I don't want to >> > rule out anything/"say no" right away, but rather try to get a better >> > understanding of your requirements first and figure out what makes the >> > most sense to do on musl's side. >> >> Absolutely. I'm happy to engage. Obviously our use case is a bit >> outside of what musl was really aiming for, but it offers a really >> nice base for very efficient containers. A lot of people want to use >> Alpine and it breaks my heart to tell them they can't. ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 23:37 ` Tim Hockin @ 2015-10-23 4:27 ` Rich Felker 2015-10-23 5:13 ` Tim Hockin 0 siblings, 1 reply; 44+ messages in thread From: Rich Felker @ 2015-10-23 4:27 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 04:37:18PM -0700, Tim Hockin wrote: > On Thu, Oct 22, 2015 at 4:00 PM, Josiah Worcester <josiahw@gmail.com> wrote: > > On Thu, Oct 22, 2015 at 3:37 PM Tim Hockin <thockin@google.com> wrote: > >> > >> On Thu, Oct 22, 2015 at 2:56 PM, Rich Felker <dalias@libc.org> wrote: > >> > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > >> >> Hi all, > >> >> > >> >> I saw this thread on the web archive but am not sure how to respond to > >> >> the thread directly as a new joinee of the ML. I hope this finds its > >> >> way... > >> > > >> > No problem; just starting a new thread like this and quoting the old > >> > one is fine. > >> > > >> >> I am one of the developers of Kubernetes and I own the DNS portion, in > >> >> particular. I desperately want to use Alpine Linux (based on musl) > >> >> but for now I have to warn people NOT to use it because of this issue. > >> >> > >> >> On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: > >> >> > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: > >> >> >> I'm writing the wonderful musl project today to open discussion > >> >> >> about the future possibility of DNS search and domain keyword > >> >> >> support. We've been using musl libc (by way of Alpine Linux) for > >> >> >> new development of applications as containers that discover each > >> >> >> other through DNS and other software defined networking. > >> >> >> > >> >> >> In particular, we are starting to use applications like SkyDNS, > >> >> >> Consul, and Kubernetes, all of which rely on local name > >> >> >> resolution in some way using search paths. Many users of the > >> >> >> Alpine Linux container image have also expressed their desire for > >> >> >> this feature at > >> >> >> https://github.com/gliderlabs/docker-alpine/issues/8. > >> >> >> > >> >> >> On the functional differences between glibc page, the domain and > >> >> >> search keyword "Support may be added in the future if there is > >> >> >> demand". So please consider this request an addition to whatever > >> >> >> demand for the feature already exists. > >> >> >> > >> >> >> Thank you for your time and great work on the musl libc project! > >> >> > > >> >> > I think this is a reasonable request. I'll look into it more. > >> >> > > >> >> > One property I do not want to break is deterministic results, so > >> >> > when a search is performed, if any step of the search ends with > >> >> > an error rather than a positive or negative result, the whole > >> >> > lookup needs to stop and report the error rather than falling > >> >> > back. Falling back is not safe and creates a situation where DoS > >> >> > can be used to control which results are returned. > >> >> > >> >> I understand your point, though the world at large tends to disagree. > >> >> Everyone has a primary and secondary `nameserver` record (or should). > >> >> If the first one times out, try the second. Most resolver libs seem > >> >> to accept a SERVFAIL response or a timeout as a signal to try the next > >> >> server, and I would encourage you to do the same. > >> > > >> > musl intentionally does not do this because it yields abysmal > >> > performance. If the first nameserver is overloaded or the packet is > >> > lost, you suffer several-second lookup latency. > >> > >> But at least it works eventually. You're faced with a choice. Wait 2 > >> seconds for ns1 to timeout and then fail in a way that most apps don't > >> handle well or wait for 2 seconds and then (usually) get a fast > >> response from ns2. > >> > >> It seems better in every way to eventually succeed, though I agree > >> it's a bit less visible. > > > > > > With musl's current design, you get a request to ns1 and ns2, and the first > > authoritative response wins. So, if ns1 fails then all is well and > > performance isn't even notably impacted. What you are describing appears to > > be how you would *have* to implement it if you decide against considering > > all servers equal, but instead try and serve the union of their responses > > (that is, wait for timeout and then fail). > > The authoritative-ness is a dimension I had not considered. I could > believe that the first authoritative answer wins, but what if you only > get a non-authoritative answer? from ns1 and ns2 never responds? I don't think Josiah's use of "authoritative" here is consistent with what musl actually does, and it's likely not a useful dimension. A stub resolver is generally not intended to communicate with authoritative nameservers. The current behavior is that a positive or negative (nxdomain) result is treated as concluding the query, and other results (servfail, refused, etc.) are treated as inconclusive and allow the query to continue until it times out. (Also, servfail results in a limited number of immediate retries; this behavior was arrived at based on the behavior of nameservers that fail with servfail.) > > Consider what would happen if ns1 and ns2 have different responses, but ns1 > > for whatever reason times out (potentially an attacker). Then you get the > > results for ns2, even though ns1 is intended to override it. > > I agree in theory. And yet this is how most resolvers work today. > Are they all broken? No, the resolvers are not broken. The configuration is, at least the way I see it. The intent of the resolvers is that they're communicating with redundant servers, not a sequence of overlaid hostname databases. If that assumption is satisfied, there's no problem. BTW I think there are other strong reasons to move to a model based on a local nameserver that does the unioning, not just performance. The most compelling is DNSSEC, which requires a trusted channel between the nameserver and the stub resolver in order for results to be meaningful/trusted. In the future everybody should be running a nameserver on localhost to do DNSSEC signature validation. In that scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted entirely since that's the default, at least on musl). Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 4:27 ` Rich Felker @ 2015-10-23 5:13 ` Tim Hockin 2015-10-23 5:31 ` Rich Felker 0 siblings, 1 reply; 44+ messages in thread From: Tim Hockin @ 2015-10-23 5:13 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 9:27 PM, Rich Felker <dalias@libc.org> wrote: > On Thu, Oct 22, 2015 at 04:37:18PM -0700, Tim Hockin wrote: >> On Thu, Oct 22, 2015 at 4:00 PM, Josiah Worcester <josiahw@gmail.com> wrote: >> > On Thu, Oct 22, 2015 at 3:37 PM Tim Hockin <thockin@google.com> wrote: >> >> >> >> On Thu, Oct 22, 2015 at 2:56 PM, Rich Felker <dalias@libc.org> wrote: >> >> > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: >> >> >> Hi all, >> >> >> >> >> >> I saw this thread on the web archive but am not sure how to respond to >> >> >> the thread directly as a new joinee of the ML. I hope this finds its >> >> >> way... >> >> > >> >> > No problem; just starting a new thread like this and quoting the old >> >> > one is fine. >> >> > >> >> >> I am one of the developers of Kubernetes and I own the DNS portion, in >> >> >> particular. I desperately want to use Alpine Linux (based on musl) >> >> >> but for now I have to warn people NOT to use it because of this issue. >> >> >> >> >> >> On Fri, Sep 04, 2015 at 02:04:29PM -0400, Rich Felker wrote: >> >> >> > On Fri, Sep 04, 2015 at 12:11:36PM -0500, Andy Shinn wrote: >> >> >> >> I'm writing the wonderful musl project today to open discussion >> >> >> >> about the future possibility of DNS search and domain keyword >> >> >> >> support. We've been using musl libc (by way of Alpine Linux) for >> >> >> >> new development of applications as containers that discover each >> >> >> >> other through DNS and other software defined networking. >> >> >> >> >> >> >> >> In particular, we are starting to use applications like SkyDNS, >> >> >> >> Consul, and Kubernetes, all of which rely on local name >> >> >> >> resolution in some way using search paths. Many users of the >> >> >> >> Alpine Linux container image have also expressed their desire for >> >> >> >> this feature at >> >> >> >> https://github.com/gliderlabs/docker-alpine/issues/8. >> >> >> >> >> >> >> >> On the functional differences between glibc page, the domain and >> >> >> >> search keyword "Support may be added in the future if there is >> >> >> >> demand". So please consider this request an addition to whatever >> >> >> >> demand for the feature already exists. >> >> >> >> >> >> >> >> Thank you for your time and great work on the musl libc project! >> >> >> > >> >> >> > I think this is a reasonable request. I'll look into it more. >> >> >> > >> >> >> > One property I do not want to break is deterministic results, so >> >> >> > when a search is performed, if any step of the search ends with >> >> >> > an error rather than a positive or negative result, the whole >> >> >> > lookup needs to stop and report the error rather than falling >> >> >> > back. Falling back is not safe and creates a situation where DoS >> >> >> > can be used to control which results are returned. >> >> >> >> >> >> I understand your point, though the world at large tends to disagree. >> >> >> Everyone has a primary and secondary `nameserver` record (or should). >> >> >> If the first one times out, try the second. Most resolver libs seem >> >> >> to accept a SERVFAIL response or a timeout as a signal to try the next >> >> >> server, and I would encourage you to do the same. >> >> > >> >> > musl intentionally does not do this because it yields abysmal >> >> > performance. If the first nameserver is overloaded or the packet is >> >> > lost, you suffer several-second lookup latency. >> >> >> >> But at least it works eventually. You're faced with a choice. Wait 2 >> >> seconds for ns1 to timeout and then fail in a way that most apps don't >> >> handle well or wait for 2 seconds and then (usually) get a fast >> >> response from ns2. >> >> >> >> It seems better in every way to eventually succeed, though I agree >> >> it's a bit less visible. >> > >> > >> > With musl's current design, you get a request to ns1 and ns2, and the first >> > authoritative response wins. So, if ns1 fails then all is well and >> > performance isn't even notably impacted. What you are describing appears to >> > be how you would *have* to implement it if you decide against considering >> > all servers equal, but instead try and serve the union of their responses >> > (that is, wait for timeout and then fail). >> >> The authoritative-ness is a dimension I had not considered. I could >> believe that the first authoritative answer wins, but what if you only >> get a non-authoritative answer? from ns1 and ns2 never responds? > > I don't think Josiah's use of "authoritative" here is consistent with > what musl actually does, and it's likely not a useful dimension. A > stub resolver is generally not intended to communicate with > authoritative nameservers. The current behavior is that a positive or > negative (nxdomain) result is treated as concluding the query, and > other results (servfail, refused, etc.) are treated as inconclusive > and allow the query to continue until it times out. (Also, servfail > results in a limited number of immediate retries; this behavior was > arrived at based on the behavior of nameservers that fail with > servfail.) > >> > Consider what would happen if ns1 and ns2 have different responses, but ns1 >> > for whatever reason times out (potentially an attacker). Then you get the >> > results for ns2, even though ns1 is intended to override it. >> >> I agree in theory. And yet this is how most resolvers work today. >> Are they all broken? > > No, the resolvers are not broken. The configuration is, at least the > way I see it. The intent of the resolvers is that they're > communicating with redundant servers, not a sequence of overlaid > hostname databases. If that assumption is satisfied, there's no > problem. The way you see it is not how everyone sees it, obviously. And there's not a standard or spec here that I know of. That said, as before, We can probably live with this assumption. > BTW I think there are other strong reasons to move to a model based on > a local nameserver that does the unioning, not just performance. The > most compelling is DNSSEC, which requires a trusted channel between > the nameserver and the stub resolver in order for results to be > meaningful/trusted. In the future everybody should be running a > nameserver on localhost to do DNSSEC signature validation. In that > scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted > entirely since that's the default, at least on musl). I can see a local nameserver doing resolution, but doing search expansion seems like a stretch (and superfluous since it is local). ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:13 ` Tim Hockin @ 2015-10-23 5:31 ` Rich Felker 2015-10-23 5:37 ` Tim Hockin 2015-10-27 0:30 ` Rich Felker 0 siblings, 2 replies; 44+ messages in thread From: Rich Felker @ 2015-10-23 5:31 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 10:13:18PM -0700, Tim Hockin wrote: > >> > Consider what would happen if ns1 and ns2 have different responses, but ns1 > >> > for whatever reason times out (potentially an attacker). Then you get the > >> > results for ns2, even though ns1 is intended to override it. > >> > >> I agree in theory. And yet this is how most resolvers work today. > >> Are they all broken? > > > > No, the resolvers are not broken. The configuration is, at least the > > way I see it. The intent of the resolvers is that they're > > communicating with redundant servers, not a sequence of overlaid > > hostname databases. If that assumption is satisfied, there's no > > problem. > > The way you see it is not how everyone sees it, obviously. And > there's not a standard or spec here that I know of. That said, as > before, We can probably live with this assumption. Indeed. But if you want to consider the overlaid model as the proper one, then as we already noted, all existing implementations have an exploitable bug where an attacker who can cause transient failures can change the result of queries (rather than just making queries fail). That seems rather undesirable. > > BTW I think there are other strong reasons to move to a model based on > > a local nameserver that does the unioning, not just performance. The > > most compelling is DNSSEC, which requires a trusted channel between > > the nameserver and the stub resolver in order for results to be > > meaningful/trusted. In the future everybody should be running a > > nameserver on localhost to do DNSSEC signature validation. In that > > scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted > > entirely since that's the default, at least on musl). > > I can see a local nameserver doing resolution, but doing search > expansion seems like a stretch (and superfluous since it is local). Search would also get a lot of performance benefit from doing in the caching nameserver, but I agree with your assessment that it's a separate issue and that there's no _need_ to do it at that level to ensure correctness. So for now let's focus on a plan for adding suitable search domain support in musl. I believe search only affects DNS queries, not hosts file lookups, right? So it should be at the name_from_dns stage in lookup_name.c. The simplest implementation approach is probably to wrap name_from_dns with a name_from_dns_search function that reads the search domains and repeatedly calls name_from_dns until it gets success. One oddity/ugliness of search that needs to be considered is that querying different address families may lead to differently-sourced results. For example if you query example.us with ndots=2, a search domain of example.com, and the following records present: example.us.example.com A example.us AAAA then an AF_UNSPEC or AF_INET query yields a v4-only result for example.us.example.com while an AF_INET6 query "sees through" to the example.us record because the search of example.com fails. This seems quite ugly and counter-intuitive, but I don't see any way to do better that makes sense. Also, are we agreed that queries with >=ndots dots, or a final dot (as in example.com.) should never be searched (rather than searched after first trying them in the root)? Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:31 ` Rich Felker @ 2015-10-23 5:37 ` Tim Hockin 2015-10-23 6:00 ` Rich Felker 2016-01-29 0:57 ` Rich Felker 2015-10-27 0:30 ` Rich Felker 1 sibling, 2 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-23 5:37 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 10:31 PM, Rich Felker <dalias@libc.org> wrote: >> > BTW I think there are other strong reasons to move to a model based on >> > a local nameserver that does the unioning, not just performance. The >> > most compelling is DNSSEC, which requires a trusted channel between >> > the nameserver and the stub resolver in order for results to be >> > meaningful/trusted. In the future everybody should be running a >> > nameserver on localhost to do DNSSEC signature validation. In that >> > scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted >> > entirely since that's the default, at least on musl). >> >> I can see a local nameserver doing resolution, but doing search >> expansion seems like a stretch (and superfluous since it is local). > > Search would also get a lot of performance benefit from doing in the > caching nameserver, but I agree with your assessment that it's a > separate issue and that there's no _need_ to do it at that level to > ensure correctness. So for now let's focus on a plan for adding > suitable search domain support in musl. Sounds right. > I believe search only affects DNS queries, not hosts file lookups, Also my understanding > right? So it should be at the name_from_dns stage in lookup_name.c. > The simplest implementation approach is probably to wrap name_from_dns > with a name_from_dns_search function that reads the search domains and > repeatedly calls name_from_dns until it gets success. > > One oddity/ugliness of search that needs to be considered is that > querying different address families may lead to differently-sourced > results. For example if you query example.us with ndots=2, a search > domain of example.com, and the following records present: > > example.us.example.com A > example.us AAAA > > then an AF_UNSPEC or AF_INET query yields a v4-only result for > example.us.example.com while an AF_INET6 query "sees through" to the > example.us record because the search of example.com fails. This seems > quite ugly and counter-intuitive, but I don't see any way to do better > that makes sense. If that is how the records are laid out, I think you have to allow it. > Also, are we agreed that queries with >=ndots dots, or a final dot (as > in example.com.) should never be searched (rather than searched after > first trying them in the root)? That sounds right. I saw from a different thread that musl doesn't or didn't do TCP fallbacks - is that still the case? I know we need that for things like large multi-SRV sets (which I do not expect libc to support), and we have some people who have large A sets (which I do expect libc to support). ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:37 ` Tim Hockin @ 2015-10-23 6:00 ` Rich Felker 2015-10-23 6:04 ` Tim Hockin 2016-01-29 0:57 ` Rich Felker 1 sibling, 1 reply; 44+ messages in thread From: Rich Felker @ 2015-10-23 6:00 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 10:37:53PM -0700, Tim Hockin wrote: > I saw from a different thread that musl doesn't or didn't do TCP > fallbacks - is that still the case? I know we need that for things > like large multi-SRV sets (which I do not expect libc to support), and > we have some people who have large A sets (which I do expect libc to > support). Indeed. The only way you can overflow the UDP size limit with the records the stub resolver uses is with a max-length CNAME pointing to a max or near-max length record with little or no overlap to allow for compression. Of course you might run out of space for all the address results in other cases, but the truncated packet will still have usable results. While I'm not aware of any official document to this effect, for practical purposes you just have to avoid making names that long. There are too many nameservers that don't do TCP at all, as well as locked-down networks that don't allow TCP except on a few specific ports, to be able to rely on doing DNS over TCP. Naturally other non-stub-resolver things like zone transfers may need TCP, but that's outside the domain of the stub resolver. Note that the libc res_*/dn_*/ns_* APIs should be capable of working with longer messages over TCP as long as you setup the socket and do the send/recv yourself. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 6:00 ` Rich Felker @ 2015-10-23 6:04 ` Tim Hockin 0 siblings, 0 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-23 6:04 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 11:00 PM, Rich Felker <dalias@libc.org> wrote: > On Thu, Oct 22, 2015 at 10:37:53PM -0700, Tim Hockin wrote: >> I saw from a different thread that musl doesn't or didn't do TCP >> fallbacks - is that still the case? I know we need that for things >> like large multi-SRV sets (which I do not expect libc to support), and >> we have some people who have large A sets (which I do expect libc to >> support). > > Indeed. The only way you can overflow the UDP size limit with the > records the stub resolver uses is with a max-length CNAME pointing to > a max or near-max length record with little or no overlap to allow for > compression. Of course you might run out of space for all the address > results in other cases, but the truncated packet will still have > usable results. While I'm not aware of any official document to this > effect, for practical purposes you just have to avoid making names > that long. There are too many nameservers that don't do TCP at all, as > well as locked-down networks that don't allow TCP except on a few > specific ports, to be able to rely on doing DNS over TCP. Our case is exposing sets of fungible backends as a DNS name with multiple A records. Truncating the set will cause incorrect results for clients who need to discover the whole set. We can cross that bridge when we get there. > Naturally other non-stub-resolver things like zone transfers may need > TCP, but that's outside the domain of the stub resolver. Note that the > libc res_*/dn_*/ns_* APIs should be capable of working with longer > messages over TCP as long as you setup the socket and do the send/recv > yourself. > > Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:37 ` Tim Hockin 2015-10-23 6:00 ` Rich Felker @ 2016-01-29 0:57 ` Rich Felker 1 sibling, 0 replies; 44+ messages in thread From: Rich Felker @ 2016-01-29 0:57 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 10:37:53PM -0700, Tim Hockin wrote: > > right? So it should be at the name_from_dns stage in lookup_name.c. > > The simplest implementation approach is probably to wrap name_from_dns > > with a name_from_dns_search function that reads the search domains and > > repeatedly calls name_from_dns until it gets success. > > > > One oddity/ugliness of search that needs to be considered is that > > querying different address families may lead to differently-sourced > > results. For example if you query example.us with ndots=2, a search > > domain of example.com, and the following records present: > > > > example.us.example.com A > > example.us AAAA > > > > then an AF_UNSPEC or AF_INET query yields a v4-only result for > > example.us.example.com while an AF_INET6 query "sees through" to the > > example.us record because the search of example.com fails. This seems > > quite ugly and counter-intuitive, but I don't see any way to do better > > that makes sense. > > If that is how the records are laid out, I think you have to allow it. It turns out this is a non-issue. If a name lacks the requested A or AAAA record, but has some records, then we get rcode==0 rather than rcode==3 (success rather than NxDomain) with 0 results. In that case we can stop the search rather than continuing and there is no inconsistency. Making this work right required a small fix in musl's current rcode handling but it was easy. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:31 ` Rich Felker 2015-10-23 5:37 ` Tim Hockin @ 2015-10-27 0:30 ` Rich Felker 2015-10-27 0:37 ` Tim Hockin ` (2 more replies) 1 sibling, 3 replies; 44+ messages in thread From: Rich Felker @ 2015-10-27 0:30 UTC (permalink / raw) To: musl On Fri, Oct 23, 2015 at 01:31:09AM -0400, Rich Felker wrote: > > > BTW I think there are other strong reasons to move to a model based on > > > a local nameserver that does the unioning, not just performance. The > > > most compelling is DNSSEC, which requires a trusted channel between > > > the nameserver and the stub resolver in order for results to be > > > meaningful/trusted. In the future everybody should be running a > > > nameserver on localhost to do DNSSEC signature validation. In that > > > scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted > > > entirely since that's the default, at least on musl). > > > > I can see a local nameserver doing resolution, but doing search > > expansion seems like a stretch (and superfluous since it is local). > > Search would also get a lot of performance benefit from doing in the > caching nameserver, but I agree with your assessment that it's a > separate issue and that there's no _need_ to do it at that level to > ensure correctness. So for now let's focus on a plan for adding > suitable search domain support in musl. > > I believe search only affects DNS queries, not hosts file lookups, > right? So it should be at the name_from_dns stage in lookup_name.c. > The simplest implementation approach is probably to wrap name_from_dns > with a name_from_dns_search function that reads the search domains and > repeatedly calls name_from_dns until it gets success. I noticed in the process of trying to draft code to do this that there will be a lot of code duplication with the resolv.conf parsing in res_msend.c, and that this code has some stupid bugs (for example it stops parsing after it gets 3 nameservers, so it might miss options later in the file), so I think I'll take a look at factoring it into a new function to gather all the interesting information from resolv.conf that can be used in both places. A couple additional things I noticed from resolv.conf(5): 1. The default domain used by glibc is not the dns root but rather the domain portion of the local hostname determined by gethostname(). Is there any value in duplicating this? Does anyone want/need it? 2. It's not clear from the documentation of "search" whether its presence overrides/suppresses the "domain" (default or set by resolv.conf) or adds additional searches before or after it. Which should it do? While glibc/legacy behavior is worth looking at, I don't think we need to look at things from a standpoint of exactly duplicating that. Meeting real-world modern application needs while avoiding inconveniencing users with stupid/unwanted behavior should be the primary goal. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-27 0:30 ` Rich Felker @ 2015-10-27 0:37 ` Tim Hockin 2015-10-27 0:45 ` Rich Felker 2015-10-27 8:11 ` u-uy74 2015-11-28 22:48 ` Jan Broer 2 siblings, 1 reply; 44+ messages in thread From: Tim Hockin @ 2015-10-27 0:37 UTC (permalink / raw) To: musl wrt 2) my understanding is that you get at-most-one-of `search` or `domain`. On Mon, Oct 26, 2015 at 5:30 PM, Rich Felker <dalias@libc.org> wrote: > On Fri, Oct 23, 2015 at 01:31:09AM -0400, Rich Felker wrote: >> > > BTW I think there are other strong reasons to move to a model based on >> > > a local nameserver that does the unioning, not just performance. The >> > > most compelling is DNSSEC, which requires a trusted channel between >> > > the nameserver and the stub resolver in order for results to be >> > > meaningful/trusted. In the future everybody should be running a >> > > nameserver on localhost to do DNSSEC signature validation. In that >> > > scheme, resolv.conf would just contain 127.0.0.1 (or could be omitted >> > > entirely since that's the default, at least on musl). >> > >> > I can see a local nameserver doing resolution, but doing search >> > expansion seems like a stretch (and superfluous since it is local). >> >> Search would also get a lot of performance benefit from doing in the >> caching nameserver, but I agree with your assessment that it's a >> separate issue and that there's no _need_ to do it at that level to >> ensure correctness. So for now let's focus on a plan for adding >> suitable search domain support in musl. >> >> I believe search only affects DNS queries, not hosts file lookups, >> right? So it should be at the name_from_dns stage in lookup_name.c. >> The simplest implementation approach is probably to wrap name_from_dns >> with a name_from_dns_search function that reads the search domains and >> repeatedly calls name_from_dns until it gets success. > > I noticed in the process of trying to draft code to do this that there > will be a lot of code duplication with the resolv.conf parsing in > res_msend.c, and that this code has some stupid bugs (for example it > stops parsing after it gets 3 nameservers, so it might miss options > later in the file), so I think I'll take a look at factoring it into a > new function to gather all the interesting information from > resolv.conf that can be used in both places. > > A couple additional things I noticed from resolv.conf(5): > > 1. The default domain used by glibc is not the dns root but rather the > domain portion of the local hostname determined by gethostname(). > Is there any value in duplicating this? Does anyone want/need it? > > 2. It's not clear from the documentation of "search" whether its > presence overrides/suppresses the "domain" (default or set by > resolv.conf) or adds additional searches before or after it. Which > should it do? > > While glibc/legacy behavior is worth looking at, I don't think we need > to look at things from a standpoint of exactly duplicating that. > Meeting real-world modern application needs while avoiding > inconveniencing users with stupid/unwanted behavior should be the > primary goal. > > Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-27 0:37 ` Tim Hockin @ 2015-10-27 0:45 ` Rich Felker 0 siblings, 0 replies; 44+ messages in thread From: Rich Felker @ 2015-10-27 0:45 UTC (permalink / raw) To: musl On Mon, Oct 26, 2015 at 05:37:20PM -0700, Tim Hockin wrote: > wrt 2) my understanding is that you get at-most-one-of `search` or `domain`. Ah, yes, it's lower in the man page after the list of options: The domain and search keywords are mutually exclusive. If more than one instance of these keywords is present, the last instance wins. I see no reason to do differently than glibc (and legacy resolvers) does here; that would just be gratuitous difference for applications to deal with. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-27 0:30 ` Rich Felker 2015-10-27 0:37 ` Tim Hockin @ 2015-10-27 8:11 ` u-uy74 2015-11-28 22:48 ` Jan Broer 2 siblings, 0 replies; 44+ messages in thread From: u-uy74 @ 2015-10-27 8:11 UTC (permalink / raw) To: musl On Mon, Oct 26, 2015 at 08:30:21PM -0400, Rich Felker wrote: > 1. The default domain used by glibc is not the dns root but rather the > domain portion of the local hostname determined by gethostname(). > Is there any value in duplicating this? Does anyone want/need it? This is an annoying design misfeature - annoying because it reflects and perpetuates confusion between DNS as a global name space (for _interfaces_) and the "nodename" (for the _host_), which came from UUCP with a very different name semantics. It just makes no sense, besides reflecting the many existing "well misunderstood" setups and contributing to creation of new ones. If you ask me, don't ever rely on a certain choice of the nodename to be the same as some record in DNS or resemble it. Too bad, a lot of programs have been written with assumptions "a single network interface [with a single DNS name?] and the host administrator certainly having reused the nodename to be set to one of the dns names of the single interface; let's report anything else as a misconfiguration". The purpose of DNS was hardly understood when it was conceived, dealing with "hosts" while actually it concerns "services" (the port part of the adressing was thus left out of DNS until many years later, having been added with SRV). The misunderstanding are now built in into the traditions and among others in interpreting resolv.conf. Using DNS names as if they'd refer to hardware / OS-instances ("hosts") is in fact a misunderstanding, internet is about interfaces, not hosts, but a way too many people fall for it. IOW, please skip this reliance on gethostname() whenever possible at all. Rune ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-27 0:30 ` Rich Felker 2015-10-27 0:37 ` Tim Hockin 2015-10-27 8:11 ` u-uy74 @ 2015-11-28 22:48 ` Jan Broer 2015-11-28 23:20 ` Rich Felker 2016-01-29 0:58 ` Rich Felker 2 siblings, 2 replies; 44+ messages in thread From: Jan Broer @ 2015-11-28 22:48 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1473 bytes --] Hi Rich, i was wondering about the progress of implementing support for 'search' in resolv.conf. Is this on the road map? Thanks! On Tue, Oct 27, 2015 at 1:30 AM, Rich Felker <dalias@libc.org> wrote: > > I noticed in the process of trying to draft code to do this that there > will be a lot of code duplication with the resolv.conf parsing in > res_msend.c, and that this code has some stupid bugs (for example it > stops parsing after it gets 3 nameservers, so it might miss options > later in the file), so I think I'll take a look at factoring it into a > new function to gather all the interesting information from > resolv.conf that can be used in both places. > > A couple additional things I noticed from resolv.conf(5): > > 1. The default domain used by glibc is not the dns root but rather the > domain portion of the local hostname determined by gethostname(). > Is there any value in duplicating this? Does anyone want/need it? > > 2. It's not clear from the documentation of "search" whether its > presence overrides/suppresses the "domain" (default or set by > resolv.conf) or adds additional searches before or after it. Which > should it do? > > While glibc/legacy behavior is worth looking at, I don't think we need > to look at things from a standpoint of exactly duplicating that. > Meeting real-world modern application needs while avoiding > inconveniencing users with stupid/unwanted behavior should be the > primary goal. > > Rich > [-- Attachment #2: Type: text/html, Size: 1994 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-11-28 22:48 ` Jan Broer @ 2015-11-28 23:20 ` Rich Felker 2015-11-29 3:06 ` Jan Broer 2016-01-29 0:58 ` Rich Felker 1 sibling, 1 reply; 44+ messages in thread From: Rich Felker @ 2015-11-28 23:20 UTC (permalink / raw) To: musl On Sat, Nov 28, 2015 at 11:48:09PM +0100, Jan Broer wrote: > Hi Rich, i was wondering about the progress of implementing support for > 'search' in resolv.conf. Is this on the road map? I started work on implementing it when this discussion wrapped up and realized that it made sense to do a little bit of refactoring first and fix some (non-serious, but possibly annoying) bugs in resolv.conf parsing at the same time. I added it to the roadmap for this release cycle and hope to get to it soon. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-11-28 23:20 ` Rich Felker @ 2015-11-29 3:06 ` Jan Broer 0 siblings, 0 replies; 44+ messages in thread From: Jan Broer @ 2015-11-29 3:06 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 645 bytes --] That sounds great and is much appreciated! :-) On Sun, Nov 29, 2015 at 12:20 AM, Rich Felker <dalias@libc.org> wrote: > On Sat, Nov 28, 2015 at 11:48:09PM +0100, Jan Broer wrote: > > Hi Rich, i was wondering about the progress of implementing support for > > 'search' in resolv.conf. Is this on the road map? > > I started work on implementing it when this discussion wrapped up and > realized that it made sense to do a little bit of refactoring first > and fix some (non-serious, but possibly annoying) bugs in resolv.conf > parsing at the same time. I added it to the roadmap for this release > cycle and hope to get to it soon. > > Rich > [-- Attachment #2: Type: text/html, Size: 1061 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-11-28 22:48 ` Jan Broer 2015-11-28 23:20 ` Rich Felker @ 2016-01-29 0:58 ` Rich Felker 1 sibling, 0 replies; 44+ messages in thread From: Rich Felker @ 2016-01-29 0:58 UTC (permalink / raw) To: musl On Sat, Nov 28, 2015 at 11:48:09PM +0100, Jan Broer wrote: > Hi Rich, i was wondering about the progress of implementing support for > 'search' in resolv.conf. Is this on the road map? > > Thanks! I'm working on it now and plan to have it committed soon. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 23:00 ` Josiah Worcester 2015-10-22 23:37 ` Tim Hockin @ 2015-10-23 8:12 ` u-uy74 2015-10-23 9:35 ` Laurent Bercot 1 sibling, 1 reply; 44+ messages in thread From: u-uy74 @ 2015-10-23 8:12 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 11:00:32PM +0000, Josiah Worcester wrote: > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > But at least it works eventually. You're faced with a choice. Wait 2 > > seconds for ns1 to timeout and then fail in a way that most apps don't > > handle well or wait for 2 seconds and then (usually) get a fast > > response from ns2. > > > > It seems better in every way to eventually succeed, though I agree > > it's a bit less visible. > Consider what would happen if ns1 and ns2 have different responses, but ns1 > for whatever reason times out (potentially an attacker). Then you get the > results for ns2, even though ns1 is intended to override it. > > I have to disagree. Some non-forwarding DNS servers use SERVFAIL to > > indicate "I am not serving for that domain" specifically to make the > > client move to their next nameserver. if ns1 returns SERVFAIL, try > > ns2. If ns1 times out, try ns2. Otherwise what good is ns2? > Note that this means that any condition where ns1 can't be accessed changes > what DNS resolves to. If you wish to prevent unexpected behavior when ns1 > can't be accessed, you simply have to get a response from ns1 first, and > only ever query ns2 when the response from ns1 indicates you may. > > Sure it's faster but it's WRONG. Returning a random number would be > > faster, too, but it is equally wrong. This is why netstat (and myriad > > other tools) has a `-n` flag. > > Again, musl's design assumes that all nameservers are hosting the same > space, and thus a single nxdomain is authoritative. If this is the case, > then this is perfectly correct. If it's not the case, then it's wrong (and > you need to wait for literally everything to nxdomain, and if there's a > timeout from a single server you *need* to report that). > > search path. I might run 100 or more containers on a single machine - > > I can't run 100 DNS caches, and I can't put that back on users. > > Why not? 100 DNS caches shouldn't be particularly expensive. +1 to the level 1 comments +1 to solving the particular problem of a certain deployment (the massive reliance on search paths) inside the deployment, not in a general purpose library It looks unrealistic to both have redundancy and differing/conflicting data on different servers in a consistent fashion, by the mere mechanisms of resolv.conf. Nice that musl chose consistency and redundancy. Good if there is an option suitable for systems which need differing functionality, but please not at an extra cost for other parties. A DNS cache looks of course like the right option for fine tuning the desired resolution behaviour. Rune ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 8:12 ` u-uy74 @ 2015-10-23 9:35 ` Laurent Bercot 2015-10-23 12:23 ` Laurent Bercot 2015-10-23 15:57 ` Tim Hockin 0 siblings, 2 replies; 44+ messages in thread From: Laurent Bercot @ 2015-10-23 9:35 UTC (permalink / raw) To: musl On 23/10/2015 10:12, u-uy74@aetey.se wrote: > +1 to the level 1 comments > > +1 to solving the particular problem of a certain deployment > (the massive reliance on search paths) > inside the deployment, not in a general purpose library > > It looks unrealistic to both have redundancy and differing/conflicting > data on different servers in a consistent fashion, by the mere mechanisms > of resolv.conf. > > Nice that musl chose consistency and redundancy. > > Good if there is an option suitable for systems which need differing > functionality, but please not at an extra cost for other parties. > > A DNS cache looks of course like the right option for fine tuning the > desired resolution behaviour. +1 to all this. "search" in /etc/resolv.conf was *never* meant to list resolvers serving different results. It was always meant to list backups serving consistent results. Any other use of "search" lines is abusing the mechanism; if it works with glibc, it is a testimony that glibc doesn't care about consistency, not an argument that it is a valid use case. The right way to proceed is to implement an ad-hoc resolver that suits the specific wanted usage, not to modify a libc that implements the proper behaviour. -- Laurent ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 9:35 ` Laurent Bercot @ 2015-10-23 12:23 ` Laurent Bercot 2015-10-23 15:57 ` Tim Hockin 1 sibling, 0 replies; 44+ messages in thread From: Laurent Bercot @ 2015-10-23 12:23 UTC (permalink / raw) To: musl On 23/10/2015 11:35, Laurent Bercot wrote: > "search" in /etc/resolv.conf was *never* meant to list resolvers > serving different results. I meant "nameserver", of course. :) ("search" is also abused, but that's another topic.) -- Laurent ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 9:35 ` Laurent Bercot 2015-10-23 12:23 ` Laurent Bercot @ 2015-10-23 15:57 ` Tim Hockin 1 sibling, 0 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-23 15:57 UTC (permalink / raw) To: musl On Fri, Oct 23, 2015 at 2:35 AM, Laurent Bercot <ska-dietlibc@skarnet.org> wrote: > On 23/10/2015 10:12, u-uy74@aetey.se wrote: >> >> +1 to the level 1 comments >> >> +1 to solving the particular problem of a certain deployment >> (the massive reliance on search paths) >> inside the deployment, not in a general purpose library >> >> It looks unrealistic to both have redundancy and differing/conflicting >> data on different servers in a consistent fashion, by the mere mechanisms >> of resolv.conf. >> >> Nice that musl chose consistency and redundancy. >> >> Good if there is an option suitable for systems which need differing >> functionality, but please not at an extra cost for other parties. >> >> A DNS cache looks of course like the right option for fine tuning the >> desired resolution behaviour. > > > +1 to all this. > "search" in /etc/resolv.conf was *never* meant to list resolvers > serving different results. It was always meant to list backups citation needed ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-22 21:24 Tim Hockin 2015-10-22 21:56 ` Rich Felker @ 2015-10-23 5:26 ` Kurt H Maier 2015-10-24 21:33 ` Tim Hockin 1 sibling, 1 reply; 44+ messages in thread From: Kurt H Maier @ 2015-10-23 5:26 UTC (permalink / raw) To: musl On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > I understand your point, though the world at large tends to disagree. The world at large uses bad software. Please don't use this sort of reasoning as a justification for and embrace-extend operation on actual standards. > The real world is not ideal. Not all nameservers are identically > scoped - you MUST respect the ordering in resolv.conf - to do > otherwise is semantically broken. If implementation simplicity means > literally doing queries in serial, then that is what you should do. You absolutely cannot respect the ordering in resolv.conf; at least not if you're relying on someone else's resolver. If the orchestration software depends on specific results being returned in particular orders, the orchestration software should provide a mechanism to generate them. > Similarly, you can't just search all search domains in parallel and > take the first response. The ordering is meaningful. It should not be, and more to the point will not reliably be, meaningful. You are arguing for introducing performance penalties into musl that do not affect you but do very much affect lots of other users. I hope they do not happen -- musl is not the right place to fix your problem. khm ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-23 5:26 ` Kurt H Maier @ 2015-10-24 21:33 ` Tim Hockin 2015-10-24 21:57 ` Kurt H Maier 2015-10-24 22:02 ` Rich Felker 0 siblings, 2 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-24 21:33 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1666 bytes --] On Oct 24, 2015 12:20 PM, "Kurt H Maier" <khm@sdf.org> wrote: > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > > > I understand your point, though the world at large tends to disagree. > > The world at large uses bad software. Please don't use this sort of > reasoning as a justification for and embrace-extend operation on actual > standards. Where is the standard that defines ordering semantics in resolv.conf? > > The real world is not ideal. Not all nameservers are identically > > scoped - you MUST respect the ordering in resolv.conf - to do > > otherwise is semantically broken. If implementation simplicity means > > literally doing queries in serial, then that is what you should do. > > You absolutely cannot respect the ordering in resolv.conf; at least not > if you're relying on someone else's resolver. If the orchestration > software depends on specific results being returned in particular > orders, the orchestration software should provide a mechanism to > generate them. > > > Similarly, you can't just search all search domains in parallel and > > take the first response. The ordering is meaningful. > > It should not be, and more to the point will not reliably be, > meaningful. Search has to be ordered. You can not possibly argue otherwise? > You are arguing for introducing performance penalties into musl that do > not affect you but do very much affect lots of other users. I hope they > do not happen -- musl is not the right place to fix your problem. I am arguing for adding a very standard feature (search) to open musl to a whole new space of users. Nobody is forcing you to use search paths or ndots. [-- Attachment #2: Type: text/html, Size: 2083 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 21:33 ` Tim Hockin @ 2015-10-24 21:57 ` Kurt H Maier 2015-10-24 23:31 ` Rich Felker 2015-10-24 22:02 ` Rich Felker 1 sibling, 1 reply; 44+ messages in thread From: Kurt H Maier @ 2015-10-24 21:57 UTC (permalink / raw) To: musl On Sat, Oct 24, 2015 at 02:33:31PM -0700, Tim Hockin wrote: > Where is the standard that defines ordering semantics in resolv.conf? There isn't one, but that doesn't mean you get to ensconce the behavior you like. That was my point. > Search has to be ordered. You can not possibly argue otherwise? I can and do. > I am arguing for adding a very standard feature (search) to open musl to a > whole new space of users. You're not arguing for simple inclusion of search; you're arguing for specific behavior that is impossible to implement in a sane manner, because that's how glibc does it. I am not against domain and search, but I don't just want functionality poorly added just to check some box on a random other project's wishlist. > Nobody is forcing you to use search paths or ndots. Not sure how this is relevant? Nobody's forcing anyone to do anything. What is your point? khm ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 21:57 ` Kurt H Maier @ 2015-10-24 23:31 ` Rich Felker 0 siblings, 0 replies; 44+ messages in thread From: Rich Felker @ 2015-10-24 23:31 UTC (permalink / raw) To: musl On Sat, Oct 24, 2015 at 05:57:10PM -0400, Kurt H Maier wrote: > On Sat, Oct 24, 2015 at 02:33:31PM -0700, Tim Hockin wrote: > > Where is the standard that defines ordering semantics in resolv.conf? > > There isn't one, but that doesn't mean you get to ensconce the behavior > you like. That was my point. > > > Search has to be ordered. You can not possibly argue otherwise? > > I can and do. > > > I am arguing for adding a very standard feature (search) to open musl to a > > whole new space of users. > > You're not arguing for simple inclusion of search; At this point that actually does seem to be all that Tim is asking for. I think we're in agreement that there are multiple problems with trying to use ordered nameservers to overlay inconsistent data, and that even under the way glibc treats this setup, there are subtle problems. > you're arguing for > specific behavior that is impossible to implement in a sane manner, > because that's how glibc does it. I am not against domain and search, > but I don't just want functionality poorly added just to check some box > on a random other project's wishlist. > > > Nobody is forcing you to use search paths or ndots. > > Not sure how this is relevant? Nobody's forcing anyone to do anything. > What is your point? Could you try to hold off on the hostility? I don't think there's any actual disagreement left here. Support for search domains was tabled but left open for future consideration back when the last phase of dns overhaul was done, pending actual requests/usage-cases needing them. It seems like they can be added in an inexpensive and ubobtrusive way, so I don't think it should really be controversial. Note that, as I said before, search really does have to be ordered. Otherwise you have inconsistent/non-deterministic results that can be controlled by inducing intermittent failures (DoS). The proper way to handle search is the same way the current fallback sequence (ip literal, hosts file, dns) is done: a positive result at any step ends the query, a negative result causes fallback to the next search element, and any failure at any stage causes the whole query to fail with an appropriate error. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 21:33 ` Tim Hockin 2015-10-24 21:57 ` Kurt H Maier @ 2015-10-24 22:02 ` Rich Felker 2015-10-24 22:32 ` Tim Hockin ` (3 more replies) 1 sibling, 4 replies; 44+ messages in thread From: Rich Felker @ 2015-10-24 22:02 UTC (permalink / raw) To: musl On Sat, Oct 24, 2015 at 02:33:31PM -0700, Tim Hockin wrote: > On Oct 24, 2015 12:20 PM, "Kurt H Maier" <khm@sdf.org> wrote: > > > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > > > > > I understand your point, though the world at large tends to disagree. > > > > The world at large uses bad software. Please don't use this sort of > > reasoning as a justification for and embrace-extend operation on actual > > standards. > > Where is the standard that defines ordering semantics in resolv.conf? I don't think it's useful to argue about intent unless someone wants to dig up history and find out what the original implementors intended, and even then it's rather arbitrary whether people would care about that intent since it doesn't seem to have been documented explicitly. My view is that it's more useful to consider the consequences of both interpretations and draw a conclusion that one should be preferred from the bad consequences of the other. > > > The real world is not ideal. Not all nameservers are identically > > > scoped - you MUST respect the ordering in resolv.conf - to do > > > otherwise is semantically broken. If implementation simplicity means > > > literally doing queries in serial, then that is what you should do. > > > > You absolutely cannot respect the ordering in resolv.conf; at least not > > if you're relying on someone else's resolver. If the orchestration > > software depends on specific results being returned in particular > > orders, the orchestration software should provide a mechanism to > > generate them. > > > > > Similarly, you can't just search all search domains in parallel and > > > take the first response. The ordering is meaningful. > > > > It should not be, and more to the point will not reliably be, > > meaningful. > > Search has to be ordered. You can not possibly argue otherwise? Indeed, search certainly has to be ordered. Otherwise results are most definitely non-deterministic. The trivial example would be looking up "www" with 2 or more search domains. In any case, it was discussed way back that, while parallel search could be implemented as long as a result from search domain N is not accepted until negative results from domains 1 to N-1 are received, the implementation complexity cost was too high relative to the value of such a feature. > > You are arguing for introducing performance penalties into musl that do > > not affect you but do very much affect lots of other users. I hope they > > do not happen -- musl is not the right place to fix your problem. > > I am arguing for adding a very standard feature (search) to open musl to a > whole new space of users. Nobody is forcing you to use search paths or > ndots. The only place adding search support might negatively impact existing musl users is by causing hostnames with no dots to be queried with the (often useless and unwanted) default domain set by dhcp before failing. My preference would probably be having musl default to ndots=0 rather than ndots=1 so that search has to be enabled explicitly. Are there any reasons this would be undesirable? Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 22:02 ` Rich Felker @ 2015-10-24 22:32 ` Tim Hockin 2015-10-25 8:20 ` u-uy74 ` (2 subsequent siblings) 3 siblings, 0 replies; 44+ messages in thread From: Tim Hockin @ 2015-10-24 22:32 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 3763 bytes --] On Oct 24, 2015 3:02 PM, "Rich Felker" <dalias@libc.org> wrote: > > On Sat, Oct 24, 2015 at 02:33:31PM -0700, Tim Hockin wrote: > > On Oct 24, 2015 12:20 PM, "Kurt H Maier" <khm@sdf.org> wrote: > > > > > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > > > > > > > I understand your point, though the world at large tends to disagree. > > > > > > The world at large uses bad software. Please don't use this sort of > > > reasoning as a justification for and embrace-extend operation on actual > > > standards. > > > > Where is the standard that defines ordering semantics in resolv.conf? > > I don't think it's useful to argue about intent unless someone wants > to dig up history and find out what the original implementors My point was only that you can't call this "embrace and extend" of a "standard" when no such standard exists AND the most widely used software doesn't conform to your supposed standard. I can see both sides of the argument and have already conceded this as the least important part of the proposal, for our use case. > intended, and even then it's rather arbitrary whether people would > care about that intent since it doesn't seem to have been documented > explicitly. My view is that it's more useful to consider the > consequences of both interpretations and draw a conclusion that one > should be preferred from the bad consequences of the other. > > > > > The real world is not ideal. Not all nameservers are identically > > > > scoped - you MUST respect the ordering in resolv.conf - to do > > > > otherwise is semantically broken. If implementation simplicity means > > > > literally doing queries in serial, then that is what you should do. > > > > > > You absolutely cannot respect the ordering in resolv.conf; at least not > > > if you're relying on someone else's resolver. If the orchestration > > > software depends on specific results being returned in particular > > > orders, the orchestration software should provide a mechanism to > > > generate them. > > > > > > > Similarly, you can't just search all search domains in parallel and > > > > take the first response. The ordering is meaningful. > > > > > > It should not be, and more to the point will not reliably be, > > > meaningful. > > > > Search has to be ordered. You can not possibly argue otherwise? > > Indeed, search certainly has to be ordered. Otherwise results are most > definitely non-deterministic. The trivial example would be looking up > "www" with 2 or more search domains. OK, I thought for a minute you went off the deep end :) > In any case, it was discussed way back that, while parallel search > could be implemented as long as a result from search domain N is not > accepted until negative results from domains 1 to N-1 are received, > the implementation complexity cost was too high relative to the value > of such a feature. Agree > > > You are arguing for introducing performance penalties into musl that do > > > not affect you but do very much affect lots of other users. I hope they > > > do not happen -- musl is not the right place to fix your problem. > > > > I am arguing for adding a very standard feature (search) to open musl to a > > whole new space of users. Nobody is forcing you to use search paths or > > ndots. > > The only place adding search support might negatively impact existing > musl users is by causing hostnames with no dots to be queried with the > (often useless and unwanted) default domain set by dhcp before > failing. My preference would probably be having musl default to > ndots=0 rather than ndots=1 so that search has to be enabled > explicitly. Are there any reasons this would be undesirable? So you want "naked" names to not use search at all? Is that really useful? [-- Attachment #2: Type: text/html, Size: 4825 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 22:02 ` Rich Felker 2015-10-24 22:32 ` Tim Hockin @ 2015-10-25 8:20 ` u-uy74 2015-10-25 13:06 ` Jan Broer 2015-10-26 1:26 ` Isaac Dunham 3 siblings, 0 replies; 44+ messages in thread From: u-uy74 @ 2015-10-25 8:20 UTC (permalink / raw) To: musl On Sat, Oct 24, 2015 at 06:02:15PM -0400, Rich Felker wrote: > The only place adding search support might negatively impact existing > musl users is by causing hostnames with no dots to be queried with the > (often useless and unwanted) default domain set by dhcp before > failing. My preference would probably be having musl default to > ndots=0 rather than ndots=1 so that search has to be enabled > explicitly. +1 > Are there any reasons this would be undesirable? I don't think so. On the other side, the "design" of resolv.conf is far from well thought out or consequent (given that there is no standard, to begin with). So whatever you do, there of course will be people expecting/wanting a different behaviour. Sigh. Rune ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 22:02 ` Rich Felker 2015-10-24 22:32 ` Tim Hockin 2015-10-25 8:20 ` u-uy74 @ 2015-10-25 13:06 ` Jan Broer 2015-10-25 13:19 ` u-uy74 2015-10-25 19:08 ` Rich Felker 2015-10-26 1:26 ` Isaac Dunham 3 siblings, 2 replies; 44+ messages in thread From: Jan Broer @ 2015-10-25 13:06 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 4594 bytes --] > The only place adding search support might negatively impact existing > musl users is by causing hostnames with no dots to be queried with the > (often useless and unwanted) default domain set by dhcp before > failing. My preference would probably be having musl default to > ndots=0 rather than ndots=1 so that search has to be enabled > explicitly. Are there any reasons this would be undesirable? I don't think it is a good idea do default to ndots=0. This would essentially break search for systems where resolv.conf values are managed by the DHCP server. DHCP expects search to work when there is at least one entry in the domain-search option returned by the DHCP server. There is no DHCP option for configuring ndots (see http://linux.die.net/man/5/dhcp-options) and therefore search would not work in these configurations when ndots defaults to 0. Also, i don't think your argument for setting ndots to 0 by default is valid: Search always to be enabled explicitly even when the default for ndots=1: There is no search unless the user explicitely writes search paths in his resolv.conf or configures his DHCP client to get the domain-search option via DHCP. So the resolvers search functionality is already something the user has to enable explicitely. On Sun, Oct 25, 2015 at 12:02 AM, Rich Felker <dalias@libc.org> wrote: > On Sat, Oct 24, 2015 at 02:33:31PM -0700, Tim Hockin wrote: > > On Oct 24, 2015 12:20 PM, "Kurt H Maier" <khm@sdf.org> wrote: > > > > > > On Thu, Oct 22, 2015 at 02:24:11PM -0700, Tim Hockin wrote: > > > > > > > > I understand your point, though the world at large tends to disagree. > > > > > > The world at large uses bad software. Please don't use this sort of > > > reasoning as a justification for and embrace-extend operation on actual > > > standards. > > > > Where is the standard that defines ordering semantics in resolv.conf? > > I don't think it's useful to argue about intent unless someone wants > to dig up history and find out what the original implementors > intended, and even then it's rather arbitrary whether people would > care about that intent since it doesn't seem to have been documented > explicitly. My view is that it's more useful to consider the > consequences of both interpretations and draw a conclusion that one > should be preferred from the bad consequences of the other. > > > > > The real world is not ideal. Not all nameservers are identically > > > > scoped - you MUST respect the ordering in resolv.conf - to do > > > > otherwise is semantically broken. If implementation simplicity means > > > > literally doing queries in serial, then that is what you should do. > > > > > > You absolutely cannot respect the ordering in resolv.conf; at least not > > > if you're relying on someone else's resolver. If the orchestration > > > software depends on specific results being returned in particular > > > orders, the orchestration software should provide a mechanism to > > > generate them. > > > > > > > Similarly, you can't just search all search domains in parallel and > > > > take the first response. The ordering is meaningful. > > > > > > It should not be, and more to the point will not reliably be, > > > meaningful. > > > > Search has to be ordered. You can not possibly argue otherwise? > > Indeed, search certainly has to be ordered. Otherwise results are most > definitely non-deterministic. The trivial example would be looking up > "www" with 2 or more search domains. > > In any case, it was discussed way back that, while parallel search > could be implemented as long as a result from search domain N is not > accepted until negative results from domains 1 to N-1 are received, > the implementation complexity cost was too high relative to the value > of such a feature. > > > > You are arguing for introducing performance penalties into musl that do > > > not affect you but do very much affect lots of other users. I hope > they > > > do not happen -- musl is not the right place to fix your problem. > > > > I am arguing for adding a very standard feature (search) to open musl to > a > > whole new space of users. Nobody is forcing you to use search paths or > > ndots. > > The only place adding search support might negatively impact existing > musl users is by causing hostnames with no dots to be queried with the > (often useless and unwanted) default domain set by dhcp before > failing. My preference would probably be having musl default to > ndots=0 rather than ndots=1 so that search has to be enabled > explicitly. Are there any reasons this would be undesirable? > > Rich > [-- Attachment #2: Type: text/html, Size: 5763 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-25 13:06 ` Jan Broer @ 2015-10-25 13:19 ` u-uy74 2015-10-25 13:39 ` Jan Broer 2015-10-25 19:08 ` Rich Felker 1 sibling, 1 reply; 44+ messages in thread From: u-uy74 @ 2015-10-25 13:19 UTC (permalink / raw) To: musl On Sun, Oct 25, 2015 at 02:06:29PM +0100, Jan Broer wrote: > I don't think it is a good idea do default to ndots=0. This would > essentially break search for systems where resolv.conf values are managed > by the DHCP server. DHCP expects search to work when there is at least one > entry in the domain-search option returned by the DHCP server. There is no > DHCP option for configuring ndots (see > http://linux.die.net/man/5/dhcp-options) and therefore search would not > work in these configurations when ndots defaults to 0. To be fair, it is not the dhcp server who generates the resolv.conf but the tools on the computer itself (the dhcp client implementation). IOW it is the administrator of the computer who decides what resolv.conf shall look like, even if she/he possibly uses certain data from dhcp. Rune ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-25 13:19 ` u-uy74 @ 2015-10-25 13:39 ` Jan Broer 2015-10-25 14:08 ` u-uy74 0 siblings, 1 reply; 44+ messages in thread From: Jan Broer @ 2015-10-25 13:39 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1236 bytes --] I don't think we disagree there. The user decides whether search is used by either setting the search keyword in resolv.conf or by configuring his DHCP client (dhclient.conf) to fetch the domain-search option from the DHCP server. Thats why there really is no need to misuse ndots option as "on/off switch" for searching. On Sun, Oct 25, 2015 at 2:19 PM, <u-uy74@aetey.se> wrote: > On Sun, Oct 25, 2015 at 02:06:29PM +0100, Jan Broer wrote: > > I don't think it is a good idea do default to ndots=0. This would > > essentially break search for systems where resolv.conf values are managed > > by the DHCP server. DHCP expects search to work when there is at least > one > > entry in the domain-search option returned by the DHCP server. There is > no > > DHCP option for configuring ndots (see > > http://linux.die.net/man/5/dhcp-options) and therefore search would not > > work in these configurations when ndots defaults to 0. > > To be fair, it is not the dhcp server who generates the resolv.conf but > the tools on the computer itself (the dhcp client implementation). > IOW it is the administrator of the computer who decides what resolv.conf > shall look like, even if she/he possibly uses certain data from dhcp. > > Rune > > [-- Attachment #2: Type: text/html, Size: 1739 bytes --] ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-25 13:39 ` Jan Broer @ 2015-10-25 14:08 ` u-uy74 0 siblings, 0 replies; 44+ messages in thread From: u-uy74 @ 2015-10-25 14:08 UTC (permalink / raw) To: musl On Sun, Oct 25, 2015 at 02:39:36PM +0100, Jan Broer wrote: > I don't think we disagree there. The user decides whether search is used by > either setting the search keyword in resolv.conf or by configuring his DHCP > client (dhclient.conf) to fetch the domain-search option from the DHCP > server. You seem to assume that the user can not influence the ndots option in the resolv.conf if the corresponding host is to use some other data delivered by dhcp (i.e. for building resolv.conf). For certain setups this can be true, probably meaning that they [rely on a non-existent standard and] do not bother having proper control over resolv.conf? Otherwise, in a general case, we disagree. Isn't it reasonable that people wishing to use extra and "somewhat/potentially expensive" features of resolv.conf have to put effort into actually activating them? Regards, Rune ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-25 13:06 ` Jan Broer 2015-10-25 13:19 ` u-uy74 @ 2015-10-25 19:08 ` Rich Felker 1 sibling, 0 replies; 44+ messages in thread From: Rich Felker @ 2015-10-25 19:08 UTC (permalink / raw) To: musl On Sun, Oct 25, 2015 at 02:06:29PM +0100, Jan Broer wrote: > > The only place adding search support might negatively impact existing > > musl users is by causing hostnames with no dots to be queried with the > > (often useless and unwanted) default domain set by dhcp before > > failing. My preference would probably be having musl default to > > ndots=0 rather than ndots=1 so that search has to be enabled > > explicitly. Are there any reasons this would be undesirable? > > > I don't think it is a good idea do default to ndots=0. This would > essentially break search for systems where resolv.conf values are managed > by the DHCP server. DHCP expects search to work when there is at least one > entry in the domain-search option returned by the DHCP server. There is no > DHCP option for configuring ndots (see > http://linux.die.net/man/5/dhcp-options) and therefore search would not > work in these configurations when ndots defaults to 0. What I've found is that almost all dhcp setups include a search domain, and dhcp clients set it in resolv.conf, but it's rare that the user actually wants to search this domain. For example do Comcast (random ISP to pick on) customers actually want random bare (no domain) queries like "foo" to be tried as "foo.comcast.com"? Almost surely not. Of course, as noted before, only bare queries are affected if ndots=1, so the only behavioral change is that some such queries may unexpectedly succeed (if they exist in the search domain) or take longer to fail (since two or more queries have to fail). > Also, i don't think your argument for setting ndots to 0 by default is > valid: > > Search always to be enabled explicitly even when the default for ndots=1: > There is no search unless the user explicitely writes search paths in his > resolv.conf or configures his DHCP client to get the domain-search option > via DHCP. > > So the resolvers search functionality is already something the user has to > enable explicitely. My experience is that dhcp clients do this by default. Of course there's a good argument to be made that fixing this should be done at the dhcp client level (or resolv.conf updating system level) rather than having libc try to 'fix' it. There's certainly value to having consistent behavior between programs using different libcs on the same system, so perhaps sticking with ndots=1 is better. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-24 22:02 ` Rich Felker ` (2 preceding siblings ...) 2015-10-25 13:06 ` Jan Broer @ 2015-10-26 1:26 ` Isaac Dunham 2015-10-26 15:35 ` Rich Felker 3 siblings, 1 reply; 44+ messages in thread From: Isaac Dunham @ 2015-10-26 1:26 UTC (permalink / raw) To: musl On Sat, Oct 24, 2015 at 06:02:15PM -0400, Rich Felker wrote: > The only place adding search support might negatively impact existing > musl users is by causing hostnames with no dots to be queried with the > (often useless and unwanted) default domain set by dhcp before > failing. My preference would probably be having musl default to > ndots=0 rather than ndots=1 so that search has to be enabled > explicitly. Are there any reasons this would be undesirable? Could you explain what this all would mean to someone who has only a general understanding of how DNS works, a home network, and a desire to set up a local DNS server? I have a couple use-cases in mind, which I think involve either the "search" or "domain" keywords in resolv.conf; I'll describe them in case they pertain. 1: the university I attended had a bunch of resources which were available as sub-domains. The way DHCP/DNS/resolv.conf was set up, local sites (for example, http://myweb.csuchico.edu/) were available using only the bare subdomain (in the same example, http://myweb/); I forget whether they used "domain" or "search" for this. This was something I appreciated. 2: On my home network (configured via DHCP, no DNS server yet), I have a network printer that advertises itself using a name in the general format of MFC0000DEADBEEF (where 0000DEADBEEF is the MAC address, stripped of separators). It uses this hostname for DHCP and avahi. However, this name is only available for avahi clients like cups; I want to be able to access it by the same name from non-avahi programs like 'ping', 'links', etcetera, since it's rather annoying to have a 'magic' name that usually works in your printing daemon, but no way to map it to an IP for any troupleshooting/configuration tools. Currently, I've got it set up so the printer has a static lease and I thus know the IP, but I want to use DNS because that's the only way all tools on all computers on the network will automatically know that MFC0000DEADBEEF is (for example) IP 192.168.255.255. It would be rather annoying to have some tools where I can use that name, and some where I need to specify "mfc0000deadbeef.local" instead. Thanks, Isaac Dunham ^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: Re: Would love to see reconsideration for domain and search 2015-10-26 1:26 ` Isaac Dunham @ 2015-10-26 15:35 ` Rich Felker 0 siblings, 0 replies; 44+ messages in thread From: Rich Felker @ 2015-10-26 15:35 UTC (permalink / raw) To: musl On Sun, Oct 25, 2015 at 06:26:43PM -0700, Isaac Dunham wrote: > On Sat, Oct 24, 2015 at 06:02:15PM -0400, Rich Felker wrote: > > The only place adding search support might negatively impact existing > > musl users is by causing hostnames with no dots to be queried with the > > (often useless and unwanted) default domain set by dhcp before > > failing. My preference would probably be having musl default to > > ndots=0 rather than ndots=1 so that search has to be enabled > > explicitly. Are there any reasons this would be undesirable? > > Could you explain what this all would mean to someone who has only a > general understanding of how DNS works, a home network, and a desire > to set up a local DNS server? > > I have a couple use-cases in mind, which I think involve either the > "search" or "domain" keywords in resolv.conf; I'll describe them in case > they pertain. > > 1: the university I attended had a bunch of resources which were available > as sub-domains. The way DHCP/DNS/resolv.conf was set up, local sites > (for example, http://myweb.csuchico.edu/) were available using only the > bare subdomain (in the same example, http://myweb/); I forget whether > they used "domain" or "search" for this. This was something I > appreciated. Yes, this is what adding search domain support would allow. Note that with ndots=1, "myweb" would work, but "myweb.math" would not. > 2: On my home network (configured via DHCP, no DNS server yet), I have a > network printer that advertises itself using a name in the general format > of MFC0000DEADBEEF (where 0000DEADBEEF is the MAC address, stripped of > separators). It uses this hostname for DHCP and avahi. > However, this name is only available for avahi clients like cups; I > want to be able to access it by the same name from non-avahi programs > like 'ping', 'links', etcetera, since it's rather annoying to have a > 'magic' name that usually works in your printing daemon, but no way > to map it to an IP for any troupleshooting/configuration tools. > > Currently, I've got it set up so the printer has a static lease and I > thus know the IP, but I want to use DNS because that's the only way all > tools on all computers on the network will automatically know that > MFC0000DEADBEEF is (for example) IP 192.168.255.255. > It would be rather annoying to have some tools where I can use that name, > and some where I need to specify "mfc0000deadbeef.local" instead. Doing this requires dunning a local nameserver that joins the avahi (mdns, I think) results under some domain you control with the global dns namespace. I suspect dnsmasq or one of the other dns servers made for home network use can do this but I haven't checked. In principle it's even possible with BIND and some scripting to query mdns and use the results to update dynamic dns entries but that's rather ugly. Rich ^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2016-01-29 0:58 UTC | newest] Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-10-23 15:30 Re: Would love to see reconsideration for domain and search Jan Broer 2015-10-23 16:01 ` Tim Hockin 2015-10-23 17:40 ` Jan Broer 2015-10-23 18:02 ` Tim Hockin 2015-10-23 18:03 ` Jan Broer 2015-10-23 18:29 ` Rich Felker 2015-10-23 21:27 ` Jan Broer -- strict thread matches above, loose matches on Subject: below -- 2015-10-22 21:24 Tim Hockin 2015-10-22 21:56 ` Rich Felker 2015-10-22 22:36 ` Tim Hockin 2015-10-22 23:00 ` Josiah Worcester 2015-10-22 23:37 ` Tim Hockin 2015-10-23 4:27 ` Rich Felker 2015-10-23 5:13 ` Tim Hockin 2015-10-23 5:31 ` Rich Felker 2015-10-23 5:37 ` Tim Hockin 2015-10-23 6:00 ` Rich Felker 2015-10-23 6:04 ` Tim Hockin 2016-01-29 0:57 ` Rich Felker 2015-10-27 0:30 ` Rich Felker 2015-10-27 0:37 ` Tim Hockin 2015-10-27 0:45 ` Rich Felker 2015-10-27 8:11 ` u-uy74 2015-11-28 22:48 ` Jan Broer 2015-11-28 23:20 ` Rich Felker 2015-11-29 3:06 ` Jan Broer 2016-01-29 0:58 ` Rich Felker 2015-10-23 8:12 ` u-uy74 2015-10-23 9:35 ` Laurent Bercot 2015-10-23 12:23 ` Laurent Bercot 2015-10-23 15:57 ` Tim Hockin 2015-10-23 5:26 ` Kurt H Maier 2015-10-24 21:33 ` Tim Hockin 2015-10-24 21:57 ` Kurt H Maier 2015-10-24 23:31 ` Rich Felker 2015-10-24 22:02 ` Rich Felker 2015-10-24 22:32 ` Tim Hockin 2015-10-25 8:20 ` u-uy74 2015-10-25 13:06 ` Jan Broer 2015-10-25 13:19 ` u-uy74 2015-10-25 13:39 ` Jan Broer 2015-10-25 14:08 ` u-uy74 2015-10-25 19:08 ` Rich Felker 2015-10-26 1:26 ` Isaac Dunham 2015-10-26 15:35 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).