From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 22747 invoked from network); 13 Mar 2023 16:36:15 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 13 Mar 2023 16:36:15 -0000 Received: (qmail 7812 invoked by uid 550); 13 Mar 2023 16:36:12 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 7774 invoked from network); 13 Mar 2023 16:36:11 -0000 Date: Mon, 13 Mar 2023 12:35:58 -0400 From: Rich Felker To: Briner =?utf-8?B?Q8OpZHJpYyAoREkp?= Cc: "musl@lists.openwall.com" Message-ID: <20230313163558.GP4163@brightrain.aerifal.cx> References: <71e40202617b4912a98671fa73d30df4@etat.ge.ch> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <71e40202617b4912a98671fa73d30df4@etat.ge.ch> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] resolv.conf and ndots:5 On Mon, Mar 13, 2023 at 03:30:44PM +0000, Briner Cédric (DI) wrote: > Hi, > > We do run kubernetes. And some of the applications are shipped with > the musl libc library brought by alpine images. We have discovered a > bad behaviour on how the system "nss" is interpreting our > /etc/resolv.conf I followed up as you were leaving IRC: cedric-briner, ok let me go thru what seems to be happening... because of ndots:5, ge.ch is first looked up as ge.ch.ceti.etat-ge.ch, yes? i just performed those queries and it should work fine, provided your nameserver is giving truthful answers if you're in a kubernetes environment, maybe there's some (coredns or otherwise?) nameserver in between that's messing things up and replying NODATA in place of NxDomain i.e. saying "ge.ch.ceti.etat-ge.ch exists but doesn't have an address" instead of "ge.ch.ceti.etat-ge.ch does not exist" that would lead to the behavior you're seeing And indeed based on your strace that's what's happening: > cat /etc/resolv.conf > # search app-5580-capitastra-rec-01.svc.cluster.local svc.cluster.local cluster.local ceti.etat-ge.ch > # nameserver 10.177.0.10 > # options ndots:5 > > For checking the behaviour of libc, we have used the command "getent hosts".. Below are the command that we have tested and the result o fit > > getent hosts ge.ch > # > > getent hosts ge.ch. > # 160.53.144.68 ge.ch ge.ch ge.ch. > > As you can see, the fully qualified ge.ch. (with the leading dot) is resolving. And the ge.ch (without the leading dot) is not resolving. > > We have done some test for the following domain to better test and you can see on the right if is succeed or not > - ge.ch. sucesss > - ge.ch failed > - etat-ge.ch. success > - etat-ge.ch failed > - ge.fr success > - tsr.ch success > > We have looked at the calls with the help of strace for this different command. The ones that failed has such pattern on the strace > - try to solve ge.ch. app-5580-capitastra-rec-01.svc.cluster.local > - try to solve ge.ch.svc.cluster.local > - try to solve ge.ch.cluster.local > - try to solve ge.ch.ceti.etat-ge.ch > - Do not try to solve ge.ch > > And the one that works well have the same trace but try to solve the last and they solve it. > - try to solve ge.fr. app-5580-capitastra-rec-01.svc.cluster.local > - try to solve ge.fr.svc.cluster.local > - try to solve ge.fr.cluster.local > - try to solve ge.fr.ceti.etat-ge.ch > - try to solve ge.fr -> success > > A bit puzzled by this, we imagined that the problem came from the > libc and the fact that the last search domain "ceti.etat-ge.ch" does > finish as the domain (ge.ch) we want to resolve. > > That seems a bit strange.. but.. we saw the same when trying to > solve "etat-ge.ch" > > Furthermore, we have discovered that "getent hosts ge.ch" is working > well if we change the line "options ndots:1" in the resolv.conf > > Below the signature are the capture made with strace. The firsts are > the sparse one with the element you should look at, the second one > (full) shows all the call > > Regards, > Cédric BRINER > > > # Strace capture > ## sparse and commented > ### ge.ch > > As said, the ge.ch doesn't resolve, and this is shown in the strace of `getent hosts ge.ch` > > ``` > .. > ge.ch.app-5580-capitastra-rec-01.svc.cluster.local > [pid 1228780] recvfrom(3, "\273\226\205\3\0\1\0\0\0\1\0\1\2ge\2ch\32app-5580-capi"..., 512, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.177.0.10")}, [16]) = 172 > .. > ge.ch.app-5580-capitastra-rec-01.svc.cluster.local > [pid 1228780] recvfrom(3, "\377\16\205\3\0\1\0\0\0\1\0\1\2ge\2ch\3svc\7cluster\5l"..., 512, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.177.0.10")}, [16]) = 145 > .. > ge.ch.app-5580-capitastra-rec-01.svc.cluster.local > [pid 1228780] recvfrom(3, "\310\v\205\3\0\1\0\0\0\1\0\1\2ge\2ch\7cluster\5local"..., 512, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.177.0.10")}, [16]) = 141 > .. > ge.ch.app-5580-capitastra-rec-01.svc.cluster.local > [pid 1228780] sendto(3, "\2567\1\0\0\1\0\0\0\0\0\0\2ge\2ch\4ceti\7etat-ge\2"..., 39, MSG_NOSIGNAL, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.177.0.10")}, 16) = 39 > .. > ge.ch.app-5580-capitastra-rec-01.svc.cluster.local > [pid 1228780] recvfrom(3, "\2567\205\200\0\1\0\0\0\1\0\1\2ge\2ch\4ceti\7etat-ge\2"..., 512, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.177.0.10")}, [16]) = 140 ^^^^^^^^ The low 4 bits of the underlined part of the response packet form RCODE=0, success. This is 10.177.0.10 saying the queried name does exist and just does not have an A record. (It might have AAAA records, MX records, whatever -- we can't know.) Since normal recursive nameservers on the public internet correctly report RCODE=3 (NxDomain) for ge.ch.ceti.etat-ge.ch, something in your k8s cluster must be rewriting the answer to give an incorrect result. glibc (like most traditional stub resolvers) handles this case sloppily and just treats NODATA and NxDomain the same, continuing search. This makes the results potentially unstable depending on whether the caller requested both v4 and v6 results or just one or the other, and semantically mismatches what's in DNS. musl handles this very intentionally with the aim of delivering only consistent results. Rich