From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI, NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,T_SCC_BODY_TEXT_LINE,WEIRD_PORT autolearn=ham autolearn_force=no version=3.4.4 Received: from second.openwall.net (second.openwall.net [193.110.157.125]) by inbox.vuxu.org (Postfix) with SMTP id C2FBA21CD3 for ; Thu, 7 Mar 2024 01:18:09 +0100 (CET) Received: (qmail 10199 invoked by uid 550); 7 Mar 2024 00:14:10 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 10166 invoked from network); 7 Mar 2024 00:14:10 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709770676; x=1710375476; darn=lists.openwall.com; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=B3iruyiYeVFS/ZHirDgsFlsgHFl9oLouctLv8ouvXvY=; b=QNr8pi+I+KMDmcT7r0MKBo8qxDaiyrrM9mnY5lv7/Zhl+pabb3ZR1fRAjOGshkZnFr 3bROS/NXVsn8mw9QmO9W3Mn0V/vsGM1iR5TZGwR/gwezKpo5z/kzV3G+CW1ltzPpdb5y EmsY5/4rKROY/msIzj9v0c8NrVeNT27uUdC7tSbaYn3AjIxxHbc+HkuZfzxF8bMhZ1Zn 24isVnjvlWIb41TDNts/oGTcIFuqU4F9nzvhTv+qH1cCEsZQzx8jxQor4RwnhfoUrrSU /Vl7GvFRX4JdRqeUSpZbDY5y9oa3IqQlDpsn/PxiZA+yYPfs4JHVIzKJql51uwquo5UT cUaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709770676; x=1710375476; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=B3iruyiYeVFS/ZHirDgsFlsgHFl9oLouctLv8ouvXvY=; b=eb1o3u3I+a4TklR7zkPabWf2CHPZPadSA6du0SSfka23jP0GQBgFdk2T6F1rI+quR/ CIiJoOAvwlcqlnQaQFmQvNnVeud/aBi8aiSAtjB+khcAOgJQAaLfVhxTK5SYRq498tFt n1HhoRmYH2euF9XJooFcPA9gpM2vLIYL4rshnzyK5FKFvq0MBfioQE4ALVP0swTjZx4I gDvuyQ0L1AYr6MqNPT52Dj7xL+AypMqE/es/scMOS/DDewm5WBTGmehWng9dq0vtkafn gVAfoP+DzjrAza0KoVxktCdqEsYnHwua9iaVXjowRa1kZBliMdgKXroVhMSk89XFyqMQ KIEw== X-Gm-Message-State: AOJu0YwN3h50PjG+ODVsKj6wVQk2W4fztWYfU/olUKxuNJZma7IZuneE hNKSYzLRo0NvHBb/VqJ16a3qYlwAlpDN5xGphk73K4L+fPQyCWsi7OuuoXCiFOgqcohxjXywWdO Ajmyg83pqCJYJ8S6PLTYiMuZIN42R9+OBY84= X-Google-Smtp-Source: AGHT+IEM47ZOi4gLIdle0+wPfbn2bh47YFyow9cZCBzbDXBSz1fWC3SgLnQXrLCSHYdQZk85WouB1zHBhS776xBkUlA= X-Received: by 2002:a17:906:3c4f:b0:a3f:c4f9:eb16 with SMTP id i15-20020a1709063c4f00b00a3fc4f9eb16mr11452287ejg.20.1709770675747; Wed, 06 Mar 2024 16:17:55 -0800 (PST) MIME-Version: 1.0 References: <20240306161544.GH4163@brightrain.aerifal.cx> In-Reply-To: <20240306161544.GH4163@brightrain.aerifal.cx> From: David Schinazi Date: Wed, 6 Mar 2024 16:17:44 -0800 Message-ID: To: Rich Felker Cc: musl@lists.openwall.com Content-Type: multipart/alternative; boundary="000000000000c68197061307017d" Subject: Re: [musl] mDNS in musl --000000000000c68197061307017d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the detailed responses, everyone! I'm sending replies inline. David On Wed, Mar 6, 2024 at 8:15=E2=80=AFAM Rich Felker wrote: > On Tue, Mar 05, 2024 at 11:29:03PM -0800, David Schinazi wrote: > > Hi everyone, > > > > I was debugging a network connectivity issue on Alpine and have tracked > it > > down to lack of support for mDNS in musl gethostbyname / getaddrinfo > [1]. I > > looked through the musl codebase to understand why, and it would be > pretty > > straightforward to fix. I'd be interested in writing a patch for this, > so I > > was wondering: would you be at all interested in potentially taking suc= h > a > > patch? > > > > Some more info on mDNS: all names that end in ".local" are reserved for > use > > by mDNS, and instead of sending them to the DNS resolver, they're sent > > locally over multicast - and the machine with that name replies with it= s > IP > > address. It's used today to discover printers and pretty much everythin= g > in > > home networks. > > Last I checked, .local is not actually reserved by any relevant > specification/authority. It was basically just appropriated by mDNS. > The protocol spoken is also not exactly DNS (for example, it uses raw > UTF-8 rather than IDN/punycode, which would need to be special-cased > once we support the latter). > On Wed, Mar 6, 2024 at 8:45=E2=80=AFAM Jeffrey Walton = wrote: > It looks like IANA reserves it, and cites RFC 6762, > < > https://www.iana.org/assignments/special-use-domain-names/special-use-dom= ain-names.xhtml > >. As Jeffrey points out, when the IETF decided to standardize mDNS, they published it (RFC 6762) at the same time as the Special-Use Domain Registry (RFC 6761) which created a process for reserving domain names for custom purposes, and ".local" was one of the initial entries into that registry. The UTF-8 vs punycode issue when it comes to mDNS and DNS is somewhat of a mess. It was discussed in Section 16 of RFC 6762 but at the end of the day punycode won. Even Apple's implementation of getaddrinfo will perform punycode conversion for .local instead of sending the UTF-8. So in practice you wouldn't need to special-case anything here. There's also very much a policy matter of what "locally over > multicast" means (what the user wants it to mean). Which interfaces > should be queried? Wired and wireless ethernet? VPN links or other > sorts of tunnels? Just one local interface (which one to prioritize) > or all of them? Only if the network is "trusted"? Etc. > You're absolutely right. Most mDNS systems try all non-loopback non-p2p multicast-supporting interfaces, but sending to the default route interface would be a good start, more on that below. My view has always been that the right way to do something like this, > where there's no existing interface or contract/expectations for how > the libc stub resolver does it, is that it belongs in a resolver > speaking dns protocol on localhost. That way policy isn't baked-in to > individual executables (which may be static linked) but kept in a > place that's reasonable to have policy controls and where the user can > customize them. > I agree that providing an option to have these policy decisions in user-space makes a lot of sense. That's what glibc and Apple's libsystem do, but that comes at a higher indirection cost. For components that don't have as much flexibility though, it would be nice to be able to send these queries without requiring additional software. > From looking through musl, both gethostbyname() and getaddrinfo() route > > through __lookup_name(), which eventually calls name_from_dns(). From > > looking at that function, the issue is that it doesn't treat .local > > specifically - instead of sending those queries to multicast, it sends > them > > to the regularly configured DNS nameservers. > > > > The fix would be to modify name_from_dns() [2] such that if `name` ends > in > > ".local", then pass in a different conf variable to __res_msend_rc(). T= he > > conf variable contains (amongst other things) the DNS nameservers to se= nd > > the query to. So, when the name ends in .local, instead of passing in t= he > > regular nameservers, we pass the multicast addresses and ports dedicate= d > to > > mDNS (224.0.0.251:5353 and [ff02::fb]:5353). > > When you do that, how do you control which interface(s) it goes over? > I think that's an important missing ingredient. > You're absolutely right. In IPv4, sending to a link-local multicast address like this will send it over the IPv4 default route interface. In IPv6, the interface needs to be specified in the scope_id. So we'd need to pull that out of the kernel with rtnetlink. > And that's it! This implementation is compatible with the "One-Shot > > Multicast DNS Queries" mode of the mDNS RFC [3]. (Other versions of lib= c > > have a mode to send the query over dbus to avahi so that it can cache > mDNS > > results locally. But that's the more complicated "Continuous Multicast > DNS > > Querying" mode of the RFC, and we don't need that here.) > > > > So what do you think, would you be interested in support for mDNS? (In > case > > it matters, I've made changes in getaddrinfo inside Apple's libc, so I'= m > > comfortable in this kind of code even though I have zero prior experien= ce > > with musl) > > If at some point there's a consensus on stub resolvers having an > expectation to support this themselves, and on untanging the details > like the above, and on "ownership" of the ".local" TLD, it might make > sense to have a resolv.conf option to do this. So there's at least IETF consensus on these things. The ownership is well-defined in RFC 6761, and the support by stub resolvers is discussed in RFC 6762 Section 22.1 paragraph 3 <> > Unlike general unioning > of sources, which is really problematic, the mDNS stuff seems to be > putting the decision which source to use *before* making any queries, > which is a lot less problematic. > I'm not familiar with what you mean by unioning here, are you referring to interface selection, DNS name server selection, or something else? On Wed, Mar 6, 2024 at 8:16=E2=80=AFAM Markus Wichmann w= rote: > So is there something wrong with the solution presented in the wiki > page? Because that is generally the answer we recommend: If you want any > name resolution other than DNS, write a proxy that does what you want > and point resolv.conf to it. Similarly, if you want any user database > lookup other than local files, write an nscd proxy that does what you > want. > That's certainly an option. Ideally I'd rather avoid adding additional processes that can be failure points, when the stub can send these itself with a very small modification. > Reason for that is that that is the most generic way to support any > other name service besides DNS. It avoids the dependency on dynamic > loading that something like glibc's nsswitch would create, and would > avoid having multiple backends in libc. I really don't think anyone > wants to open that particular door. Once mDNS is in there, someone will > add NetBIOS, just you wait. I'm definitely supportive of the slippery slope argument, but I think there's still a real line between mDNS and NetBIOS. mDNS uses a different transport but lives inside the DNS namespace, whereas NetBIOS is really its own thing - NetBIOS names aren't valid DNS hostnames. Let me know what you think of the above. If you think of mDNS as its own beast then I can see how including it wouldn't really make sense. But if you see it as an actual part of the DNS, then it might be worth a small code change :-) Cheers, David --000000000000c68197061307017d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for the detailed responses, everyo= ne!
I'm sending replies inline.
David

On Wed, Mar 6,= 2024 at 8:15=E2=80=AFAM Rich Felker <dalias@libc.org> wrote:
On Tue, Mar 05, 2024 at 11:29:03PM -0800, David Schinazi wr= ote:
> Hi everyone,
>
> I was debugging a network connectivity issue on Alpine and have tracke= d it
> down to lack of support for mDNS in musl gethostbyname / getaddrinfo [= 1]. I
> looked through the musl codebase to understand why, and it would be pr= etty
> straightforward to fix. I'd be interested in writing a patch for t= his, so I
> was wondering: would you be at all interested in potentially taking su= ch a
> patch?
>
> Some more info on mDNS: all names that end in ".local" are r= eserved for use
> by mDNS, and instead of sending them to the DNS resolver, they're = sent
> locally over multicast - and the machine with that name replies with i= ts IP
> address. It's used today to discover printers and pretty much ever= ything in
> home networks.

Last I checked, .local is not actually reserved by any relevant
specification/authority. It was basically just appropriated by mDNS.
The protocol spoken is also not exactly DNS (for example, it uses raw
UTF-8 rather than IDN/punycode, which would need to be special-cased
once we support the latter).

On Wed, Mar 6, 2024 at 8:45=E2=80=AFAM Jeffre= y Walton <noloader@gmail.com&g= t; wrote:
It looks l= ike IANA reserves it, and cites RFC 6762,
<https://www.iana.org/assignments/speci= al-use-domain-names/special-use-domain-names.xhtml>.

As Jeffrey points out, when the IETF decided to stan= dardize mDNS, they published=C2=A0it (RFC 6762) at the same time as the Spe= cial-Use Domain Registry (RFC 6761) which created a process for reserving d= omain names for custom purposes, and ".local" was one of the init= ial entries into that registry. The UTF-8 vs punycode issue when it comes t= o mDNS and DNS is somewhat of a mess. It was discussed in Section 16 of RFC= 6762 but at the end of the day punycode won. Even Apple's implementati= on of getaddrinfo will perform punycode conversion for .local instead of se= nding the UTF-8. So in practice you wouldn't need to special-case anyth= ing here.

There's also very much a policy matter of what "locally over
multicast" means (what the user wants it to mean). Which interfaces should be queried? Wired and wireless ethernet? VPN links or other
sorts of tunnels? Just one local interface (which one to prioritize)
or all of them? Only if the network is "trusted"? Etc.
=C2=A0
You're absolutely right. Most mDNS systems = try all non-loopback non-p2p multicast-supporting interfaces, but sending t= o the default route interface would be a good start, more on that below.

My view has always been that the right way to do something like this,
where there's no existing interface or contract/expectations for how the libc stub resolver does it, is that it belongs in a resolver
speaking dns protocol on localhost. That way policy isn't baked-in to individual executables (which may be static linked) but kept in a
place that's reasonable to have policy controls and where the user can<= br> customize them.

I agree that providing = an option to have these policy decisions in user-space makes a lot of sense= . That's what glibc and Apple's libsystem do, but that comes at a h= igher indirection cost. For components that don't have as much flexibil= ity though, it would be nice to be able to send these queries without requi= ring additional software.

> From looking through musl, both gethostbyname() and getaddrinfo() rout= e
> through __lookup_name(), which eventually calls name_from_dns(). From<= br> > looking at that function, the issue is that it doesn't treat .loca= l
> specifically - instead of sending those queries to multicast, it sends= them
> to the regularly configured DNS nameservers.
>
> The fix would be to modify name_from_dns() [2] such that if `name` end= s in
> ".local", then pass in a different conf variable to __res_ms= end_rc(). The
> conf variable contains (amongst other things) the DNS nameservers to s= end
> the query to. So, when the name ends in .local, instead of passing in = the
> regular nameservers, we pass the multicast addresses and ports dedicat= ed to
> mDNS (224.0.0.251:5353 and [ff02::fb]:5353).

When you do that, how do you control which interface(s) it goes over?
I think that's an important missing ingredient.
You're absolutely right. In IPv4, sending to a link-local = multicast address like this will send it over the IPv4 default route interf= ace. In IPv6, the interface needs to be specified in the scope_id. So we= 9;d need to pull that out of the kernel with rtnetlink.

> And that's it! This implementation is compatible with the "On= e-Shot
> Multicast DNS Queries" mode of the mDNS RFC [3]. (Other versions = of libc
> have a mode to send the query over dbus to avahi so that it can cache = mDNS
> results locally. But that's the more complicated "Continuous = Multicast DNS
> Querying" mode of the RFC, and we don't need that here.)
>
> So what do you think, would you be interested in support for mDNS? (In= case
> it matters, I've made changes in getaddrinfo inside Apple's li= bc, so I'm
> comfortable in this kind of code even though I have zero prior experie= nce
> with musl)

If at some point there's a consensus on stub resolvers having an
expectation to support this themselves, and on untanging the details
like the above, and on "ownership" of the ".local" TLD,= it might make
sense to have a resolv.conf option to do this.

<= div>So there's at least IETF consensus on these things. The ownership i= s well-defined in RFC 6761, and the support by stub resolvers is discussed = in RFC 6762 Section 22.1 paragraph 3 <<Name resolution APIs and libra= ries SHOULD recognize these names as special and SHOULD NOT send queries fo= r these names to their=C2=A0configured (unicast) caching DNS server(s).>= >
=C2=A0
Unlike general unioning
of sources, which is really problematic, the mDNS stuff seems to be
putting the decision which source to use *before* making any queries,
which is a lot less problematic.

I'= m not familiar with what you mean by unioning here, are you referring=C2=A0= to interface selection, DNS name server selection, or something else?
=

On Wed, Mar 6, 20= 24 at 8:16=E2=80=AFAM Markus Wichmann <nullplan@gmx.net> wrote:
So is there something wrong with the solution presented in t= he wiki
page? Because that is generally the answer we recommend: If you = want any
name resolution other than DNS, write a proxy that does what yo= u want
and point resolv.conf to it. Similarly, if you want any user data= base
lookup other than local files, write an nscd proxy that does what y= ou
want.

That's certainly an opt= ion. Ideally I'd rather avoid adding additional processes that can be f= ailure points, when the stub can send these itself with a very small modifi= cation.
=C2=A0
Reason for that is that that is the most generic way to support anyother name service besides DNS. It avoids the dependency on dynamic
loa= ding that something like glibc's nsswitch would create, and would
av= oid having multiple backends in libc. I really don't think anyone
wa= nts to open that particular door. Once mDNS is in there, someone will
ad= d NetBIOS, just you wait.

I'm definitel= y supportive of the slippery slope argument, but I think there's still = a real line between mDNS and NetBIOS. mDNS uses a different transport but l= ives inside the DNS namespace, whereas NetBIOS is really its own thing - Ne= tBIOS names aren't valid DNS hostnames.

= Let me know what you think of the above. If you think of mDNS as its own be= ast then I can see how including it wouldn't really make sense. But if = you see it as an actual part of the DNS, then it might be worth a small cod= e change :-)

Cheers,
David
--000000000000c68197061307017d--