[musl] musl libc seems to use case-sensitive comparison for hostnames in /etc/hosts

[musl] musl libc seems to use case-sensitive comparison for hostnames in /etc/hosts

[Denis Ovsienko]

Hello all.

$ /usr/lib/libc.so 
musl libc (aarch64)
Version 1.2.5
Dynamic Program Loader
Usage: /usr/lib/libc.so [options] [--] pathname [args]

As far as I can tell, musl libc has a bug in Internet hostnames
processing, in that it uses case-sensitive string comparison on
hostnames that it reads from /etc/hosts.  This does not apply to
hostnames it resolves using DNS.

The problem seems to be somewhere around name_from_hosts() in
src/network/lookup_name.c.  In any case, no musl code under
src/network/ uses strcasecmp(), which hints at a likely root cause: by
design using strcmp() to compare Internet hostnames will work correctly
for a fraction of hostnames only.

One way to reproduce the problem is as follows:

$ grep -i noeth-ipv4-noipv6.host123.libpcap.test /etc/hosts
10.20.30.40 noeth-ipv4-noipv6.host123.libpcap.test

$ ping -n -c1 noeth-ipv4-noipv6.host123.libpcap.test
PING noeth-ipv4-noipv6.host123.libpcap.test (10.20.30.40): 56 data bytes

--- noeth-ipv4-noipv6.host123.libpcap.test ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

$ ping -n -c1 NOETH-IPV4-NOIPV6.HOST123.LIBPCAP.TEST
ping: bad address 'NOETH-IPV4-NOIPV6.HOST123.LIBPCAP.TEST'

$ ping -n -c1 tcpdump.org
PING tcpdump.org (195.22.157.118): 56 data bytes
64 bytes from 195.22.157.118: seq=0 ttl=42 time=11.420 ms

--- tcpdump.org ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 11.420/11.420/11.420 ms

$ ping -n -c1 TCPDUMP.ORG
PING TCPDUMP.ORG (195.22.157.118): 56 data bytes
64 bytes from 195.22.157.118: seq=0 ttl=42 time=11.306 ms

--- TCPDUMP.ORG ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 11.306/11.306/11.306 ms


(The ping above is actually busybox because it is Alpine Linux.)  This
reproduces exactly the same using libpcap, which uses getaddrinfo().
However, curl and ssh manage to resolve the upper case version of the
same hostname.  In the case of ssh it seems it lower-cases the hostname
before the lookup, because placing an upper-cased hostname into
/etc/hosts prevents ssh from resolving it:

$ grep -i somehost /etc/hosts
10.20.30.40 SOMEHOST

$ ssh somehost
ssh: Could not resolve hostname somehost: Name does not resolve

$ ssh SOMEHOST
ssh: Could not resolve hostname somehost: Name does not resolve

That said, curl seems more robust and manages to resolve the hostname
whichever the letter case on the command line and in /etc/hosts, so it
seems to be using a different resolver library.

Please review and confirm.

-- 
    Denis Ovsienko

Re: [musl] musl libc seems to use case-sensitive comparison for hostnames in /etc/hosts

[Rich Felker]

On Wed, Feb 19, 2025 at 08:56:17PM +0000, Denis Ovsienko wrote:
> Hello all.
> 
> $ /usr/lib/libc.so 
> musl libc (aarch64)
> Version 1.2.5
> Dynamic Program Loader
> Usage: /usr/lib/libc.so [options] [--] pathname [args]
> 
> As far as I can tell, musl libc has a bug in Internet hostnames
> processing, in that it uses case-sensitive string comparison on
> hostnames that it reads from /etc/hosts.  This does not apply to
> hostnames it resolves using DNS.
> 
> The problem seems to be somewhere around name_from_hosts() in
> src/network/lookup_name.c.  In any case, no musl code under
> src/network/ uses strcasecmp(), which hints at a likely root cause: by
> design using strcmp() to compare Internet hostnames will work correctly
> for a fraction of hostnames only.

Yes, I think this was vaguely known but ignored, probably on account
of hosts file mainly being used with manual inputs from the user in
lowercase -- not that this is an excuse for it, just likely why it
wasn't prioritized to fix.

I recall something about it coming up in the context of adding IDN,
which also sheds light on what the fix should be: not strcasecmp (a
nasty function that's underspecified and only matches what's needed
here because we took advantage of behavior outside the C locale being
unspecified) but normalizing both the input and the strings from the
hosts file so that we can strcmp.

Rich

Re: [musl] musl libc seems to use case-sensitive comparison for hostnames in /etc/hosts

[Thorsten Glaser]

On Thu, 20 Feb 2025, Rich Felker wrote:

>I recall something about it coming up in the context of adding IDN,

You don’t need IDN for /etc/hosts or calls to the resolver, they
all use the xn--* punycode form.

>which also sheds light on what the fix should be: not strcasecmp (a

Definitely not, yeah…

bye,
//mirabilos
-- 
(gnutls can also be used, but if you are compiling lynx for your own use,
there is no reason to consider using that package)
	-- Thomas E. Dickey on the Lynx mailing list, about OpenSSL

Re: [musl] musl libc seems to use case-sensitive comparison for hostnames in /etc/hosts

[Rich Felker]

On Fri, Feb 21, 2025 at 02:52:04AM +0100, Thorsten Glaser wrote:
> On Thu, 20 Feb 2025, Rich Felker wrote:
> 
> >I recall something about it coming up in the context of adding IDN,
> 
> You don’t need IDN for /etc/hosts or calls to the resolver, they
> all use the xn--* punycode form.

What IDN support means is that the resolver accepts the actual names,
not just the punycode encoding of them. So that, like, you can type
ping 納豆.w3.mag.keio.ac.jp and have it work without ping having to
link some 3p library.

The WIP proposal for implementing it canonicalizes both the input and
the entries in the hosts file to punycode before comparison so that it
works consistently regardless of whether you have unicode literals or
punycode in your hosts file.

Rich