Re: Agenda for next release?

[Rich Felker <dalias@aerifal.cx>]

On Mon, Dec 31, 2012 at 01:37:39PM -0800, Brad Conroy wrote:
> Not functionally necessary for the release, but nice for future
> compatibility reasons, a few paths are hard-coded into the c files
> rather than defining them in the (non-)"standard" location
> Ex.src/network/getaddrinfo.c has "/etc/hosts", "/etc/resolve.conf"
> and "/etc/services" hard-coded rather than defining them in netdb.h

The following should be read with the understanding that I have been
convinced to change my mind on issues of similar scope in the past.
However, basically the way it is right now is intentional:

Unlike glibc, musl is intended to be useful for static linking, for
making binaries you can just drop onto an arbitrary system and have
them work. As such, I've aimed to eliminate most usages of any
external files in the filesystem; for example, even iconv uses builtin
tables rather than loading character table files. But in some cases
where there's a need for configurability or reading system-specific
information (like the user/group databases or dns servers), some
minimum amount of filesystem policy needs to be encoded into libc (and
thus, with static linking, into the application as well). To my
knowledge, so far all such hard-coded paths correspond to universal
(at least on Linux, and for the most part on ALL historical unices) so
that only intentional gratuitous breakage could break them.

Anyone who gets musl has access to the source, and can of course go
changing all the hard-coded paths to whatever they like. There's
nothing you or I can do to stop that. But there's also no reason to
encourage it or make it easier than it needs to be.

As for paths.h, it's purely there to make certain legacy programs
which expect it to exist easier to build. Many of the macros defined
in it are utter nonsense, or at least not policies I wish to impose on
anybody. Perhaps this should be better-documented.

Speaking of documentation, the actual documentation for musl, which I
hope to have ready as part of the 1.0 release, will document all
assumptions it makes about the layout of the filesystem, including
which functionality depends on the existence of certain
files/devices/mountpoints in the filesystem (for example, shm_open and
sem_open depending on /dev/shm, or fexecve depending on /proc, or
system and popen depending on /bin/sh).

> also android has etc in /system, so the actual path
> is /system/etc/hosts (though /etc is typically a symlink to
> /system/etc), but other strange layouts exist and it would be a lot
> easier/cleaner for work arounds to have the ifdefs confined to one
> location in the header files and use those definitions in the .c
> files so that port patches only hit a few headers without polluting
> the .c files with a lot of ifdef nests

The problem with making your binaries for alternate layouts like this
is that then they won't work on any system that lacks the alternate
layout. Now, let N people come up with their own N brilliant ideas of
how the FS layout should be, and every system needs N different
symlinks and fake directories all over the place in order to be able
to run arbitrary binaries... This approach does not scale.

On the other hand, if you want to use an alternate layout locally with
symlinks in the standard locations, then your nonstandard layout just
needs 1 set of symlinks (not N sets), and systems with standard
layouts don't need any hackery at all.

> I ran into this while building a static wget-like downloader that
> would use an alternate hosts file to complement a custom local
> server that locally hosts some problematic CDNs (to work around
> problems such as: waiting for ajax.googleapis.com,
> platform.twitter.com etc...) but this is an obvious hack, not the
> typical use case for a standard library... but then again part of
> the charm of musl is its "hackability"

Using a custom hosts file or resolv.conf file via environment vars is
actually an interesting idea we could explore at some point, but it
would have to be secured so as not to affect suid programs or programs
with other sorts of elevated privileges.

Rich