musl vs. Debian policy

musl vs. Debian policy

[Isaac Dunham]

I started writing a short explanation of the musl installation for packagers, and realized that there's one area that's inconvenient:
$syslib/ld-musl-*.so.1 is a symlink to libc.so.

Debian policy requires that any public libraries have a version number. 
Specifically, Debian Policy 8.2 
(http://www.debian.org/doc/debian-policy/ch-sharedlibs.html): 
If your package contains files whose names do not change with each change in the library shared object version, you must not put them in the shared library package. Otherwise, several versions of the shared library cannot be installed at the same time without filename clashes, making upgrades and transitions unnecessarily difficult.

The apparent solution to this is to ship only the dynamic linker, since this is all we need (the dependency on libc.so is disregarded when it comes to running dynamically linked programs).  But currently, actually doing this would be somewhat of a hack.

Is there any prospect of installing lib/libc.so straight to ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:

install-ldso: $(DESTDIR)$(LDSO_PATHNAME)

$(DESTDIR)$(LDSO_PATHNAME): lib/libc.so
        install -D -m 755 $< $@

I realize it would also be necessary to adjust the rules for installing libc.so, however. If the change is welcome, I could prepare a patch.

Thanks, 
Isaac Dunham <idunham@lavabit.com>

Re: musl vs. Debian policy

[Rich Felker]

On Wed, Mar 06, 2013 at 03:29:13PM -0800, Isaac Dunham wrote:
> I started writing a short explanation of the musl installation for packagers, and realized that there's one area that's inconvenient:
> $syslib/ld-musl-*.so.1 is a symlink to libc.so.
> 
> Debian policy requires that any public libraries have a version number. 
> Specifically, Debian Policy 8.2 
> (http://www.debian.org/doc/debian-policy/ch-sharedlibs.html): 
> If your package contains files whose names do not change with each
> change in the library shared object version, you must not put them
> in the shared library package. Otherwise, several versions of the
> shared library cannot be installed at the same time without filename
> clashes, making upgrades and transitions unnecessarily difficult.

Are you sure this applies? I don't think the libc.so in musl counts as
a "public library" as nothing references it by name. Moreover, if
installing as a package on a glibc-based distro, it should not be
installed in /usr/lib (where it would clash with glibc's libc.so
linker script) but rather its own path.

> The apparent solution to this is to ship only the dynamic linker,
> since this is all we need (the dependency on libc.so is disregarded
> when it comes to running dynamically linked programs). But
> currently, actually doing this would be somewhat of a hack.
> 
> Is there any prospect of installing lib/libc.so straight to
> ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:

This has been proposed before, and the main obstacle was build-system
difficulties if I remember right. I'd still like to consider doing it,
but it would be nice to be able to do it for its own sake rather than
for the sake of satisfying distro policy being applied where it
doesn't make sense. Maybe we can try to figure out Debian's stance
before we rush into making the change for their sake.

Rich

Re: musl vs. Debian policy

[Christian Neukirchen]

Rich Felker <dalias@aerifal.cx> writes:

> On Wed, Mar 06, 2013 at 03:29:13PM -0800, Isaac Dunham wrote:
>> The apparent solution to this is to ship only the dynamic linker,
>> since this is all we need (the dependency on libc.so is disregarded
>> when it comes to running dynamically linked programs). But
>> currently, actually doing this would be somewhat of a hack.
>> 
>> Is there any prospect of installing lib/libc.so straight to
>> ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
>
> This has been proposed before, and the main obstacle was build-system
> difficulties if I remember right. I'd still like to consider doing it,
> but it would be nice to be able to do it for its own sake rather than
> for the sake of satisfying distro policy being applied where it
> doesn't make sense. Maybe we can try to figure out Debian's stance
> before we rush into making the change for their sake.

In this case, could we also change the SONAME of the library itself to
something not libc.so?  It would avoid this "bogus" warning of glibc
ldconfig...

ldconfig: /usr/lib/libc.so is not a symbolic link

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: musl vs. Debian policy

[Kurt H Maier]

I understand the need to play nice with everyone, but the idea of
changing musl's build scripts to satisfy glibc-based desires strikes me
as incongruous with the idea of an independent libc project.  Debian
packagers should be able to make whatever changes they need in their
packaging scripts.  I'm quite certain it's possible.

khm

Re: musl vs. Debian policy

[Daniel Cegiełka]

@ Kurt H Maier

totally agree

Re: musl vs. Debian policy

[Rich Felker]

On Thu, Mar 07, 2013 at 07:17:56PM +0100, Christian Neukirchen wrote:
> Rich Felker <dalias@aerifal.cx> writes:
> 
> > On Wed, Mar 06, 2013 at 03:29:13PM -0800, Isaac Dunham wrote:
> >> The apparent solution to this is to ship only the dynamic linker,
> >> since this is all we need (the dependency on libc.so is disregarded
> >> when it comes to running dynamically linked programs). But
> >> currently, actually doing this would be somewhat of a hack.
> >> 
> >> Is there any prospect of installing lib/libc.so straight to
> >> ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
> >
> > This has been proposed before, and the main obstacle was build-system
> > difficulties if I remember right. I'd still like to consider doing it,
> > but it would be nice to be able to do it for its own sake rather than
> > for the sake of satisfying distro policy being applied where it
> > doesn't make sense. Maybe we can try to figure out Debian's stance
> > before we rush into making the change for their sake.
> 
> In this case, could we also change the SONAME of the library itself to
> something not libc.so?  It would avoid this "bogus" warning of glibc
> ldconfig...

No, this is a lot more problematic and I see no benefits. For each
possible SONAME musl may have been linked by, musl must contain a
special-case to refuse to load this SONAME when it appears in
DT_NEEDED. "libc.so" is a name that should never appear elsewhere. I
don't want to keep expanding this list of names, and of course
programs linked using a new SONAME would be gratuitously incompatible
with an older musl ld.so that didn't have the new name included in its
refuse-to-load list.

> ldconfig: /usr/lib/libc.so is not a symbolic link

IIRC this is happening due to some other misconfiguration. If nothing
else, it means glibc and musl were both installed in /usr/lib, or
ldconfig is configured for the wrong paths (since ldconfig has nothing
to do with musl).

Rich

Re: musl vs. Debian policy

[Christian Neukirchen]

Rich Felker <dalias@aerifal.cx> writes:

> On Thu, Mar 07, 2013 at 07:17:56PM +0100, Christian Neukirchen wrote:
>> Rich Felker <dalias@aerifal.cx> writes:
>> 
>> > On Wed, Mar 06, 2013 at 03:29:13PM -0800, Isaac Dunham wrote:
>> >> The apparent solution to this is to ship only the dynamic linker,
>> >> since this is all we need (the dependency on libc.so is disregarded
>> >> when it comes to running dynamically linked programs). But
>> >> currently, actually doing this would be somewhat of a hack.
>> >> 
>> >> Is there any prospect of installing lib/libc.so straight to
>> >> ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
>> >
>> > This has been proposed before, and the main obstacle was build-system
>> > difficulties if I remember right. I'd still like to consider doing it,
>> > but it would be nice to be able to do it for its own sake rather than
>> > for the sake of satisfying distro policy being applied where it
>> > doesn't make sense. Maybe we can try to figure out Debian's stance
>> > before we rush into making the change for their sake.
>> 
>> In this case, could we also change the SONAME of the library itself to
>> something not libc.so?  It would avoid this "bogus" warning of glibc
>> ldconfig...
>
> No, this is a lot more problematic and I see no benefits. For each
> possible SONAME musl may have been linked by, musl must contain a
> special-case to refuse to load this SONAME when it appears in
> DT_NEEDED. "libc.so" is a name that should never appear elsewhere. I
> don't want to keep expanding this list of names, and of course
> programs linked using a new SONAME would be gratuitously incompatible
> with an older musl ld.so that didn't have the new name included in its
> refuse-to-load list.

ld-musl-x86_64.so shouldn't appear elsewhere either.

>> ldconfig: /usr/lib/libc.so is not a symbolic link
>
> IIRC this is happening due to some other misconfiguration. If nothing
> else, it means glibc and musl were both installed in /usr/lib, or
> ldconfig is configured for the wrong paths (since ldconfig has nothing
> to do with musl).

This happens because /lib/ld-musl-x86_64.so.1 has a SONAME of libc.so
(which should be the correct place).  The message is not harmful, but
annoying.

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: musl vs. Debian policy

[Rich Felker]

On Fri, Mar 08, 2013 at 04:13:59PM +0100, Christian Neukirchen wrote:
> >> In this case, could we also change the SONAME of the library itself to
> >> something not libc.so?  It would avoid this "bogus" warning of glibc
> >> ldconfig...
> >
> > No, this is a lot more problematic and I see no benefits. For each
> > possible SONAME musl may have been linked by, musl must contain a
> > special-case to refuse to load this SONAME when it appears in
> > DT_NEEDED. "libc.so" is a name that should never appear elsewhere. I
> > don't want to keep expanding this list of names, and of course
> > programs linked using a new SONAME would be gratuitously incompatible
> > with an older musl ld.so that didn't have the new name included in its
> > refuse-to-load list.
> 
> ld-musl-x86_64.so shouldn't appear elsewhere either.

Yes and no. Formally, libc.so is in a sort of reserved namespace (or
at least, -lc is), whereas there's nothing "reserved" about the name
ld-musl-$(ARCH).so.1. I agree this is fairly irrelevant however as
nobody else is going to use that library name unless they're trying to
break things.

> >> ldconfig: /usr/lib/libc.so is not a symbolic link
> >
> > IIRC this is happening due to some other misconfiguration. If nothing
> > else, it means glibc and musl were both installed in /usr/lib, or
> > ldconfig is configured for the wrong paths (since ldconfig has nothing
> > to do with musl).
> 
> This happens because /lib/ld-musl-x86_64.so.1 has a SONAME of libc.so
> (which should be the correct place).  The message is not harmful, but
> annoying.

Well the message should never happen unless ldconfig is processing the
directory containing libc.so, right? It doesn't happen for me on
Debian when I have musl's ld-musl-i386.so.1 in /lib and ldconfig
processes the default library path.

Rich

Re: musl vs. Debian policy

[Christian Neukirchen]

Rich Felker <dalias@aerifal.cx> writes:

> On Fri, Mar 08, 2013 at 04:13:59PM +0100, Christian Neukirchen wrote:
>> >> In this case, could we also change the SONAME of the library itself to
>> >> something not libc.so?  It would avoid this "bogus" warning of glibc
>> >> ldconfig...
>> >
>> > No, this is a lot more problematic and I see no benefits. For each
>> > possible SONAME musl may have been linked by, musl must contain a
>> > special-case to refuse to load this SONAME when it appears in
>> > DT_NEEDED. "libc.so" is a name that should never appear elsewhere. I
>> > don't want to keep expanding this list of names, and of course
>> > programs linked using a new SONAME would be gratuitously incompatible
>> > with an older musl ld.so that didn't have the new name included in its
>> > refuse-to-load list.
>> 
>> ld-musl-x86_64.so shouldn't appear elsewhere either.
>
> Yes and no. Formally, libc.so is in a sort of reserved namespace (or
> at least, -lc is), whereas there's nothing "reserved" about the name
> ld-musl-$(ARCH).so.1. I agree this is fairly irrelevant however as
> nobody else is going to use that library name unless they're trying to
> break things.
>
>> >> ldconfig: /usr/lib/libc.so is not a symbolic link
>> >
>> > IIRC this is happening due to some other misconfiguration. If nothing
>> > else, it means glibc and musl were both installed in /usr/lib, or
>> > ldconfig is configured for the wrong paths (since ldconfig has nothing
>> > to do with musl).
>> 
>> This happens because /lib/ld-musl-x86_64.so.1 has a SONAME of libc.so
>> (which should be the correct place).  The message is not harmful, but
>> annoying.
>
> Well the message should never happen unless ldconfig is processing the
> directory containing libc.so, right? It doesn't happen for me on
> Debian when I have musl's ld-musl-i386.so.1 in /lib and ldconfig
> processes the default library path.

That is weird...

stat("/usr/lib/libc.so", {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
stat("/usr/lib/ld-musl-x86_64.so.1", {st_mode=S_IFREG|0755, st_size=583007, ...}) = 0
lstat("/usr/lib/libc.so", {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
... glibc printf crap ...
write(2, "ldconfig: ", 10ldconfig: )              = 10
write(2, "/usr/lib/libc.so is not a symbol"..., 40/usr/lib/libc.so is not a symbolic link

Note that /usr/lib/libc.so is a glibc file containing an ld script here.

If I remove /usr/lib/ld-musl-x86_64.so.1, the message goes away.

But I think it also had something to do with 
lrwxrwxrwx   1 root root     7 Jan 27 08:29 lib -> usr/lib/

I thought the issue was that it found ld-musl-x86_64.so.1 and tried to
ensure libc.so points to it, due to the SONAME.

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: musl vs. Debian policy

[Rich Felker]

On Sat, Mar 09, 2013 at 09:08:17PM +0100, Christian Neukirchen wrote:
> Rich Felker <dalias@aerifal.cx> writes:
> 
> > On Fri, Mar 08, 2013 at 04:13:59PM +0100, Christian Neukirchen wrote:
> >> >> In this case, could we also change the SONAME of the library itself to
> >> >> something not libc.so?  It would avoid this "bogus" warning of glibc
> >> >> ldconfig...
> >> >
> >> > No, this is a lot more problematic and I see no benefits. For each
> >> > possible SONAME musl may have been linked by, musl must contain a
> >> > special-case to refuse to load this SONAME when it appears in
> >> > DT_NEEDED. "libc.so" is a name that should never appear elsewhere. I
> >> > don't want to keep expanding this list of names, and of course
> >> > programs linked using a new SONAME would be gratuitously incompatible
> >> > with an older musl ld.so that didn't have the new name included in its
> >> > refuse-to-load list.
> >> 
> >> ld-musl-x86_64.so shouldn't appear elsewhere either.
> >
> > Yes and no. Formally, libc.so is in a sort of reserved namespace (or
> > at least, -lc is), whereas there's nothing "reserved" about the name
> > ld-musl-$(ARCH).so.1. I agree this is fairly irrelevant however as
> > nobody else is going to use that library name unless they're trying to
> > break things.
> >
> >> >> ldconfig: /usr/lib/libc.so is not a symbolic link
> >> >
> >> > IIRC this is happening due to some other misconfiguration. If nothing
> >> > else, it means glibc and musl were both installed in /usr/lib, or
> >> > ldconfig is configured for the wrong paths (since ldconfig has nothing
> >> > to do with musl).
> >> 
> >> This happens because /lib/ld-musl-x86_64.so.1 has a SONAME of libc.so
> >> (which should be the correct place).  The message is not harmful, but
> >> annoying.
> >
> > Well the message should never happen unless ldconfig is processing the
> > directory containing libc.so, right? It doesn't happen for me on
> > Debian when I have musl's ld-musl-i386.so.1 in /lib and ldconfig
> > processes the default library path.
> 
> That is weird...
> 
> stat("/usr/lib/libc.so", {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
> stat("/usr/lib/ld-musl-x86_64.so.1", {st_mode=S_IFREG|0755, st_size=583007, ...}) = 0
> lstat("/usr/lib/libc.so", {st_mode=S_IFREG|0644, st_size=255, ...}) = 0
> .... glibc printf crap ...
> write(2, "ldconfig: ", 10ldconfig: )              = 10
> write(2, "/usr/lib/libc.so is not a symbol"..., 40/usr/lib/libc.so is not a symbolic link
> 
> Note that /usr/lib/libc.so is a glibc file containing an ld script here.
> 
> If I remove /usr/lib/ld-musl-x86_64.so.1, the message goes away.
> 
> But I think it also had something to do with 
> lrwxrwxrwx   1 root root     7 Jan 27 08:29 lib -> usr/lib/
> 
> I thought the issue was that it found ld-musl-x86_64.so.1 and tried to
> ensure libc.so points to it, due to the SONAME.

OK, I think we've found something nasty: on your system this warning
is happening because /usr/lib and /lib are the same directory. On my
system, /lib/libc.so is being silently created by ldconfig, pointing
to ld-musl. The latter behavior is much worse; it could cause -lc to
try to pull in musl, depending on the library path order. I'm working
on a fix.

Rich

Re: musl vs. Debian policy

[Rich Felker]

On Sat, Mar 09, 2013 at 10:29:56PM -0500, Rich Felker wrote:
> > Note that /usr/lib/libc.so is a glibc file containing an ld script here.
> > 
> > If I remove /usr/lib/ld-musl-x86_64.so.1, the message goes away.
> > 
> > But I think it also had something to do with 
> > lrwxrwxrwx   1 root root     7 Jan 27 08:29 lib -> usr/lib/
> > 
> > I thought the issue was that it found ld-musl-x86_64.so.1 and tried to
> > ensure libc.so points to it, due to the SONAME.
> 
> OK, I think we've found something nasty: on your system this warning
> is happening because /usr/lib and /lib are the same directory. On my
> system, /lib/libc.so is being silently created by ldconfig, pointing
> to ld-musl. The latter behavior is much worse; it could cause -lc to
> try to pull in musl, depending on the library path order. I'm working
> on a fix.

Fixed by removing the soname. It was completely unnecessary.

Rich

Re: musl vs. Debian policy

[Isaac Dunham]

On Thu, 7 Mar 2013 08:04:24 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> > I started writing a short explanation of the musl installation for packagers, and realized that there's one area that's inconvenient:
> > $syslib/ld-musl-*.so.1 is a symlink to libc.so.
> > 
> > Debian policy requires that any public libraries have a version number. 
> > Specifically, Debian Policy 8.2 
> > (http://www.debian.org/doc/debian-policy/ch-sharedlibs.html): 
> > If your package contains files whose names do not change with each
> > change in the library shared object version, you must not put them
> > in the shared library package. Otherwise, several versions of the
> > shared library cannot be installed at the same time without filename
> > clashes, making upgrades and transitions unnecessarily difficult.
> 
> Are you sure this applies? I don't think the libc.so in musl counts as
> a "public library" as nothing references it by name. Moreover, if
> installing as a package on a glibc-based distro, it should not be
> installed in /usr/lib (where it would clash with glibc's libc.so
> linker script) but rather its own path.

Public library==any library other than a "convenience library" (a library that is included in a binary package such as xorg-xserver-core, and is not intended to be linked against except programs in the binary package itself).

What the policy is saying is that if you have a shared library 
package (for our example, musl1), it cannot have any file that 
doesn't change names if you have to break ABI compatability.  

The reason the Debian policy gives is that this ensures that 
if we do end up breaking ABI compatability (such as when fixing 
incorrect types or revising pthreads), the musl2 package that will 
be needed can be installed on the same system as musl1, thus 
allowing people to keep using musl1-linked programs until updates 
are available.  
I know it's our policy to maintain a stable ABI, but I doubt that we can guarantee that there will NEVER be an abi breakage.  Not wanting to do something is a bad reason for acting like it won't happen.

> > The apparent solution to this is to ship only the dynamic linker,
> > since this is all we need (the dependency on libc.so is disregarded
> > when it comes to running dynamically linked programs). But
> > currently, actually doing this would be somewhat of a hack.
> > 
> > Is there any prospect of installing lib/libc.so straight to
> > ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
> 
> This has been proposed before, and the main obstacle was build-system
> difficulties if I remember right. I'd still like to consider doing it,
> but it would be nice to be able to do it for its own sake rather than
> for the sake of satisfying distro policy being applied where it
> doesn't make sense. Maybe we can try to figure out Debian's stance
> before we rush into making the change for their sake.

I'm actually more interested in it for the reasons Debian gives (explained above) than "because Debian says that's how to do it."

Of course, the possibe side-effects if Debian packagers were to go 
about it their own way and ended up with a different linker name are also rather obnoxious.  I realize that's not a very pressing issue, 
since there isn't an RFP or ITP yet.

But my main point is to ask whether, if I prepare a patch that 
does the job, it would be welcome.

-- 
Isaac Dunham <idunham@lavabit.com>

Re: musl vs. Debian policy

[Rich Felker]

On Thu, Mar 07, 2013 at 05:03:26PM -0800, Isaac Dunham wrote:
> > > The apparent solution to this is to ship only the dynamic linker,
> > > since this is all we need (the dependency on libc.so is disregarded
> > > when it comes to running dynamically linked programs). But
> > > currently, actually doing this would be somewhat of a hack.
> > > 
> > > Is there any prospect of installing lib/libc.so straight to
> > > ${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
> > 
> > This has been proposed before, and the main obstacle was build-system
> > difficulties if I remember right. I'd still like to consider doing it,
> > but it would be nice to be able to do it for its own sake rather than
> > for the sake of satisfying distro policy being applied where it
> > doesn't make sense. Maybe we can try to figure out Debian's stance
> > before we rush into making the change for their sake.
> 
> I'm actually more interested in it for the reasons Debian gives
> (explained above) than "because Debian says that's how to do it."

The problem is that you don't understand that the reasons DON'T APPLY
to musl. libc.so is purely a filename for the "ld" command to find. It
has nothing to do with runtime or versioning. The only musl library
file that will be searched when loading applications is
/lib/ld-musl-$(ARCH).so.1, which DOES have a version number. The
version number is encoded in the PT_INTERP rather than a DT_NEEDED
header.

> Of course, the possibe side-effects if Debian packagers were to go 
> about it their own way and ended up with a different linker name are
> also rather obnoxious. I realize that's not a very pressing issue,
> since there isn't an RFP or ITP yet.

I don't think this would happen.

> But my main point is to ask whether, if I prepare a patch that 
> does the job, it would be welcome.

If the change is desirable for other reasons, which it might be (we
should review the old threads), then it may very well be accepted. If
the only purpose is making the scripts for packaging on Debian
simpler, I don't think that's a good justification for making a
change.

Overall I think there are probably reasons that ld.so should be the
real file and libc.so the symlink, but we need to evaluate them and
weigh them against any difficulties or incompatibilities they'll
create. I'd like to discuss these issues on their own rather than as
part of a Debian policy discussion.

Rich

Re: musl vs. Debian policy

[Isaac Dunham]

On Thu, 7 Mar 2013 20:10:10 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> 
> The problem is that you don't understand that the reasons DON'T APPLY
> to musl. libc.so is purely a filename for the "ld" command to find. It
> has nothing to do with runtime or versioning. The only musl library
> file that will be searched when loading applications is
> /lib/ld-musl-$(ARCH).so.1, which DOES have a version number. The
> version number is encoded in the PT_INTERP rather than a DT_NEEDED
> header.

I understand that /lib/ld-musl-$(ARCH).so.1 is the only filename that a dynamically-linked binary sees. 
However, if ld-musl-$(ARCH).so.1 is a symlink to libc.so, then anyone preparing a binary package must include libc.so in that package. 
Incrementing the soversion (.so.1 -> .so.2) would not allow parallel installs. To illustrate, here's the layout I'm expecting with multiarch based on the current build system.
musl1:
/usr/lib/i486-linux-musl/libc.so
/lib/ld-musl-i386.so.1 => /usr/lib/i486-linux-musl/libc.so

musl2 (if ABI breaks):
/usr/lib/i486-linux-musl/libc.so
/lib/ld-musl-i386.so.2 => /usr/lib/i486-linux-musl/libc.so

I don't see any way for musl2 to be co-installed in a functional way, unless you change the triplet to i486-linux-musl2. Otherwise, you get the new ABI for both symlinks, breaking old binaries.
A more conventional layout, with all libraries in /usr/lib or /lib, would
have the same problem, but would not have the option of just changing the triplet/libdir.
Even if you are not using a binary-package-based distro, a library with the incremented soversion would still break old libraries, since ld-musl-$(ARCH).so.1 points to the new libc.so.

Is this analysis missing something?

Thank you,
Isaac Dunham <idunham@lavabit.com>

Re: musl vs. Debian policy

[Rich Felker]

On Thu, Mar 07, 2013 at 08:03:54PM -0800, Isaac Dunham wrote:
> On Thu, 7 Mar 2013 20:10:10 -0500
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > 
> > The problem is that you don't understand that the reasons DON'T APPLY
> > to musl. libc.so is purely a filename for the "ld" command to find. It
> > has nothing to do with runtime or versioning. The only musl library
> > file that will be searched when loading applications is
> > /lib/ld-musl-$(ARCH).so.1, which DOES have a version number. The
> > version number is encoded in the PT_INTERP rather than a DT_NEEDED
> > header.
> 
> I understand that /lib/ld-musl-$(ARCH).so.1 is the only filename that a dynamically-linked binary sees. 
> However, if ld-musl-$(ARCH).so.1 is a symlink to libc.so, then anyone preparing a binary package must include libc.so in that package. 
> Incrementing the soversion (.so.1 -> .so.2) would not allow parallel installs. To illustrate, here's the layout I'm expecting with multiarch based on the current build system.
> musl1:
> /usr/lib/i486-linux-musl/libc.so
> /lib/ld-musl-i386.so.1 => /usr/lib/i486-linux-musl/libc.so
> 
> musl2 (if ABI breaks):
> /usr/lib/i486-linux-musl/libc.so
> /lib/ld-musl-i386.so.2 => /usr/lib/i486-linux-musl/libc.so

This is mostly correct. I agree that for this usage the symlink should
go in the opposite direction. Perhaps that's a good enough argument
that the build system should install it to go in the opposite
direction.

However note that if "musl-1" and "musl-2" were really incompatible
enough to need a different version number, sets of libraries linked
against them would probably be likewise different, and it would
probably not work to use the same lib directory for both... That's the
whole reason we _don't_ want to be changing things.

Rich

Re: musl vs. Debian policy

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

On 6 Mar 2013 23:29, "Isaac Dunham" <idunham@lavabit.com> wrote:
>
> I started writing a short explanation of the musl installation for
packagers, and realized that there's one area that's inconvenient:
> $syslib/ld-musl-*.so.1 is a symlink to libc.so.
>
> Debian policy requires that any public libraries have a version number.
> Specifically, Debian Policy 8.2
> (http://www.debian.org/doc/debian-policy/ch-sharedlibs.html):
> If your package contains files whose names do not change with each change
in the library shared object version, you must not put them in the shared
library package. Otherwise, several versions of the shared library cannot
be installed at the same time without filename clashes, making upgrades and
transitions unnecessarily difficult.
>
> The apparent solution to this is to ship only the dynamic linker, since
this is all we need (the dependency on libc.so is disregarded when it comes
to running dynamically linked programs).  But currently, actually doing
this would be somewhat of a hack.
>
> Is there any prospect of installing lib/libc.so straight to
${LDSO_PATHNAME} ? I'm thinking it could be done via something like:
>
> install-ldso: $(DESTDIR)$(LDSO_PATHNAME)
>
> $(DESTDIR)$(LDSO_PATHNAME): lib/libc.so
>         install -D -m 755 $< $@
>
> I realize it would also be necessary to adjust the rules for installing
libc.so, however. If the change is welcome, I could prepare a patch.
>
> Thanks,
> Isaac Dunham <idunham@lavabit.com>
>

What is the idea of packaging Musl for Debian? I can see several options
but none of them seem very plausible. No other package is likely to require
Musl. A Musl based Debian might be nice but that's a very different
requirement. Maybe I am missing something.

Justin

[-- Attachment #2: Type: text/html, Size: 2192 bytes --]

Re: musl vs. Debian policy

[Isaac Dunham]

On Thu, 7 Mar 2013 18:56:35 +0000
Justin Cormack <justin@specialbusservice.com> wrote:

> 
> What is the idea of packaging Musl for Debian? I can see several options
> but none of them seem very plausible. No other package is likely to require
> Musl. A Musl based Debian might be nice but that's a very different
> requirement. Maybe I am missing something.

Mainly the same reason they include klibc, dietlibc, and uclibc-source: a small libc for initrds, small static binaries, and cross-compilation.
mksh, ngetty, and slidentd all refer to use of dietlibc, and klibc is used for initrds. 

musl's policy about maintaining ABI and availability as a shared library may prove advantageous to Debian, also.  

Up till now I've been speaking about reasons a musl package would be useful within a libc6-based distro; but a musl package is necessary for a Debian musl port. A good number of embedded systems start off Debian, Emdebian, or the uclibc port of Debian. The ability to build a Debian system based around musl may aid in adoption of musl, and Debian's attitude towards compatability makes it a good place to start.

I should note that with multiarch, it actually would be possible to use a prepackaged musl for cross-compilation. 

HTH, 
Isaac Dunham <idunham@lavabit.com>

Re: musl vs. Debian policy

[Rob Landley]

On 03/06/2013 05:29:13 PM, Isaac Dunham wrote:
> I started writing a short explanation of the musl installation for  
> packagers, and realized that there's one area that's inconvenient:
> $syslib/ld-musl-*.so.1 is a symlink to libc.so.
> 
> Debian policy requires that any public libraries have a version  
> number.

Looks like it's "1" here.

> Specifically, Debian Policy 8.2
> (http://www.debian.org/doc/debian-policy/ch-sharedlibs.html):
> If your package contains files whose names do not change with each  
> change in the
> library shared object version, you must not put them in the shared  
> library package.
> Otherwise, several versions of the shared library cannot be installed  
> at the same
> time without filename clashes, making upgrades and transitions  
> unnecessarily
> difficult.

Debian is incapable of renaming files when packaging them into .debs or  
installing them, in order to enforce Debian's own policies?

> The apparent solution to this is to ship only the dynamic linker,  
> since this is all
> we need (the dependency on libc.so is disregarded when it comes to  
> running
> dynamically linked programs).  But currently, actually doing this  
> would be somewhat
> of a hack.

Um, you said the dynamic linker name is a symlink to libc.so? So what  
does "ship only the dynamic linker" mean in this context?

I'm confused.

Rob

Re: musl vs. Debian policy

[Isaac Dunham]

On Thu, 07 Mar 2013 16:30:30 -0600
Rob Landley <rob@landley.net> wrote:

> > Debian policy requires that any public libraries have a version  
> > number.
> 
> Looks like it's "1" here.
True, though shipping a "libc.so.1" might not be the ideal choice (mainly for publicity-related reasons).

> > Specifically, Debian Policy 8.2
> > (http://www.debian.org/doc/debian-policy/ch-sharedlibs.html):
> > If your package contains files whose names do not change with each  
> > change in the
> > library shared object version, you must not put them in the shared  
> > library package.
> > Otherwise, several versions of the shared library cannot be installed  
> > at the same
> > time without filename clashes, making upgrades and transitions  
> > unnecessarily
> > difficult.
> 
> Debian is incapable of renaming files when packaging them into .debs or  
> installing them, in order to enforce Debian's own policies?

It's quite possible; the issue is whether we'd end up "encouraging" them to package musl in a way that guarantees incompatability with everyone else.  If they install the x86_64 version as 
"/lib/ld-musl-amd64-el.so.1" 
(what dpkg-architecture might encourage if debian/rules installs libc.so itself), then musl on Debian amd64 would be incompatible with musl elsewhere.

> > The apparent solution to this is to ship only the dynamic linker,  
> > since this is all
> > we need (the dependency on libc.so is disregarded when it comes to  
> > running
> > dynamically linked programs).  But currently, actually doing this  
> > would be somewhat
> > of a hack.
> 
> Um, you said the dynamic linker name is a symlink to libc.so? So what  
> does "ship only the dynamic linker" mean in this context?
> 
mv libc.so ld-musl-$ARCH.so.1
ln -s ld-musl-$ARCH.so.1 libc.so # link goes in musl-dev

Does that clarify things?

-- 
Isaac Dunham <idunham@lavabit.com>

Re: musl vs. Debian policy

[Kurt H Maier]

On Thu, Mar 07, 2013 at 04:22:06PM -0800, Isaac Dunham wrote:
> 
> It's quite possible; the issue is whether we'd end up "encouraging" them to package musl in a way that guarantees incompatability with everyone else.  If they install the x86_64 version as 
> "/lib/ld-musl-amd64-el.so.1" 
> (what dpkg-architecture might encourage if debian/rules installs libc.so itself), then musl on Debian amd64 would be incompatible with musl elsewhere.

Then their packaging policies have failed.  This is still a Debian
problem and not a musl problem.

khm