Preparing to release 0.9.12

Preparing to release 0.9.12

[Rich Felker]

Hi everyone,

The list of changes since 0.9.11 has grown quite large, and although
we haven't met some of the roadmap goals for 0.9.12, others that were
marked for 0.9.13 have already been finished. So I think it's
reasonable to aim to release very soon now. There are still a few
pending items I'd like to get committed before the release:

- orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
- Andre's ARM memcpy optimizations
- New crt1.c code for adding PIE support for more archs
- MAYBE the symlink direction issue...

Otherwise, between now and release I want to focus on detecting any
regressions. I don't expect any functional regressions, but it's
possible that the header refactoring broke some fragile, incorrect
applications. Reportedly sabotage needed 3 extra patches to work
around broken programs using __uint64_t and similar. Feedback from
testing with current git (especially current git's headers) would be
very helpful.

Rich

Re: Preparing to release 0.9.12

[Timo Teras]

On Wed, 24 Jul 2013 16:02:21 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> The list of changes since 0.9.11 has grown quite large, and although
> we haven't met some of the roadmap goals for 0.9.12, others that were
> marked for 0.9.13 have already been finished. So I think it's
> reasonable to aim to release very soon now. There are still a few
> pending items I'd like to get committed before the release:
> 
> - orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
> - Andre's ARM memcpy optimizations
> - New crt1.c code for adding PIE support for more archs
> - MAYBE the symlink direction issue...

Since the C++ ABI was fixed, it means that any current native musl
toolchain will get C++ ABI breakage?

In this case the symlink direction issue would help with smoother
transitions. It would be also crucial to start using proper SONAME
versioning, so we could handle binary upgrades smoothly.

Relatedly, commit f389c4984a (make the dynamic linker find its path
file relative to its own location) introduced the armeb, armhf
variants. Fundamentally, these are distribution specific names. I
believe debian has/had armeb (big-endian OABI; being retired), arm
(little-endian OABI; dead port), armel (little-endian EABI), and now
armhf (little-endian EABI with hard-float). But these are by no means
standard. While it is good that LDSO_ARCH gets good default with this
distinguished. It should be allowed to be overridden by distributions.

So basically I'd like to give at configure time:
 DISTRO_ARCH=armel

Which would then make LDSO_ARCH, and SONAME accordingly. And with the
soname version set, we could do proper binary ugprades (even when not
everything is yet rebuilt).

Thanks,
 Timo

Re: Preparing to release 0.9.12

[Szabolcs Nagy]

* Timo Teras <timo.teras@iki.fi> [2013-07-25 10:44:59 +0300]:
> Relatedly, commit f389c4984a (make the dynamic linker find its path
> file relative to its own location) introduced the armeb, armhf
> variants. Fundamentally, these are distribution specific names. I
> believe debian has/had armeb (big-endian OABI; being retired), arm
> (little-endian OABI; dead port), armel (little-endian EABI), and now
> armhf (little-endian EABI with hard-float). But these are by no means
> standard. While it is good that LDSO_ARCH gets good default with this
> distinguished. It should be allowed to be overridden by distributions.
> 
> So basically I'd like to give at configure time:
>  DISTRO_ARCH=armel
> 
> Which would then make LDSO_ARCH, and SONAME accordingly. And with the
> soname version set, we could do proper binary ugprades (even when not
> everything is yet rebuilt).

why do you care about the name of the loader?

musl could use a random string, the only requirement
is that each abi has a unique identifier so there is
no collision

if the name changes between distros then a simple
binary that only depends on libc becomes non-portable

Re: Preparing to release 0.9.12

[Timo Teras]

On Thu, 25 Jul 2013 17:05:23 +0200
Szabolcs Nagy <nsz@port70.net> wrote:

> * Timo Teras <timo.teras@iki.fi> [2013-07-25 10:44:59 +0300]:
> > Relatedly, commit f389c4984a (make the dynamic linker find its path
> > file relative to its own location) introduced the armeb, armhf
> > variants. Fundamentally, these are distribution specific names. I
> > believe debian has/had armeb (big-endian OABI; being retired), arm
> > (little-endian OABI; dead port), armel (little-endian EABI), and now
> > armhf (little-endian EABI with hard-float). But these are by no
> > means standard. While it is good that LDSO_ARCH gets good default
> > with this distinguished. It should be allowed to be overridden by
> > distributions.
> > 
> > So basically I'd like to give at configure time:
> >  DISTRO_ARCH=armel
> > 
> > Which would then make LDSO_ARCH, and SONAME accordingly. And with
> > the soname version set, we could do proper binary ugprades (even
> > when not everything is yet rebuilt).
> 
> why do you care about the name of the loader?
> 
> musl could use a random string, the only requirement
> is that each abi has a unique identifier so there is
> no collision
> 
> if the name changes between distros then a simple
> binary that only depends on libc becomes non-portable

Well, when we have native tool chain, the loader name will have musl in
it, and it's not portable to non-musl systems.

I can probably live with the default LDSO_ARCH name as long as so
versioning is done properly. But I think it would still be useful to
let it be overriden, in case distro decides to add patches that break
ABI.

- Timo

Re: Preparing to release 0.9.12

[Rich Felker]

On Thu, Jul 25, 2013 at 06:45:51PM +0300, Timo Teras wrote:
> > > Which would then make LDSO_ARCH, and SONAME accordingly. And with
> > > the soname version set, we could do proper binary ugprades (even
> > > when not everything is yet rebuilt).
> > 
> > why do you care about the name of the loader?
> > 
> > musl could use a random string, the only requirement
> > is that each abi has a unique identifier so there is
> > no collision
> > 
> > if the name changes between distros then a simple
> > binary that only depends on libc becomes non-portable
> 
> Well, when we have native tool chain, the loader name will have musl in
> it, and it's not portable to non-musl systems.

The binaries won't work without musl anyway.

> I can probably live with the default LDSO_ARCH name as long as so
> versioning is done properly. But I think it would still be useful to

We should figure out what "properly" means then. I think arranging for
the soname to contain the version for package management purposes, but
no other use, would probably meet your needs. Unlike glibc, musl aims
not to force you to update libc.so just because an application was
compiled against a newer version, but only if it actually needs a
feature in the newer version. (This is similar to how versioning works
with the kernel, BTW.) But distros often will want to impose this
requirement just to make sure a sufficiently new version is available
to have all features any of their packages might need.

> let it be overriden, in case distro decides to add patches that break
> ABI.

If the distro is doing this, the patch could (and should) change the
dynamic linker name at the same time it makes the other changes.
However I think you really over-estimate the need for breaking ABI.
It should not happen. glibc has not broken ABI in over a decade, and
aside from the C++ ABI (which is not part of the app-to-libc ABI),
musl has not broken ABI at all except to fix some types that were
defined so wrong that the associated interfaces didn't work.

If ABI changes were needed, moving to a new verison number for the
dynamic linker would be the right approach, I think. But I don't see
that happening anytime soon.

Rich

Re: Preparing to release 0.9.12

[Rich Felker]

On Thu, Jul 25, 2013 at 10:44:59AM +0300, Timo Teras wrote:
> On Wed, 24 Jul 2013 16:02:21 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > The list of changes since 0.9.11 has grown quite large, and although
> > we haven't met some of the roadmap goals for 0.9.12, others that were
> > marked for 0.9.13 have already been finished. So I think it's
> > reasonable to aim to release very soon now. There are still a few
> > pending items I'd like to get committed before the release:
> > 
> > - orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
> > - Andre's ARM memcpy optimizations
> > - New crt1.c code for adding PIE support for more archs
> > - MAYBE the symlink direction issue...
> 
> Since the C++ ABI was fixed, it means that any current native musl
> toolchain will get C++ ABI breakage?
> 
> In this case the symlink direction issue would help with smoother
> transitions. It would be also crucial to start using proper SONAME
> versioning, so we could handle binary upgrades smoothly.

This would not help. The ABI between the application and musl's
libc.so has not changed since just after the initial public release,
except for some bugs that had arch-specific structures laid out wrong,
etc. -- and in this case, programs built before the fix were not
working at all anyway when using the affected feature.

What's affected by the C++ ABI compatibility changes is the ABI
between C++ applications and third-party libraries built against
musl's headers. As far as I know, there is no decent way around this
with versioning (library versioning or symbol versioning). There are
two ways this type of ABI linkage can come into play:

- When the size of a type is not relevant to the interface between
  libc and its caller, but affects the size of structs containing the
  type, which another library might define and use as part of its
  interface with its caller.

- When the C++ compiler encodes the exact underlying type (for
  non-aggregates) or the struct/union/enum tag into the mangled
  symbol name for functions which take the type as an argument, and
  both the caller and callee have to agree or you'll get unsatisfied
  symbol references.

Because C++ ABI is so fragile, musl has not had an official C++ ABI
until now. In fact, it's changed in subtle ways several times already
this year due to other bugs found, where musl's idea of a type's
identity disagreed with the compiler's idea of what it should be for
the particular psABI. (These needed to be fixed because they affect
things like printf format string warnings, and, for wchar_t, size_t,
ptrdiff_t, and perhaps a few others, they really need to match the
type of certain expressions (L"x", sizeof(x), p1-p2)). While
compatibility with C++ libraries built against glibc is part of the
motivation of the ABI-compat changes recently made, and equally
important aspect is just _stabilizing_ the ABI. By having a target ABI
to match (and test that we match), it's easy to know we don't have
lingering type mismatch bugs and thus that we won't need to break the
C++ ABI in the future.

I apologize that the lack of C++ ABI stability was not warned about
prominently in previous releases. If you (or anyone else) have
deployed packages that you think will break, I can look into adding a
layer in the dynamic linker that would, when a C++ symbol lookup
fails, try substituting differently-mangled symbols which differ from
the desired one only in the nominal identity of the type but not in
its size or representation. This would not be too hard, but unless
it's needed, it's probably a bad idea.

Finally, please note that any breakage from the C++ ABI changes will
be unresolved symbol errors at startup, not application misbehavior.

> Relatedly, commit f389c4984a (make the dynamic linker find its path
> file relative to its own location) introduced the armeb, armhf
> variants. Fundamentally, these are distribution specific names. I
> believe debian has/had armeb (big-endian OABI; being retired), arm
> (little-endian OABI; dead port), armel (little-endian EABI), and now
> armhf (little-endian EABI with hard-float). But these are by no means
> standard. While it is good that LDSO_ARCH gets good default with this
> distinguished. It should be allowed to be overridden by distributions.
> 
> So basically I'd like to give at configure time:
>  DISTRO_ARCH=armel

Why? The goal isn't to match the distro's naming. It's to have a
unified name for any musl-linked binaries using this arch, so that if
you move them from one system to another (possibly with different
distro) they still work. The arch and subarch names used are unique
within the "ld-musl-..." namespace, so there's no danger of them
conflicting with other things in the distro. If you would prefer
different names, I would be happy to consider changing them at this
point (since there hasn't been a release with them anyway). However, I
think it's really counter-productive to use distro-specific PT_INTERP.
All that does is prevent someone from using your musl binaries on
another distro, or vice versa; the ability to do that is intended to
be one of musl's advantages over glibc.

Rich

Re: Preparing to release 0.9.12

[Timo Teras]

On Thu, 25 Jul 2013 12:16:55 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Thu, Jul 25, 2013 at 10:44:59AM +0300, Timo Teras wrote:
> > On Wed, 24 Jul 2013 16:02:21 -0400
> > Rich Felker <dalias@aerifal.cx> wrote:
> > 
> > > The list of changes since 0.9.11 has grown quite large, and
> > > although we haven't met some of the roadmap goals for 0.9.12,
> > > others that were marked for 0.9.13 have already been finished. So
> > > I think it's reasonable to aim to release very soon now. There
> > > are still a few pending items I'd like to get committed before
> > > the release:
> > > 
> > > - orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
> > > - Andre's ARM memcpy optimizations
> > > - New crt1.c code for adding PIE support for more archs
> > > - MAYBE the symlink direction issue...
> > 
> > Since the C++ ABI was fixed, it means that any current native musl
> > toolchain will get C++ ABI breakage?
> > 
> > In this case the symlink direction issue would help with smoother
> > transitions. It would be also crucial to start using proper SONAME
> > versioning, so we could handle binary upgrades smoothly.
> 
> This would not help. The ABI between the application and musl's
> libc.so has not changed since just after the initial public release,
> except for some bugs that had arch-specific structures laid out wrong,
> etc. -- and in this case, programs built before the fix were not
> working at all anyway when using the affected feature.

The so versioning will not help for C++ related things. The most
important use case I had in mind is that, package managers that use
soversions for automatic dependencies, can insert proper "require
version XXX or later of this .so". That is, if we built with musl X, we
can detect that from .so, and record it. And later ensure that musl X-1
will not satisfy the newly built package's dependencies. Especially
important when we are introducing new symbols.

> What's affected by the C++ ABI compatibility changes is the ABI
> between C++ applications and third-party libraries built against
> musl's headers. As far as I know, there is no decent way around this
> with versioning (library versioning or symbol versioning). There are
> two ways this type of ABI linkage can come into play:
> 
> - When the size of a type is not relevant to the interface between
>   libc and its caller, but affects the size of structs containing the
>   type, which another library might define and use as part of its
>   interface with its caller.
> 
> - When the C++ compiler encodes the exact underlying type (for
>   non-aggregates) or the struct/union/enum tag into the mangled
>   symbol name for functions which take the type as an argument, and
>   both the caller and callee have to agree or you'll get unsatisfied
>   symbol references.
> 
> Because C++ ABI is so fragile, musl has not had an official C++ ABI
> until now. In fact, it's changed in subtle ways several times already
> this year due to other bugs found, where musl's idea of a type's
> identity disagreed with the compiler's idea of what it should be for
> the particular psABI. (These needed to be fixed because they affect
> things like printf format string warnings, and, for wchar_t, size_t,
> ptrdiff_t, and perhaps a few others, they really need to match the
> type of certain expressions (L"x", sizeof(x), p1-p2)). While
> compatibility with C++ libraries built against glibc is part of the
> motivation of the ABI-compat changes recently made, and equally
> important aspect is just _stabilizing_ the ABI. By having a target ABI
> to match (and test that we match), it's easy to know we don't have
> lingering type mismatch bugs and thus that we won't need to break the
> C++ ABI in the future.
> 
> I apologize that the lack of C++ ABI stability was not warned about
> prominently in previous releases. If you (or anyone else) have
> deployed packages that you think will break, I can look into adding a
> layer in the dynamic linker that would, when a C++ symbol lookup
> fails, try substituting differently-mangled symbols which differ from
> the desired one only in the nominal identity of the type but not in
> its size or representation. This would not be too hard, but unless
> it's needed, it's probably a bad idea.

Full agree. And no, no need for the symbol name "re"-mangling hack.
Still working on my initial build.

> Finally, please note that any breakage from the C++ ABI changes will
> be unresolved symbol errors at startup, not application misbehavior.

Yes. I figured this. And it would be namely on the C++ libraries. Not
on musl itself, as the musl symbols never changed. It's just C++
compiler's perception of the type names.

> > Relatedly, commit f389c4984a (make the dynamic linker find its path
> > file relative to its own location) introduced the armeb, armhf
> > variants. Fundamentally, these are distribution specific names. I
> > believe debian has/had armeb (big-endian OABI; being retired), arm
> > (little-endian OABI; dead port), armel (little-endian EABI), and now
> > armhf (little-endian EABI with hard-float). But these are by no
> > means standard. While it is good that LDSO_ARCH gets good default
> > with this distinguished. It should be allowed to be overridden by
> > distributions.
> > 
> > So basically I'd like to give at configure time:
> >  DISTRO_ARCH=armel
> 
> Why? The goal isn't to match the distro's naming. It's to have a
> unified name for any musl-linked binaries using this arch, so that if
> you move them from one system to another (possibly with different
> distro) they still work. The arch and subarch names used are unique
> within the "ld-musl-..." namespace, so there's no danger of them
> conflicting with other things in the distro. If you would prefer
> different names, I would be happy to consider changing them at this
> point (since there hasn't been a release with them anyway). However, I
> think it's really counter-productive to use distro-specific PT_INTERP.
> All that does is prevent someone from using your musl binaries on
> another distro, or vice versa; the ability to do that is intended to
> be one of musl's advantages over glibc.

Well, now that you've added most needed subarches, I'm probably ok. I
was just anticipating that if toolchain gets new feature affecting ABI,
you need to track those all. But if you aim universal compatibility
then this needs to be done. I guess there will not be insane amount of
subarches, but you never know about future.

- Timo

Re: Preparing to release 0.9.12

[Rich Felker]

On Thu, Jul 25, 2013 at 07:59:55PM +0300, Timo Teras wrote:
> > I apologize that the lack of C++ ABI stability was not warned about
> > prominently in previous releases. If you (or anyone else) have
> > deployed packages that you think will break, I can look into adding a
> > layer in the dynamic linker that would, when a C++ symbol lookup
> > fails, try substituting differently-mangled symbols which differ from
> > the desired one only in the nominal identity of the type but not in
> > its size or representation. This would not be too hard, but unless
> > it's needed, it's probably a bad idea.
> 
> Full agree. And no, no need for the symbol name "re"-mangling hack.
> Still working on my initial build.

Great. This is why I've been rushing to get these changes done sooner
rather than later -- I want users of musl to be starting off with
something that's stable.

> > Finally, please note that any breakage from the C++ ABI changes will
> > be unresolved symbol errors at startup, not application misbehavior.
> 
> Yes. I figured this. And it would be namely on the C++ libraries. Not
> on musl itself, as the musl symbols never changed. It's just C++
> compiler's perception of the type names.

Exactly.

> > > Relatedly, commit f389c4984a (make the dynamic linker find its path
> > > file relative to its own location) introduced the armeb, armhf
> > > variants. Fundamentally, these are distribution specific names. I
> > > believe debian has/had armeb (big-endian OABI; being retired), arm
> > > (little-endian OABI; dead port), armel (little-endian EABI), and now
> > > armhf (little-endian EABI with hard-float). But these are by no
> > > means standard. While it is good that LDSO_ARCH gets good default
> > > with this distinguished. It should be allowed to be overridden by
> > > distributions.
> [...]
> Well, now that you've added most needed subarches, I'm probably ok. I
> was just anticipating that if toolchain gets new feature affecting ABI,
> you need to track those all. But if you aim universal compatibility
> then this needs to be done. I guess there will not be insane amount of
> subarches, but you never know about future.

I should add some documentation on this on the wiki. Defining what is
part of "the ABI" is tricky. For example, you could say code built for
armv5 is a different ABI from code built for armv4, since it's using
instructions that aren't available on armv4. However, the key reason I
don't consider those "different ABIs" is that, as long as the cpu
you're running on supports all the instructions used in the main
program and all libraries, you can use a mix of armv4 and armv5 built
modules with no restrictions.

The ABI variants musl is dealing with in the subarch naming system are
basically limited to:

- Archs that have both little and big endian variants, which really
  should be thought of as completely different archs, but since they
  share all the code, it's more practical to treat them as subarchs.

- Variations in calling convention, for example, passing floating
  point arguments in floating point registers versus general purpose
  registers or on the stack. For example, on arm, -mfloat-abi=soft and
  -mfloat-abi=softfp are both the same subarch, because the calling
  convention is the same, and code built with the two modes can be
  linked together. But -mfloat-abi=hard is a separate subarch because
  you cannot link it with code built with the other options (unless
  none of the interfaces between the modules use float).

Basically, configurations A and B are different archs or subarchs if
you can't link code built with configuration A to code built with
configuration B.

Rich

Re: Preparing to release 0.9.12

[Rich Felker]

On Thu, Jul 25, 2013 at 01:18:06PM -0400, Rich Felker wrote:
> I should add some documentation on this on the wiki. Defining what is
> part of "the ABI" is tricky. For example, you could say code built for
> armv5 is a different ABI from code built for armv4, since it's using
> instructions that aren't available on armv4. However, the key reason I
> don't consider those "different ABIs" is that, as long as the cpu
> you're running on supports all the instructions used in the main
> program and all libraries, you can use a mix of armv4 and armv5 built
> modules with no restrictions.
> [...]
> Basically, configurations A and B are different archs or subarchs if
> you can't link code built with configuration A to code built with
> configuration B.

For what it's worth, this agrees with the outcome of Debian's
discussion on adding the armhf port:

    The eventual conclusion was that port names in Debian should
    encode incompatible ABIs, not compatible variations within an ABI
    (such as CPU optimisations, referred to as 'flavours'). The
    default flavour for a port can change over time as older CPUs
    become obsolete. (e.g. the i386 architecture has been built for
    386, 486 and 586 flavours over time). Rebuilds of a port for a new
    flavour within the ABI are possible to gain performance
    improvements, but Debian itself normally provides builds to the
    lowest common denominator still in widespread use, maximising
    generality. Thus attempts to encode all the possible flavour
    options in the port name were unnecessary and produced long and
    awkward names. A better solution to the problem of recording the
    flavour to which a port or package is built is suitable package
    metadata.

Source: http://wiki.debian.org/ArmHardFloatPort

Rich

Re: Preparing to release 0.9.12

[Isaac]

On Thu, Jul 25, 2013 at 07:59:55PM +0300, Timo Teras wrote:
> On Thu, 25 Jul 2013 12:16:55 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > On Thu, Jul 25, 2013 at 10:44:59AM +0300, Timo Teras wrote:
> > > On Wed, 24 Jul 2013 16:02:21 -0400
> > > Rich Felker <dalias@aerifal.cx> wrote:
> > > 
> > > > The list of changes since 0.9.11 has grown quite large, and
> > > > although we haven't met some of the roadmap goals for 0.9.12,
> > > > others that were marked for 0.9.13 have already been finished. So
> > > > I think it's reasonable to aim to release very soon now. There
> > > > are still a few pending items I'd like to get committed before
> > > > the release:
> > > > 
> > > > - orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
> > > > - Andre's ARM memcpy optimizations
> > > > - New crt1.c code for adding PIE support for more archs
> > > > - MAYBE the symlink direction issue...
> > > 
> > > Since the C++ ABI was fixed, it means that any current native musl
> > > toolchain will get C++ ABI breakage?
> > > 
> > > In this case the symlink direction issue would help with smoother
> > > transitions. It would be also crucial to start using proper SONAME
> > > versioning, so we could handle binary upgrades smoothly.
> > 
> > This would not help. The ABI between the application and musl's
> > libc.so has not changed since just after the initial public release,
> > except for some bugs that had arch-specific structures laid out wrong,
> > etc. -- and in this case, programs built before the fix were not
> > working at all anyway when using the affected feature.
> 
> The so versioning will not help for C++ related things. The most
> important use case I had in mind is that, package managers that use
> soversions for automatic dependencies, can insert proper "require
> version XXX or later of this .so". That is, if we built with musl X, we
> can detect that from .so, and record it. And later ensure that musl X-1
> will not satisfy the newly built package's dependencies. Especially
> important when we are introducing new symbols.
 

On Debian, there's the "symbols" system; this lists all symbols with the 
version they appeared in, and the tools look through the symbols and 
find the lowest version providing all the symbols.

But as a standard rule, _added_ symbols _do_ _not_ call for a new SONAME,
since they do not break the ABI.


--
Isaac Dunham

Re: Preparing to release 0.9.12

[Rich Felker]

On Thu, Jul 25, 2013 at 05:38:51PM -0700, Isaac wrote:
> > The so versioning will not help for C++ related things. The most
> > important use case I had in mind is that, package managers that use
> > soversions for automatic dependencies, can insert proper "require
> > version XXX or later of this .so". That is, if we built with musl X, we
> > can detect that from .so, and record it. And later ensure that musl X-1
> > will not satisfy the newly built package's dependencies. Especially
> > important when we are introducing new symbols.
> 
> On Debian, there's the "symbols" system; this lists all symbols with the 
> version they appeared in, and the tools look through the symbols and 
> find the lowest version providing all the symbols.
> 
> But as a standard rule, _added_ symbols _do_ _not_ call for a new SONAME,
> since they do not break the ABI.

Good point. In that case, it sounds like it would be wrong to have the
SONAME of musl reflect the version, no?

Rich

Re: Preparing to release 0.9.12

[Timo Teras]

On Thu, 25 Jul 2013 17:38:51 -0700
Isaac <idunham@lavabit.com> wrote:

> On Thu, Jul 25, 2013 at 07:59:55PM +0300, Timo Teras wrote:
> > On Thu, 25 Jul 2013 12:16:55 -0400
> > Rich Felker <dalias@aerifal.cx> wrote:
> > 
> > > On Thu, Jul 25, 2013 at 10:44:59AM +0300, Timo Teras wrote:
> > > > On Wed, 24 Jul 2013 16:02:21 -0400
> > > > Rich Felker <dalias@aerifal.cx> wrote:
> > > > 
> > > > > The list of changes since 0.9.11 has grown quite large, and
> > > > > although we haven't met some of the roadmap goals for 0.9.12,
> > > > > others that were marked for 0.9.13 have already been
> > > > > finished. So I think it's reasonable to aim to release very
> > > > > soon now. There are still a few pending items I'd like to get
> > > > > committed before the release:
> > > > > 
> > > > > - orc's getaddrinfo fix for AF_UNSPEC with NULL hostname
> > > > > - Andre's ARM memcpy optimizations
> > > > > - New crt1.c code for adding PIE support for more archs
> > > > > - MAYBE the symlink direction issue...
> > > > 
> > > > Since the C++ ABI was fixed, it means that any current native
> > > > musl toolchain will get C++ ABI breakage?
> > > > 
> > > > In this case the symlink direction issue would help with
> > > > smoother transitions. It would be also crucial to start using
> > > > proper SONAME versioning, so we could handle binary upgrades
> > > > smoothly.
> > > 
> > > This would not help. The ABI between the application and musl's
> > > libc.so has not changed since just after the initial public
> > > release, except for some bugs that had arch-specific structures
> > > laid out wrong, etc. -- and in this case, programs built before
> > > the fix were not working at all anyway when using the affected
> > > feature.
> > 
> > The so versioning will not help for C++ related things. The most
> > important use case I had in mind is that, package managers that use
> > soversions for automatic dependencies, can insert proper "require
> > version XXX or later of this .so". That is, if we built with musl
> > X, we can detect that from .so, and record it. And later ensure
> > that musl X-1 will not satisfy the newly built package's
> > dependencies. Especially important when we are introducing new
> > symbols.
>  
> 
> On Debian, there's the "symbols" system; this lists all symbols with
> the version they appeared in, and the tools look through the symbols
> and find the lowest version providing all the symbols.
> 
> But as a standard rule, _added_ symbols _do_ _not_ call for a new
> SONAME, since they do not break the ABI.

Correct.

The usual way is:
soname = <lib-name>.<abiver>
filename = <soname>.<lib-version>

And then have symlink soname -> filename. This would allow side-by-side
installation of different library versions if needed.

But the "symbols" system looks interesting too. If doing that, the
lib-version would be then. A good related read was (explains also the
soname/filename concept):
http://developer.ubuntu.com/packaging/html/libraries.html

While SONAME we want to keep stable, and change only in the unlikely
event of abi breakage. I think it'd be still nice allow easily the
<lib-version> suffix to the generated file.

Thanks,
 Timo

Re: Preparing to release 0.9.12

[Rich Felker]

On Fri, Jul 26, 2013 at 08:13:27AM +0300, Timo Teras wrote:
> > > The so versioning will not help for C++ related things. The most
> > > important use case I had in mind is that, package managers that use
> > > soversions for automatic dependencies, can insert proper "require
> > > version XXX or later of this .so". That is, if we built with musl
> > > X, we can detect that from .so, and record it. And later ensure
> > > that musl X-1 will not satisfy the newly built package's
> > > dependencies. Especially important when we are introducing new
> > > symbols.
> >  
> > 
> > On Debian, there's the "symbols" system; this lists all symbols with
> > the version they appeared in, and the tools look through the symbols
> > and find the lowest version providing all the symbols.
> > 
> > But as a standard rule, _added_ symbols _do_ _not_ call for a new
> > SONAME, since they do not break the ABI.
> 
> Correct.
> 
> The usual way is:
> soname = <lib-name>.<abiver>
> filename = <soname>.<lib-version>
> 
> And then have symlink soname -> filename. This would allow side-by-side
> installation of different library versions if needed.
> 
> But the "symbols" system looks interesting too. If doing that, the
> lib-version would be then. A good related read was (explains also the
> soname/filename concept):
> http://developer.ubuntu.com/packaging/html/libraries.html
> 
> While SONAME we want to keep stable, and change only in the unlikely
> event of abi breakage. I think it'd be still nice allow easily the
> <lib-version> suffix to the generated file.

One thing to keep in mind with libc is that you want to be able to
safely and atomically replace it during an upgrade without any
intermediate state where the system is unusable. This means the actual
filename (as opposed to symlink) needs to be something that does not
change between versions. If, for example, you had:

/lib/ld-musl-$(ARCH).so.1 -> /lib/musl.so.1.0.0

and wanted to upgrade to musl 1.0.1, you would have to change the
symlink to point to a different name. But there is (as far as I know)
no way to replace a symlink atomically; you have to unlink it first
then make a new symlink. And this leaves a race window during which
exec() could fail.

If the real file's name is something version-independent, however
(either the current direction symlink or the reverse), then upgrading
is simple:

rename("libc.so.tmp12345", "libc.so");

or

rename("/lib/ld-musl-i386.so.1.tmp12345", "/lib/ld-musl-i386.so.1");

It's atomic at runtime, and on a robust filesystem, there will not be
a chance of ending up with an unusable system even if it crashes
during the upgrade.

I don't see any good way to bring version numbers into this.

Rich

Re: Preparing to release 0.9.12

[Timo Teras]

On Fri, 26 Jul 2013 01:34:20 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> One thing to keep in mind with libc is that you want to be able to
> safely and atomically replace it during an upgrade without any
> intermediate state where the system is unusable. This means the actual
> filename (as opposed to symlink) needs to be something that does not
> change between versions. If, for example, you had:
> 
> /lib/ld-musl-$(ARCH).so.1 -> /lib/musl.so.1.0.0
> 
> and wanted to upgrade to musl 1.0.1, you would have to change the
> symlink to point to a different name. But there is (as far as I know)
> no way to replace a symlink atomically; you have to unlink it first
> then make a new symlink. And this leaves a race window during which
> exec() could fail.

man rename(2) says:
       If  oldpath  refers  to a symbolic link the link is renamed; if
       newpath refers to a symbolic link the link will be overwritten.

But if this is linux specific, and we want to support other operating
system kernels than linux, then this might be an issue.

- Timo

Re: Preparing to release 0.9.12

[Rich Felker]

On Fri, Jul 26, 2013 at 09:07:51AM +0300, Timo Teras wrote:
> On Fri, 26 Jul 2013 01:34:20 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > One thing to keep in mind with libc is that you want to be able to
> > safely and atomically replace it during an upgrade without any
> > intermediate state where the system is unusable. This means the actual
> > filename (as opposed to symlink) needs to be something that does not
> > change between versions. If, for example, you had:
> > 
> > /lib/ld-musl-$(ARCH).so.1 -> /lib/musl.so.1.0.0
> > 
> > and wanted to upgrade to musl 1.0.1, you would have to change the
> > symlink to point to a different name. But there is (as far as I know)
> > no way to replace a symlink atomically; you have to unlink it first
> > then make a new symlink. And this leaves a race window during which
> > exec() could fail.
> 
> man rename(2) says:
>        If  oldpath  refers  to a symbolic link the link is renamed; if
>        newpath refers to a symbolic link the link will be overwritten.
> 
> But if this is linux specific, and we want to support other operating
> system kernels than linux, then this might be an issue.

Oh, I was being stupid. Of course this isn't Linux-specific. See
http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html

So if you want the unversioned name to be a symlink to the versioned
one, then what you do is:

symlink("/lib/musl.so.1.0.1", "/lib/tmp12345");
rename("/lib/tmp12345", "/lib/ld-musl-i386.so.1");

From a shell script, however, it seems you need to explicitly perform
these operations with ln and mv. The "ln -sf" command is specified to
operate via unlink, rather than renaming a temporary link, so it has
the race condition.

This gives us some more flexibility in how the installation and links
work.

Rich