lfs64 api removal

lfs64 api removal

[Rich Felker]

One roadmap item I had down for this release cycle, whose practicality
at this point I need to assess, is removal of the lfs64 stuff at the
API level. The intent is that all the macros defining "64"-suffixed
alternate names for interfaces and types involving off_t would be
removed, but the symbols would still be present for linking only
(possibly dynamic linking only). These macros tend to break C++ stuff
since GCC's default _GNU_SOURCE in C++ mode exposes them all, and
unlike glibc's definitions, musl's are at the preprocessor level where
they can't be namespaced or shadowed. And they're just ugly, useless,
and possibly misleading.

Ideally we could just remove all of this from the public headers, but
at least at one point in the past, lots of software used broken
configure tests which were link-only to infer their existence, then
without the corresponding declarations or macros in the public
headers, produced implicit function definitions using them with
horribly wrong behavior as a result. As I recall it, this was the
motivation for adding them to begin with.

If that's still an issue, removing the API while keeping the ABI
requires some mechanism to make them unavailable at ld-time but
available at ldso time. The easiest way seems to be replacing

weak_alias(foo,foo64);

with something like:

__asm__(".symver foo,foo64@");

which produces a non-default empty-versioned symbol. This seems to
also require -Wl,--default-symver to work. ("Seems", because nothing
connected to symbol versioning hell is actually documented.) It also
depends on ldso ignoring non-default symbol versions and only
resolving to the default version, which probably isn't done right now
because we don't use versions for libc/ldso itself at all. So while
the source-level change is simple, there's some real ugliness here
that might interact badly with other things in the future.

The other simple way is also a hack: have ldso recheck failed lookups
just in libc, with any 64 removed, before committing to failure.

Further alternatives are making a fake virtual dso to hook
glibc-linked (DT_NEEDED:libc.so.6) apps/libs up to and providing the
lfs64 symbols from there, or dropping them entirely from musl, moving
them to gcompat, and instead adding a feature for auto-loading gcompat
for glibc apps/libs. (These approaches involve more detail I haven't
gone into here.)

Ideally, I'd be really happy to find out that just removing the cruft
from the headers suffices, so that we can consider getting rid of the
linkable symbols as s completely separate matter later if desired
rather than having to do them together.

Rich

Re: lfs64 api removal

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 3579 bytes --]

On 29/10/2019 10:20, Rich Felker wrote:
> One roadmap item I had down for this release cycle, whose practicality
> at this point I need to assess, is removal of the lfs64 stuff at the
> API level. The intent is that all the macros defining "64"-suffixed
> alternate names for interfaces and types involving off_t would be
> removed, but the symbols would still be present for linking only
> (possibly dynamic linking only). These macros tend to break C++ stuff
> since GCC's default _GNU_SOURCE in C++ mode exposes them all, and
> unlike glibc's definitions, musl's are at the preprocessor level where
> they can't be namespaced or shadowed. And they're just ugly, useless,
> and possibly misleading.
> 
> Ideally we could just remove all of this from the public headers, but
> at least at one point in the past, lots of software used broken
> configure tests which were link-only to infer their existence, then
> without the corresponding declarations or macros in the public
> headers, produced implicit function definitions using them with
> horribly wrong behavior as a result. As I recall it, this was the
> motivation for adding them to begin with.
> 
> If that's still an issue, removing the API while keeping the ABI
> requires some mechanism to make them unavailable at ld-time but
> available at ldso time. The easiest way seems to be replacing
> 
> weak_alias(foo,foo64);
> 
> with something like:
> 
> __asm__(".symver foo,foo64@");
> 
> which produces a non-default empty-versioned symbol. This seems to
> also require -Wl,--default-symver to work. ("Seems", because nothing
> connected to symbol versioning hell is actually documented.) It also
> depends on ldso ignoring non-default symbol versions and only
> resolving to the default version, which probably isn't done right now
> because we don't use versions for libc/ldso itself at all. So while
> the source-level change is simple, there's some real ugliness here
> that might interact badly with other things in the future.
> 
> The other simple way is also a hack: have ldso recheck failed lookups
> just in libc, with any 64 removed, before committing to failure.
> 
> Further alternatives are making a fake virtual dso to hook
> glibc-linked (DT_NEEDED:libc.so.6) apps/libs up to and providing the
> lfs64 symbols from there, or dropping them entirely from musl, moving
> them to gcompat, and instead adding a feature for auto-loading gcompat
> for glibc apps/libs. (These approaches involve more detail I haven't
> gone into here.)
> 
> Ideally, I'd be really happy to find out that just removing the cruft
> from the headers suffices, so that we can consider getting rid of the
> linkable symbols as s completely separate matter later if desired
> rather than having to do them together.
> 
> Rich


Sounds like this may be worth a try-build, like what we are going to do
for the time64 changes on PPC32 and ARMv7.

If you want to write a patch that completely removes the LFS64 API, I
can toss it to one of our ultra-fast builders (ppc64 probably) and see
where it goes.  (Likely after the time64 try-build, if that's acceptable.)

If it is possible to remove them entirely from musl, then wearing my
gcompat maintainer hat, I'd be happy to work with the musl community to
add whatever support code would be necessary from our end to make it
work for providing the LFS ABI to musl.  That was actually the original
design goal...

Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: lfs64 api removal

[Rich Felker]

On Tue, Oct 29, 2019 at 10:43:54AM -0500, A. Wilcox wrote:
> On 29/10/2019 10:20, Rich Felker wrote:
> > One roadmap item I had down for this release cycle, whose practicality
> > at this point I need to assess, is removal of the lfs64 stuff at the
> > API level. The intent is that all the macros defining "64"-suffixed
> > alternate names for interfaces and types involving off_t would be
> > removed, but the symbols would still be present for linking only
> > (possibly dynamic linking only). These macros tend to break C++ stuff
> > since GCC's default _GNU_SOURCE in C++ mode exposes them all, and
> > unlike glibc's definitions, musl's are at the preprocessor level where
> > they can't be namespaced or shadowed. And they're just ugly, useless,
> > and possibly misleading.
> > 
> > Ideally we could just remove all of this from the public headers, but
> > at least at one point in the past, lots of software used broken
> > configure tests which were link-only to infer their existence, then
> > without the corresponding declarations or macros in the public
> > headers, produced implicit function definitions using them with
> > horribly wrong behavior as a result. As I recall it, this was the
> > motivation for adding them to begin with.
> > 
> > If that's still an issue, removing the API while keeping the ABI
> > requires some mechanism to make them unavailable at ld-time but
> > available at ldso time. The easiest way seems to be replacing
> > 
> > weak_alias(foo,foo64);
> > 
> > with something like:
> > 
> > __asm__(".symver foo,foo64@");
> > 
> > which produces a non-default empty-versioned symbol. This seems to
> > also require -Wl,--default-symver to work. ("Seems", because nothing
> > connected to symbol versioning hell is actually documented.) It also
> > depends on ldso ignoring non-default symbol versions and only
> > resolving to the default version, which probably isn't done right now
> > because we don't use versions for libc/ldso itself at all. So while
> > the source-level change is simple, there's some real ugliness here
> > that might interact badly with other things in the future.
> > 
> > The other simple way is also a hack: have ldso recheck failed lookups
> > just in libc, with any 64 removed, before committing to failure.
> > 
> > Further alternatives are making a fake virtual dso to hook
> > glibc-linked (DT_NEEDED:libc.so.6) apps/libs up to and providing the
> > lfs64 symbols from there, or dropping them entirely from musl, moving
> > them to gcompat, and instead adding a feature for auto-loading gcompat
> > for glibc apps/libs. (These approaches involve more detail I haven't
> > gone into here.)
> > 
> > Ideally, I'd be really happy to find out that just removing the cruft
> > from the headers suffices, so that we can consider getting rid of the
> > linkable symbols as s completely separate matter later if desired
> > rather than having to do them together.
> 
> Sounds like this may be worth a try-build, like what we are going to do
> for the time64 changes on PPC32 and ARMv7.
> 
> If you want to write a patch that completely removes the LFS64 API, I
> can toss it to one of our ultra-fast builders (ppc64 probably) and see
> where it goes.  (Likely after the time64 try-build, if that's acceptable.)

Yes, that's roughly what I was just now thinking of proposing. I'll
probably do it after merging/pushing time64 just so as not to have
patch commutators to deal with.

> If it is possible to remove them entirely from musl, then wearing my
> gcompat maintainer hat, I'd be happy to work with the musl community to
> add whatever support code would be necessary from our end to make it
> work for providing the LFS ABI to musl.  That was actually the original
> design goal...

Yes, and I think that should still be the goal. I'd just like to see
if we can factor "API-level removal" and "move of ABI compat from libc
to libgcompat" as separate things. I think it would be just writing
thin wrapper functions (in place of the aliases, which cheated and
avoided needing any code at all) for all the functions in musl that
had 64-suffixed aliases.

Rich