Revisiting 64-bit time_t

Revisiting 64-bit time_t

[Rich Felker]

I've been thinking on and off a lot more about the time_t problem on
32-bit archs. My original idea for fixing this has always been to
introduce the ".2 ABI", fixing a lot of poorly chosen struct layouts,
etc. at the same time we make time_t 64-bit, but of course requiring
users/distros to make an active choice to switch over ABI at some
point, and not getting any benefit until then.

The idea has been that users (like embedded) who don't care
much/at-all about an ecosystem of ABI-compatible binaries, but build
everything from source with buildroot or yocto or whatever, would
switch right away so that their devices don't become Y2038 time bombs,
and desktop/server distros that receive constant updates could make
the transition at their leisure.

However Y2038 is not all that far off, desktop/server distros really
have rather little interest left in 32-bit archs (especially not
coordinating a costly ABI swap just for them), and some of the
extensibility improvements we'd get from a ".2 ABI" would be just as
desirable or more desirable on 64-bit archs, which don't even have the
time_t motivation to do it now.

So I'm thinking more and more about doing a different fix. In a way
it's like how glibc did 64-bit off_t, and how they're doing 64-bit
time_t, except it wouldn't be switchable and wouldn't default to the
old behavior; once we pull the lever, everything would be built with
64-bit time_t. This would work via symbol redirction in the headers
for the affected functions (probably via a bits header for the 32-bit
archs), which is valid because, by virtue of using time_t or a derived
type, the standard requires that you include the headers to get the
declaration rather than declaring the function yourself.

Doing it this way does not break application-to-libc ABI, because the
old symbols still exist; they're just not used for linking new
programs. It does however impact ABI between libraries outside libc if
they use time_t or any of the derived types (timespec, stat, ...) in
their public (not internal, only public) APIs. How big that impact
would be is an open question; it might mean this approach would
require some coordinated updating of affected libraries and
applications using them in sync to prevent breakage.

Aside from community feedback, what's needed to make this possible, if
it's going to happen, is some good analysis of the scope of breakage.
Such analysis would also benefit glibc -- it would help determine how
safe their _TIME_BITS=64 option will be and whether it can be turned
on safely by default in the presence of old libraries built without
it. I've already discussed this casually with a few people and it
looks like the right starting point would be getting a Debian system
(Debian because their repo is utterly huge) with ALL library packages
installed and grepping /usr/include for all headers that involve
time_t or any of the derived types. Then, manual analysis would need
to be done to determine whether the usage actually has an impact.

If there are a significant number of affected libraries and we want to
go forward with something like this anyway, there should probably be
an optional patch distros can use to make ldso refuse to load certain
tagged .so files into a process where any of the 64-bit time symbols
have been referenced. This would ensure transitioning users get an
error message rather than silent misexecution.

Rich

Re: Revisiting 64-bit time_t

[A. Wilcox]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

Overall this seems like a decent proposal. However...

On Jun 28, 2019, at 10:06 AM, Rich Felker <dalias@libc.org> wrote:
> However Y2038 is not all that far off, desktop/server distros really
> have rather little interest left in 32-bit archs (especially not
> coordinating a costly ABI swap just for them)


This is really incorrect.  We at Adélie are QUITE interested in 32-bit architectures including:

* 32-bit PowerPC
Still heavily used by people with Power Macs in addition to some people who run 32-bit userland on 64-bit POWER systems.

* 32-bit ARM
Widely, widely used in SoCs the world over, including the previous generation Raspberry Pi, the Pi Zero, many ODROIDs, and a great number of current and previous generation smartphones.

* 32-bit Intel
Of course you have the people with 32-bit binaries including Steam games (which would need something like gcompat to run on musl systems anyway), but you also have the people who cannot for whatever reason upgrade their hardware to a functional 64-bit system.  I personally know at least three people that are disadvantaged and still use a Pentium 4 era system as their only computer.


It’s not just us. The people at postmarketOS care very much about 32-bit ARM, since that is the majority of the hardware they are targeting. You also have Void Linux which supports all three of the arches I listed above.


Please, please do not write off 32-bit desktop usage.  Ubuntu does *not* speak for the wider community nor for the majority of Linux users.  And we, the distros named above, all hope to be able to support the people Ubuntu are wanting to abandon for the foreseeable future.


Thank you for researching this issue, for writing this proposal, and for all that you do in musl.

Best to you and yours,
—arw 

--
A. Wilcox (Sent from my iPhone - not signed)
Project Lead, Adélie Linux
https://adelielinux.org

[-- Attachment #2: Type: text/html, Size: 3049 bytes --]

Re: Revisiting 64-bit time_t

[Rich Felker]

On Sat, Jun 29, 2019 at 03:36:19AM -0500, A. Wilcox wrote:
> Overall this seems like a decent proposal. However...
> 
> On Jun 28, 2019, at 10:06 AM, Rich Felker <dalias@libc.org> wrote:
> > However Y2038 is not all that far off, desktop/server distros really
> > have rather little interest left in 32-bit archs (especially not
> > coordinating a costly ABI swap just for them)
> 
> This is really incorrect. We at Adélie are QUITE interested in
> 32-bit architectures including:
> 
> [...]
> 
> Please, please do not write off 32-bit desktop usage.

I'm sorry my wording contributed to a narrative that 32-bit is dead;
that's not at all my intent, but I can see how it could be harmful to
efforts to maintain support.

My intent here is the other direction -- due to dominance of 64-bit
archs on desktop and server these days, there's much less effort being
put into the future of 32-bit ones, and I don't want to make a
decision here that would incentivize distros that don't already care
strongly about keeping 32-bit arch support to just drop it, rather
than going through a painful ABI swap-out.

Thanks for your work continuing to press applications not to break
these archs.

Rich

Re: Revisiting 64-bit time_t

[Rich Felker]

On Fri, Jun 28, 2019 at 11:06:59AM -0400, Rich Felker wrote:
> [...]
> So I'm thinking more and more about doing a different fix. In a way
> it's like how glibc did 64-bit off_t, and how they're doing 64-bit
> time_t, except it wouldn't be switchable and wouldn't default to the
> old behavior; once we pull the lever, everything would be built with
> 64-bit time_t. This would work via symbol redirction in the headers
> for the affected functions (probably via a bits header for the 32-bit
> archs), which is valid because, by virtue of using time_t or a derived
> type, the standard requires that you include the headers to get the
> declaration rather than declaring the function yourself.
> 
> Doing it this way does not break application-to-libc ABI, because the
> old symbols still exist; they're just not used for linking new
> programs. It does however impact ABI between libraries outside libc if
> they use time_t or any of the derived types (timespec, stat, ...) in
> their public (not internal, only public) APIs. How big that impact
> would be is an open question; it might mean this approach would
> require some coordinated updating of affected libraries and
> applications using them in sync to prevent breakage.
> 
> Aside from community feedback, what's needed to make this possible, if
> it's going to happen, is some good analysis of the scope of breakage.
> Such analysis would also benefit glibc -- it would help determine how
> safe their _TIME_BITS=64 option will be and whether it can be turned
> on safely by default in the presence of old libraries built without
> it. I've already discussed this casually with a few people and it
> looks like the right starting point would be getting a Debian system
> (Debian because their repo is utterly huge) with ALL library packages
> installed and grepping /usr/include for all headers that involve
> time_t or any of the derived types. Then, manual analysis would need
> to be done to determine whether the usage actually has an impact.
> 
> If there are a significant number of affected libraries and we want to
> go forward with something like this anyway, there should probably be
> an optional patch distros can use to make ldso refuse to load certain
> tagged .so files into a process where any of the 64-bit time symbols
> have been referenced. This would ensure transitioning users get an
> error message rather than silent misexecution.

A couple observations:

1. dlsym is problematic.

You can't have a correct implementation where dlsym("time") returns
the old (now-signature-mismatched) time symbol rather than the
redirected one.

I think this just necessitates having affected archs also redirect the
dlsym symbol, to call a patch-up wrapper. Moderately ugly, but not a
show-stopper.

2. The glibc ABI-compat __statx stuff.

I'm not sure if glibc is using these for its time64 versions of the
interfaces, or if it finally abandoned that broken framework (that
beaks both dlsym and function pointer equality). If they're still
using it and we want to maintain support for ABI-compat, we need to
update these to handle the time64 versions too.

Rich

Re: Revisiting 64-bit time_t

[Arnd Bergmann]

On Sat, Jun 29, 2019 at 6:21 PM Rich Felker <dalias@libc.org> wrote:
> On Sat, Jun 29, 2019 at 03:36:19AM -0500, A. Wilcox wrote:
> > Overall this seems like a decent proposal. However...
> >
> > On Jun 28, 2019, at 10:06 AM, Rich Felker <dalias@libc.org> wrote:
> > > However Y2038 is not all that far off, desktop/server distros really
> > > have rather little interest left in 32-bit archs (especially not
> > > coordinating a costly ABI swap just for them)
> >
> > This is really incorrect. We at Adélie are QUITE interested in
> > 32-bit architectures including:
> >
> > Please, please do not write off 32-bit desktop usage.
>
> I'm sorry my wording contributed to a narrative that 32-bit is dead;
> that's not at all my intent, but I can see how it could be harmful to
> efforts to maintain support.
>
> My intent here is the other direction -- due to dominance of 64-bit
> archs on desktop and server these days, there's much less effort being
> put into the future of 32-bit ones, and I don't want to make a
> decision here that would incentivize distros that don't already care
> strongly about keeping 32-bit arch support to just drop it, rather
> than going through a painful ABI swap-out.
>
> Thanks for your work continuing to press applications not to break
> these archs.

I think there are three valid ways for distros to handle the time64
conversion for 32-bit targets:

a) Decide to never convert but keep the time32 until *all* users have
   stopped using 32-bit user space altogether (i.e. well before 2038).
   Advantage: no ABI break at all.
   Disadvantage: guaranteed to break in 2038, but likely earlier
   because of long timeouts like key expiration times.

b) System wide rebuild (bootstrap) against newer libc with time64.
   Advantage: No surprising ABI break
   Disadvantage: requires reinstall instead of all user space,
   breaks all third-party and custom binaries

c) Keep backwards compatibility in libraries, but convert the
   distro one package at a time.
   Advantage: If done right, users can upgrade over rolling
   releases without ABIs breaking
   Disadvantage: very hard to get right, and much more work
   than the other two.

It might be good to get feedback from distros using musl about
what model they each prefer. If everyone wants one of the first
two approaches anyway, that would simplify the implementation
in musl significantly. I assume that source based  embedded
distros (e.g. openembedded, openwrt, possibly gentoo) are
usually happy with b), while most of the glibc based desktop
distros (Ubuntu, Red Hat, SUSE, ...) are ok with just dropping
32-bit support in the long run as in a) and would do that for
unrelated reasons. Debian will probably want c), but might also
be convinced to replace the existing 32-bit ports with musl based
ones to avoid the effort and reduce RAM usage at the same
time ;-)

Where would Adélie, Alpine and others using musl fit?

        Arnd

Re: Revisiting 64-bit time_t

[Arnd Bergmann]

On Fri, Jun 28, 2019 at 5:07 PM Rich Felker <dalias@libc.org> wrote:
>
> The idea has been that users (like embedded) who don't care
> much/at-all about an ecosystem of ABI-compatible binaries, but build
> everything from source with buildroot or yocto or whatever, would
> switch right away so that their devices don't become Y2038 time bombs,
> and desktop/server distros that receive constant updates could make
> the transition at their leisure.

Distros would probably need a varying amount of time to transition,
right? Would you plan to support both time32 and time64 for a
transition period, or would a distro that is not yet confident in rebuilding
everything with time64 be stuck on the last time32 musl release
before they do?

I suppose the header files could be changed in a musl-1.2 release
if the times line up, while musl-1.1.x can still get bugfixes?

> So I'm thinking more and more about doing a different fix. In a way
> it's like how glibc did 64-bit off_t, and how they're doing 64-bit
> time_t, except it wouldn't be switchable and wouldn't default to the
> old behavior; once we pull the lever, everything would be built with
> 64-bit time_t. This would work via symbol redirction in the headers
> for the affected functions (probably via a bits header for the 32-bit
> archs), which is valid because, by virtue of using time_t or a derived
> type, the standard requires that you include the headers to get the
> declaration rather than declaring the function yourself.

Sound great to me. I don't think it would be hard to make it
conditional on top of that (just have an #ifdef __USE_TIME_BITS64
around each symbol redirect in the headers), but I also prefer the idea
that this is not something an individual compilation unit gets
to decide.

> Aside from community feedback, what's needed to make this possible, if
> it's going to happen, is some good analysis of the scope of breakage.
> Such analysis would also benefit glibc -- it would help determine how
> safe their _TIME_BITS=64 option will be and whether it can be turned
> on safely by default in the presence of old libraries built without
> it. I've already discussed this casually with a few people and it
> looks like the right starting point would be getting a Debian system
> (Debian because their repo is utterly huge) with ALL library packages
> installed and grepping /usr/include for all headers that involve
> time_t or any of the derived types. Then, manual analysis would need
> to be done to determine whether the usage actually has an impact.

Yes, this is also one of the things we eventually plan to do in Linaro,
but have not actually started.

> If there are a significant number of affected libraries and we want to
> go forward with something like this anyway, there should probably be
> an optional patch distros can use to make ldso refuse to load certain
> tagged .so files into a process where any of the 64-bit time symbols
> have been referenced. This would ensure transitioning users get an
> error message rather than silent misexecution.

For those distros that build everything from source and generally
don't update packages independently, another idea would
be to have a way to leave out all the time32 symbols. This would
immediately guarantee that they are not mixed.

      Arnd

Re: Revisiting 64-bit time_t

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 2953 bytes --]

On 07/01/19 09:41, Arnd Bergmann wrote:
> On Sat, Jun 29, 2019 at 6:21 PM Rich Felker <dalias@libc.org> wrote:
>> On Sat, Jun 29, 2019 at 03:36:19AM -0500, A. Wilcox wrote:
>>> On Jun 28, 2019, at 10:06 AM, Rich Felker <dalias@libc.org> wrote:
>>>> However Y2038 is not all that far off, desktop/server distros really
>>>> have rather little interest left in 32-bit archs (especially not
>>>> coordinating a costly ABI swap just for them)
>>> Please, please do not write off 32-bit desktop usage.
>> I'm sorry my wording contributed to a narrative that 32-bit is dead;
>> that's not at all my intent, but I can see how it could be harmful to
>> efforts to maintain support.
>>
>> My intent here is the other direction -- due to dominance of 64-bit
>> archs on desktop and server these days, there's much less effort being
>> put into the future of 32-bit ones, and I don't want to make a
>> decision here that would incentivize distros that don't already care
>> strongly about keeping 32-bit arch support to just drop it, rather
>> than going through a painful ABI swap-out.
>>
>> Thanks for your work continuing to press applications not to break
>> these archs.
> 
> I think there are three valid ways for distros to handle the time64
> conversion for 32-bit targets:
> 
> a) Decide to never convert but keep the time32 until *all* users have
>    stopped using 32-bit user space altogether (i.e. well before 2038).
>    Advantage: no ABI break at all.
>    Disadvantage: guaranteed to break in 2038, but likely earlier
>    because of long timeouts like key expiration times.
> 
> b) System wide rebuild (bootstrap) against newer libc with time64.
>    Advantage: No surprising ABI break
>    Disadvantage: requires reinstall instead of all user space,
>    breaks all third-party and custom binaries
> 
> c) Keep backwards compatibility in libraries, but convert the
>    distro one package at a time.
>    Advantage: If done right, users can upgrade over rolling
>    releases without ABIs breaking
>    Disadvantage: very hard to get right, and much more work
>    than the other two.
> 
> 
> Where would Adélie, Alpine and others using musl fit?
> 
>         Arnd
> 


Adélie rebuilds all packages for every release, and encourages users to
use `apk upgrade -al` which will /replace/ all of world for every
upgrade, so we'd be very happy with b).  Make everything as correct as
possible, as quickly as possible, with just a simple upgrade for users
(as long as they have no self-compiled software installed).

There is already no real "third-party" binary for 32-bit musl computers,
so I'm not sure if that's relevant.  For glibc binaries, we have gcompat
which could easily "shadow" time_t functions with 32-bit versions for as
long as glibc binaries continue to have a need.


Best,
--arw

-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Revisiting 64-bit time_t

[Rich Felker]

On Mon, Jul 01, 2019 at 04:42:29PM +0200, Arnd Bergmann wrote:
> On Fri, Jun 28, 2019 at 5:07 PM Rich Felker <dalias@libc.org> wrote:
> >
> > The idea has been that users (like embedded) who don't care
> > much/at-all about an ecosystem of ABI-compatible binaries, but build
> > everything from source with buildroot or yocto or whatever, would
> > switch right away so that their devices don't become Y2038 time bombs,
> > and desktop/server distros that receive constant updates could make
> > the transition at their leisure.
> 
> Distros would probably need a varying amount of time to transition,
> right? Would you plan to support both time32 and time64 for a
> transition period, or would a distro that is not yet confident in rebuilding
> everything with time64 be stuck on the last time32 musl release
> before they do?
> 
> I suppose the header files could be changed in a musl-1.2 release
> if the times line up, while musl-1.1.x can still get bugfixes?

".2" ABI doesn't mean "musl 1.2". It means ld-musl-*.so.2". If taking
this path, the plan was always to support both permanently (modulo
dropping .1 after 2038 ;) -- they'd just be different ABI targets.

> > So I'm thinking more and more about doing a different fix. In a way
> > it's like how glibc did 64-bit off_t, and how they're doing 64-bit
> > time_t, except it wouldn't be switchable and wouldn't default to the
> > old behavior; once we pull the lever, everything would be built with
> > 64-bit time_t. This would work via symbol redirction in the headers
> > for the affected functions (probably via a bits header for the 32-bit
> > archs), which is valid because, by virtue of using time_t or a derived
> > type, the standard requires that you include the headers to get the
> > declaration rather than declaring the function yourself.
> 
> Sound great to me. I don't think it would be hard to make it
> conditional on top of that (just have an #ifdef __USE_TIME_BITS64
> around each symbol redirect in the headers), but I also prefer the idea
> that this is not something an individual compilation unit gets
> to decide.

Right. Unless we took it further and completely removed the old
symbols, preventing any new linking and making them only work via
hacks at the dynamic linker level, you absolutely could still link to
the old symbols just by using headers from an old version of musl, or
hacking the headers. But I'd rather not treat this as a supported
option. Due to the additional, hidden ABI-surface between apps and
third party libs, or between pairs of third-party libs, there's enough
risk of getting this wrong even with a single-direction, one-time
change. Having folks continue to compile individual apps and libs
inconsistently would make it even worse, like what happened with glibc
_FILE_OFFSET_BITS.

> > Aside from community feedback, what's needed to make this possible, if
> > it's going to happen, is some good analysis of the scope of breakage.
> > Such analysis would also benefit glibc -- it would help determine how
> > safe their _TIME_BITS=64 option will be and whether it can be turned
> > on safely by default in the presence of old libraries built without
> > it. I've already discussed this casually with a few people and it
> > looks like the right starting point would be getting a Debian system
> > (Debian because their repo is utterly huge) with ALL library packages
> > installed and grepping /usr/include for all headers that involve
> > time_t or any of the derived types. Then, manual analysis would need
> > to be done to determine whether the usage actually has an impact.
> 
> Yes, this is also one of the things we eventually plan to do in Linaro,
> but have not actually started.

Would it be possible to prioritize starting this? It would be a big
help to deciding what direction we should take in musl and make this
move forward a lot quicker, I think. I was thinking we'd have to do it
ourselves or find someone else to convince to do it, but if Linaro
already plans to do this anyway, we could perhaps accelerate things
with no overall increase in effort to be spent.

> > If there are a significant number of affected libraries and we want to
> > go forward with something like this anyway, there should probably be
> > an optional patch distros can use to make ldso refuse to load certain
> > tagged .so files into a process where any of the 64-bit time symbols
> > have been referenced. This would ensure transitioning users get an
> > error message rather than silent misexecution.
> 
> For those distros that build everything from source and generally
> don't update packages independently, another idea would
> be to have a way to leave out all the time32 symbols. This would
> immediately guarantee that they are not mixed.

Well if you just have the new headers, there's not really any way they
could get mixed except by failing to include the headers and declaring
the symbols yourself -- but then, for almost all cases, you'd be
missing the types needed to declare them, so it'd have to be something
like "long time(long *);" in the source, which is just horribly
broken. So I'm not sure there are any real-world cases where
removing/poisoning the old symbols would help.

Rich

Re: Revisiting 64-bit time_t

[Rich Felker]

On Mon, Jul 01, 2019 at 09:50:31AM -0500, A. Wilcox wrote:
> On 07/01/19 09:41, Arnd Bergmann wrote:
> > On Sat, Jun 29, 2019 at 6:21 PM Rich Felker <dalias@libc.org> wrote:
> >> On Sat, Jun 29, 2019 at 03:36:19AM -0500, A. Wilcox wrote:
> >>> On Jun 28, 2019, at 10:06 AM, Rich Felker <dalias@libc.org> wrote:
> >>>> However Y2038 is not all that far off, desktop/server distros really
> >>>> have rather little interest left in 32-bit archs (especially not
> >>>> coordinating a costly ABI swap just for them)
> >>> Please, please do not write off 32-bit desktop usage.
> >> I'm sorry my wording contributed to a narrative that 32-bit is dead;
> >> that's not at all my intent, but I can see how it could be harmful to
> >> efforts to maintain support.
> >>
> >> My intent here is the other direction -- due to dominance of 64-bit
> >> archs on desktop and server these days, there's much less effort being
> >> put into the future of 32-bit ones, and I don't want to make a
> >> decision here that would incentivize distros that don't already care
> >> strongly about keeping 32-bit arch support to just drop it, rather
> >> than going through a painful ABI swap-out.
> >>
> >> Thanks for your work continuing to press applications not to break
> >> these archs.
> > 
> > I think there are three valid ways for distros to handle the time64
> > conversion for 32-bit targets:
> > 
> > a) Decide to never convert but keep the time32 until *all* users have
> >    stopped using 32-bit user space altogether (i.e. well before 2038).
> >    Advantage: no ABI break at all.
> >    Disadvantage: guaranteed to break in 2038, but likely earlier
> >    because of long timeouts like key expiration times.
> > 
> > b) System wide rebuild (bootstrap) against newer libc with time64.
> >    Advantage: No surprising ABI break
> >    Disadvantage: requires reinstall instead of all user space,
> >    breaks all third-party and custom binaries
> > 
> > c) Keep backwards compatibility in libraries, but convert the
> >    distro one package at a time.
> >    Advantage: If done right, users can upgrade over rolling
> >    releases without ABIs breaking
> >    Disadvantage: very hard to get right, and much more work
> >    than the other two.
> > 
> > 
> > Where would Adélie, Alpine and others using musl fit?
> 
> Adélie rebuilds all packages for every release, and encourages users to
> use `apk upgrade -al` which will /replace/ all of world for every
> upgrade, so we'd be very happy with b).  Make everything as correct as
> possible, as quickly as possible, with just a simple upgrade for users
> (as long as they have no self-compiled software installed).
> 
> There is already no real "third-party" binary for 32-bit musl computers,
> so I'm not sure if that's relevant.  For glibc binaries, we have gcompat
> which could easily "shadow" time_t functions with 32-bit versions for as
> long as glibc binaries continue to have a need.

I think this ignores the reality that a lot of users of any distro
except Debian pretty much *have to* build some software from source,
since there's so much more software out there than what you can
package without contributor bases and infrastructure that size. If you
break or switch ABI, you invalidate all the existing self-built
software a user has (unless they static-linked, but that might not be
an option if the software needs dlopen).

Admittedly distros are not doing a great job with this already. For
example when I updated Alpine, a bunch of my self-built software
stopped working because it uninstalled the old boost packages with the
matching API/ABI. I'm not sure how well Adélie fairs in this regard.
But in principle it's something that should be able to work. And this
is a large part of the motivation for why musl committed to stable
ABIs and of the motivation for musl rather than improving uclibc.

Rich

Re: Revisiting 64-bit time_t

[A. Wilcox]

[-- Attachment #1.1: Type: text/plain, Size: 3056 bytes --]

On 07/01/19 10:41, Rich Felker wrote:
> On Mon, Jul 01, 2019 at 09:50:31AM -0500, A. Wilcox wrote:
>> Adélie rebuilds all packages for every release, and encourages users to
>> use `apk upgrade -al` which will /replace/ all of world for every
>> upgrade, so we'd be very happy with b).  Make everything as correct as
>> possible, as quickly as possible, with just a simple upgrade for users
>> (as long as they have no self-compiled software installed).
>>
>> There is already no real "third-party" binary for 32-bit musl computers,
>> so I'm not sure if that's relevant.  For glibc binaries, we have gcompat
>> which could easily "shadow" time_t functions with 32-bit versions for as
>> long as glibc binaries continue to have a need.
> 
> I think this ignores the reality that a lot of users of any distro
> except Debian pretty much *have to* build some software from source,
> since there's so much more software out there than what you can
> package without contributor bases and infrastructure that size. If you
> break or switch ABI, you invalidate all the existing self-built
> software a user has (unless they static-linked, but that might not be
> an option if the software needs dlopen).
> 
> Admittedly distros are not doing a great job with this already. For
> example when I updated Alpine, a bunch of my self-built software
> stopped working because it uninstalled the old boost packages with the
> matching API/ABI. I'm not sure how well Adélie fairs in this regard.
> But in principle it's something that should be able to work. And this
> is a large part of the motivation for why musl committed to stable
> ABIs and of the motivation for musl rather than improving uclibc.
> 
> Rich


I specifically did mention "as long as they have no self-compiled
software installed".

At Adélie we have policies that we won't ever sobump things like Boost,
Qt, etc except between major versions.  So Adélie 1.x, for all three
years, through 1.9, will always have boost 1.69.0 and Qt 5.9 etc.
Smaller libraries may or may not sobump; we consider addition of APIs
fine as long as no changes or removals were made (so ABI remains
"constant" for existing apps).

Ah yes, ffmpeg, that was the other big one we decided to be careful
about in minor releases.

The only reason we're still bumping right now is because we're still in
beta and haven't released 1.0 yet, so we haven't committed to the 1.0
ABI.  And that is primarily because we don't have an installer, though
all the massive CVEs in major packages have been eating up a lot of our
time...

So, if musl were to do this, we wouldn't have the "new musl" available
until Adélie 2.0, which will be 18 months after 1.0 is released.  But,
at the time you upgrade 1.x -> 2.0, you are already aware you are
changing ABI for practically every library on the system, so you should
be fully aware of that ahead of time.

Best to you and yours,
--arw


-- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
https://www.adelielinux.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Revisiting 64-bit time_t

[Florian Weimer]

* Arnd Bergmann:

> c) Keep backwards compatibility in libraries, but convert the
>    distro one package at a time.
>    Advantage: If done right, users can upgrade over rolling
>    releases without ABIs breaking
>    Disadvantage: very hard to get right, and much more work
>    than the other two.

ctypes and other FFIs do not use sonames or symbol versioning.  Instead
they encourage that their users use the familiar, unversioned library
names.  So the chance of doing this right seem pretty slim.

Thanks,
Florian

Re: Revisiting 64-bit time_t

[Rich Felker]

On Mon, Jul 01, 2019 at 04:41:51PM +0200, Arnd Bergmann wrote:
> c) Keep backwards compatibility in libraries, but convert the
>    distro one package at a time.
>    Advantage: If done right, users can upgrade over rolling
>    releases without ABIs breaking
>    Disadvantage: very hard to get right, and much more work
>    than the other two.

I'm confused how "convert the distro one package at a time" comes up.
The only packages potentially affected are ones that provide or
consume a non-libc API with time_t or derived types as inputs or
outputs. Such providers and consumers would have to be updated in sync
with each other, but otherwise there should be no constraints about
updating packages. I think this could be implemented with distros'
standard "depends on" and "conflicts with" mechanics.

> unrelated reasons. Debian will probably want c), but might also
> be convinced to replace the existing 32-bit ports with musl based
> ones to avoid the effort and reduce RAM usage at the same
> time ;-)

Replacing libc is like the polar opposite of option (c) here... :-)

Rich

Re: Revisiting 64-bit time_t

[Arnd Bergmann]

On Mon, Jul 1, 2019 at 5:31 PM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Jul 01, 2019 at 04:42:29PM +0200, Arnd Bergmann wrote:
> > On Fri, Jun 28, 2019 at 5:07 PM Rich Felker <dalias@libc.org> wrote:
> > >
> > > The idea has been that users (like embedded) who don't care
> > > much/at-all about an ecosystem of ABI-compatible binaries, but build
> > > everything from source with buildroot or yocto or whatever, would
> > > switch right away so that their devices don't become Y2038 time bombs,
> > > and desktop/server distros that receive constant updates could make
> > > the transition at their leisure.
> >
> > Distros would probably need a varying amount of time to transition,
> > right? Would you plan to support both time32 and time64 for a
> > transition period, or would a distro that is not yet confident in rebuilding
> > everything with time64 be stuck on the last time32 musl release
> > before they do?
> >
> > I suppose the header files could be changed in a musl-1.2 release
> > if the times line up, while musl-1.1.x can still get bugfixes?
>
> ".2" ABI doesn't mean "musl 1.2". It means ld-musl-*.so.2". If taking
> this path, the plan was always to support both permanently (modulo
> dropping .1 after 2038 ;) -- they'd just be different ABI targets.

Yes, I understand that was a different idea. What I was asking about
here is how long there would be support for building with the time32
interfaces against a maintained libc after it has become possible to
build with time64 when using the same soname.


> > > Aside from community feedback, what's needed to make this possible, if
> > > it's going to happen, is some good analysis of the scope of breakage.
> > > Such analysis would also benefit glibc -- it would help determine how
> > > safe their _TIME_BITS=64 option will be and whether it can be turned
> > > on safely by default in the presence of old libraries built without
> > > it. I've already discussed this casually with a few people and it
> > > looks like the right starting point would be getting a Debian system
> > > (Debian because their repo is utterly huge) with ALL library packages
> > > installed and grepping /usr/include for all headers that involve
> > > time_t or any of the derived types. Then, manual analysis would need
> > > to be done to determine whether the usage actually has an impact.
> >
> > Yes, this is also one of the things we eventually plan to do in Linaro,
> > but have not actually started.
>
> Would it be possible to prioritize starting this? It would be a big
> help to deciding what direction we should take in musl and make this
> move forward a lot quicker, I think. I was thinking we'd have to do it
> ourselves or find someone else to convince to do it, but if Linaro
> already plans to do this anyway, we could perhaps accelerate things
> with no overall increase in effort to be spent.

I've had a first look here now, with a scripted search in /usr/include
for all packages in debian testing/main:

apt-file search /usr/include | cut -f 1 -d: | uniq |
while read i ; do
    mkdir -p ${i}
    cd ${i}
    if [ ! -e "${i}_.*.deb" ] ; then
        apt-get download ${i}
    fi
    FILE=${i}_*.deb
    dpkg-deb --fsys-tarfile ${FILE} | tar xf - ./usr/include
    cd ..
    ctags -f - -R ${i}/ | grep
'\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>' >
${i}.tags
    grep -r '\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>'
${i}/ > ${i}.files
    rm -rf ./${i}
done

There are 4288 packages that provide a file in /usr/include, and out of those,
973 match the regular expression above, see the list at

https://pastebin.com/Yu22pLqQ

This took a few hours to run and could be done faster by running bits
in parallel.
I can send you the full output, but it's at 300kb compressed, it's a bit large
for the mailing list. The regex also wasn't great, so I'm sure there
are lots of false positives and negatives, but its' a start.

> > > If there are a significant number of affected libraries and we want to
> > > go forward with something like this anyway, there should probably be
> > > an optional patch distros can use to make ldso refuse to load certain
> > > tagged .so files into a process where any of the 64-bit time symbols
> > > have been referenced. This would ensure transitioning users get an
> > > error message rather than silent misexecution.
> >
> > For those distros that build everything from source and generally
> > don't update packages independently, another idea would
> > be to have a way to leave out all the time32 symbols. This would
> > immediately guarantee that they are not mixed.
>
> Well if you just have the new headers, there's not really any way they
> could get mixed except by failing to include the headers and declaring
> the symbols yourself -- but then, for almost all cases, you'd be
> missing the types needed to declare them, so it'd have to be something
> like "long time(long *);" in the source, which is just horribly
> broken. So I'm not sure there are any real-world cases where
> removing/poisoning the old symbols would help.

I was thinking of the case where in theory everything should be built
from source, but in reality some binaries made it into system because
of sloppy procedures or lack of source code.

         Arnd

Re: Revisiting 64-bit time_t

[Rich Felker]

On Mon, Jul 01, 2019 at 11:12:20PM +0200, Arnd Bergmann wrote:
> > > > Aside from community feedback, what's needed to make this possible, if
> > > > it's going to happen, is some good analysis of the scope of breakage.
> > > > Such analysis would also benefit glibc -- it would help determine how
> > > > safe their _TIME_BITS=64 option will be and whether it can be turned
> > > > on safely by default in the presence of old libraries built without
> > > > it. I've already discussed this casually with a few people and it
> > > > looks like the right starting point would be getting a Debian system
> > > > (Debian because their repo is utterly huge) with ALL library packages
> > > > installed and grepping /usr/include for all headers that involve
> > > > time_t or any of the derived types. Then, manual analysis would need
> > > > to be done to determine whether the usage actually has an impact.
> > >
> > > Yes, this is also one of the things we eventually plan to do in Linaro,
> > > but have not actually started.
> >
> > Would it be possible to prioritize starting this? It would be a big
> > help to deciding what direction we should take in musl and make this
> > move forward a lot quicker, I think. I was thinking we'd have to do it
> > ourselves or find someone else to convince to do it, but if Linaro
> > already plans to do this anyway, we could perhaps accelerate things
> > with no overall increase in effort to be spent.
> 
> I've had a first look here now, with a scripted search in /usr/include
> for all packages in debian testing/main:
> 
> apt-file search /usr/include | cut -f 1 -d: | uniq |
> while read i ; do
>     mkdir -p ${i}
>     cd ${i}
>     if [ ! -e "${i}_.*.deb" ] ; then
>         apt-get download ${i}
>     fi
>     FILE=${i}_*.deb
>     dpkg-deb --fsys-tarfile ${FILE} | tar xf - ./usr/include
>     cd ..
>     ctags -f - -R ${i}/ | grep
> '\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>' >
> ${i}.tags
>     grep -r '\<time_t\|timespec\|timeval\|struct.stat\|struct.rusage\>'
> ${i}/ > ${i}.files
>     rm -rf ./${i}
> done
> 
> There are 4288 packages that provide a file in /usr/include, and out of those,
> 973 match the regular expression above, see the list at
> 
> https://pastebin.com/Yu22pLqQ
> 
> This took a few hours to run and could be done faster by running bits
> in parallel.
> I can send you the full output, but it's at 300kb compressed, it's a bit large
> for the mailing list. The regex also wasn't great, so I'm sure there
> are lots of false positives and negatives, but its' a start.

OK. That sounds like a lot, but upon looking, a lot of them look like
"libraries" whose main/sole consumer is a single application. So maybe
it's not so bad.

Rich

Re: Revisiting 64-bit time_t

[Arnd Bergmann]

On Tue, Jul 2, 2019 at 12:08 AM Rich Felker <dalias@libc.org> wrote:
> On Mon, Jul 01, 2019 at 11:12:20PM +0200, Arnd Bergmann wrote:
> >
> > There are 4288 packages that provide a file in /usr/include, and out of those,
> > 973 match the regular expression above, see the list at
> >
> > https://pastebin.com/Yu22pLqQ
> >
> > This took a few hours to run and could be done faster by running bits
> > in parallel.
> > I can send you the full output, but it's at 300kb compressed, it's a bit large
> > for the mailing list. The regex also wasn't great, so I'm sure there
> > are lots of false positives and negatives, but its' a start.
>
> OK. That sounds like a lot, but upon looking, a lot of them look like
> "libraries" whose main/sole consumer is a single application. So maybe
> it's not so bad.

I was trying to run 'build-rdeps' on each one to see how many packages
actually depend on them, but I could not get that working (I always
get an error "build-rdeps: unable to find sources files." but can't figure
out which file it is missing).

I would expect a distribution in which a few packages (e.g. libc) are
dependencies for thousands of others, while most packages are not
a dependency for anything else.

Looking only at direct built-dependencies instead, these are the packages
with the most dependencies:

while read i ; do echo `grep "Build-Depends.*\<$i\>"
/var/lib/apt/lists/httpredir.debian.org_debian_dists_testing_main_source_Sources
| wc -l` $i  ; done < all.list  | sort -nr | head -n 100

1026 libglib2.0-dev
848 qtbase5-dev
590 libgtk2.0-dev
583 libssl-dev
565 libgtk-3-dev
458 libpng-dev
315 libasound2-dev
307 libxt-dev
278 libreadline-dev
220 libcurl4-gnutls-dev
208 gawk
196 uuid-dev
174 libpam0g-dev
150 libgnutls28-dev
147 libpulse-dev
136 libpq-dev
129 libusb-1.0-0-dev
123 libqt4-dev
122 libcurl4-openssl-dev
118 libldap2-dev
110 libfontconfig1-dev
107 libicu-dev
101 libfuse-dev
100 libgstreamer1.0-dev
93 apache2-dev
89 libsoup2.4-dev
89 libgtest-dev
85 libpcap0.8-dev
79 qtbase5-private-dev
74 libevent-dev
73 libhdf5-dev
73 libeigen3-dev
66 libavutil-dev
65 libbsd-dev
59 libflac-dev
59 libarchive-dev
57 libglobus-common-dev
55 linux-libc-dev
54 libprotobuf-dev
51 libelf-dev
49 libgdal-dev
48 libcups2-dev
47 libdrm-dev
45 libmagic-dev
45 libgtkmm-2.4-dev
43 libmotif-dev
42 libv4l-dev
40 libkf5kdelibs4support-dev
38 libgeoip-dev
37 valgrind
36 nettle-dev
36 libfltk1.3-dev
33 libgtkmm-3.0-dev
33 dejagnu
32 kdelibs5-dev
31 tcl8.6-dev
30 libsnmp-dev
30 doctest-dev
29 libglibmm-2.4-dev
29 libapr1-dev
29 fcitx-libs-dev
28 libnm-dev
27 libgpgme-dev
26 libupower-glib-dev
26 libev-dev
25 libnspr4-dev
25 libkf5mime-dev
24 libxerces-c-dev
24 libpoppler-qt5-dev
24 libpoppler-glib-dev
24 libnl-3-dev
24 libical-dev
24 libaio-dev
23 libgnome-desktop-3-dev
23 libc6-dev
22 libzip-dev
22 libssh2-1-dev
22 libhiredis-dev
21 vdr-dev
20 libtelepathy-glib-dev
20 libkf5libkdepim-dev
20 libc-ares-dev
20 catch
19 libuv1-dev
19 libsmbclient-dev
19 libgsf-1-dev
19 libgphoto2-dev
18 libmemcached-dev
17 libtbb-dev
17 libpoppler-private-dev
17 libneon27-gnutls-dev
17 libibverbs-dev
17 libiberty-dev
17 libgimp2.0-dev
17 libclthreads-dev
17 guile-2.2-dev
17 binutils-dev
16 libtracker-sparql-2.0-dev
16 libparted-dev
16 libmbedtls-dev

about 300 packages have nothing build-depending on them in 'main', and only 155
packages that use time_t in a heade file have more than 10 source packages with
build dependencies on them.

I've looked at the first few, and there are usually only a few obscure functions
that use time_t, so we can use the debian code search to figure out what
needs those. Here are a few potentially nasty cases:

https://codesearch.debian.net/search?q=png_convert_from_time_t
https://codesearch.debian.net/search?q=g_bookmark_file_get
https://codesearch.debian.net/search?q=gtk_recent_info_get
https://codesearch.debian.net/search?q=snd_%28h%7C%29timestamp_t
https://codesearch.debian.net/search?q=history_get_time
https://codesearch.debian.net/search?q=curl_getdate
https://codesearch.debian.net/search?q=ldap_search_ext
https://codesearch.debian.net/search?q=pcap_pkthdr

      Arnd

Re: Revisiting 64-bit time_t

[Arnd Bergmann]

On Mon, Jul 1, 2019 at 6:08 PM Rich Felker <dalias@libc.org> wrote:
>
> On Mon, Jul 01, 2019 at 04:41:51PM +0200, Arnd Bergmann wrote:
> > c) Keep backwards compatibility in libraries, but convert the
> >    distro one package at a time.
> >    Advantage: If done right, users can upgrade over rolling
> >    releases without ABIs breaking
> >    Disadvantage: very hard to get right, and much more work
> >    than the other two.
>
> I'm confused how "convert the distro one package at a time" comes up.
> The only packages potentially affected are ones that provide or
> consume a non-libc API with time_t or derived types as inputs or
> outputs. Such providers and consumers would have to be updated in sync
> with each other, but otherwise there should be no constraints about
> updating packages. I think this could be implemented with distros'
> standard "depends on" and "conflicts with" mechanics.

The difference to b) would be that one allows single package updates
at all, within the constraints of the distro dependency system.
Actually figuring the dependencies and documenting them is most of
the work here.

> > unrelated reasons. Debian will probably want c), but might also
> > be convinced to replace the existing 32-bit ports with musl based
> > ones to avoid the effort and reduce RAM usage at the same
> > time ;-)
>
> Replacing libc is like the polar opposite of option (c) here... :-)

Correct, this would basically put debian-i386 and debian-armhf into
category a), and create a new debian-musl-386 and debian-musl-armhf
that starts out with time64, and have these maintained in parallel
for as long as there is demand for both.

One reason why this may be interesting is that it reduces the
impact on the 64-bit Debian ports, which don't have to go through
the pain of a rebuild with new package versions and dependencies
when the actual ABI never changes for them.

It depends a bit on what approach Debian chooses for upgrading
some of the key packages, but if they determine that e.g. libcurl,
libssl, libgtk, libpng and libreadline would all need a new soname
for their library because of unfixable ABI incompatibility, *and* they
want to keep the soname the same between 32 and 64 bit at all cost,
then a new incompatible distro port can be the easier way out.
(it's clearly too early to know what it will take to rebuild 32-bit
debian fro time64, but we an think about the possible alternatives).

Side question for this: do you have an estimate of the potential
RAM saving when comparing glibc with musl builds of the same
software?

           Arnd

Re: Revisiting 64-bit time_t

[Rich Felker]

On Tue, Jul 02, 2019 at 11:35:25AM +0200, Arnd Bergmann wrote:
> On Mon, Jul 1, 2019 at 6:08 PM Rich Felker <dalias@libc.org> wrote:
> >
> > On Mon, Jul 01, 2019 at 04:41:51PM +0200, Arnd Bergmann wrote:
> > > c) Keep backwards compatibility in libraries, but convert the
> > >    distro one package at a time.
> > >    Advantage: If done right, users can upgrade over rolling
> > >    releases without ABIs breaking
> > >    Disadvantage: very hard to get right, and much more work
> > >    than the other two.
> >
> > I'm confused how "convert the distro one package at a time" comes up.
> > The only packages potentially affected are ones that provide or
> > consume a non-libc API with time_t or derived types as inputs or
> > outputs. Such providers and consumers would have to be updated in sync
> > with each other, but otherwise there should be no constraints about
> > updating packages. I think this could be implemented with distros'
> > standard "depends on" and "conflicts with" mechanics.
> 
> The difference to b) would be that one allows single package updates
> at all, within the constraints of the distro dependency system.
> Actually figuring the dependencies and documenting them is most of
> the work here.
> 
> > > unrelated reasons. Debian will probably want c), but might also
> > > be convinced to replace the existing 32-bit ports with musl based
> > > ones to avoid the effort and reduce RAM usage at the same
> > > time ;-)
> >
> > Replacing libc is like the polar opposite of option (c) here... :-)
> 
> Correct, this would basically put debian-i386 and debian-armhf into
> category a), and create a new debian-musl-386 and debian-musl-armhf
> that starts out with time64, and have these maintained in parallel
> for as long as there is demand for both.
> 
> One reason why this may be interesting is that it reduces the
> impact on the 64-bit Debian ports, which don't have to go through
> the pain of a rebuild with new package versions and dependencies
> when the actual ABI never changes for them.
> 
> It depends a bit on what approach Debian chooses for upgrading
> some of the key packages, but if they determine that e.g. libcurl,
> libssl, libgtk, libpng and libreadline would all need a new soname
> for their library because of unfixable ABI incompatibility, *and* they
> want to keep the soname the same between 32 and 64 bit at all cost,
> then a new incompatible distro port can be the easier way out.
> (it's clearly too early to know what it will take to rebuild 32-bit
> debian fro time64, but we an think about the possible alternatives).

I don't think using musl instead of glibc is a viable choice for
Debian archs; there are fundamental differences in behavior that would
break things for users who don't expect the change, and software
that's explicitly glibc-only like systemd that they use and would have
to fight with. If they want to take on these fights, I'd love it, but
I wouldn't expect it to happen.

> Side question for this: do you have an estimate of the potential
> RAM saving when comparing glibc with musl builds of the same
> software?

It will be highly dependent on the software and processes running.
musl makes significant gains for lots of instances of small programs,
where the minimal glibc bloat will dominate the overall process size,
and possibly for multithreaded processes where glibc's large default
thread stacks, caching of exited threads' stacks for reuse, and malloc
arenas can consume a lot of memory that's nominally not in use by the
application. On the flip side, musl's malloc has fundamental problems
that can waste lots of memory under high contention. This will be
fixed in the next-gen malloc which I hope to start on sometime soon.

Rich