[musl] Making user libraries Y2038 compatible?

[musl] Making user libraries Y2038 compatible?

[Alba Mendez]

Hi,

First of all, sorry if I'm asking in the wrong list or if this has been
asked already! User libraries understandably want to stay ABI and API
compatible with the Y2038 proofness and LFS design, just like libc
does, meaning:

- on systems where time_t (and other types) are 32-bit by default,
  offer an additional "<name>64" symbol in addition to "<name>".
- also on these systems, "#define <name>64 <name>" in the header if
  _TIME_BITS (or the relevant app-defined macro for the type) is 64.

This allows the library to begin providing 64-bit interfaces without
breaking ABI, and allows it to link with a user application without
forcing the application to use the same feature test macros as the
library.

To do this, I think libraries need at a minimum:

- a C type that refers to the original default for that platform (i.e.
  if no app-defined macros were present). glibc offers __ prefixed
  types for this purpose (__time_t, __off_t, __ino_t...)
- a preprocessor define to check if the a platform has a 32-bit default
  (glibc offers __TIMESIZE for this purpose; this decides the size of
  all affected types, time or size related)

In the absence of these mechanisms libraries resort to implementation-
specific and often incorrect[1] heuristics, like testing for
__BITS_PER_LONG==32 to see if the <name>64 symbol is needed (which is
incorrect in modern 32-bit platforms that were defined after Y2038
support was in place, like x32 or RV32).

Does musl provide, or would consider providing, such a mechanism? Or am
I approaching this from the wrong angle and libraries shouldn't be
doing this in the first place?

Many thanks,
Alba

[1]:
https://github.com/SELinuxProject/selinux/commit/9395cc03226a0e1a220a37d71d1a4158635c4284

Re: [musl] Making user libraries Y2038 compatible?

[Khem Raj]

On Tue, Mar 18, 2025 at 4:55 PM Alba Mendez <me@alba.sh> wrote:
>
> Hi,
>
> First of all, sorry if I'm asking in the wrong list or if this has been
> asked already! User libraries understandably want to stay ABI and API
> compatible with the Y2038 proofness and LFS design, just like libc
> does, meaning:
>
> - on systems where time_t (and other types) are 32-bit by default,
>   offer an additional "<name>64" symbol in addition to "<name>".
> - also on these systems, "#define <name>64 <name>" in the header if
>   _TIME_BITS (or the relevant app-defined macro for the type) is 64.
>
> This allows the library to begin providing 64-bit interfaces without
> breaking ABI, and allows it to link with a user application without
> forcing the application to use the same feature test macros as the
> library.

Musl always had 64bit time_t and off_t regardless of 32bit or 64bit systems
so I guess we are good here.

>
> To do this, I think libraries need at a minimum:
>
> - a C type that refers to the original default for that platform (i.e.
>   if no app-defined macros were present). glibc offers __ prefixed
>   types for this purpose (__time_t, __off_t, __ino_t...)
> - a preprocessor define to check if the a platform has a 32-bit default
>   (glibc offers __TIMESIZE for this purpose; this decides the size of
>   all affected types, time or size related)
>
> In the absence of these mechanisms libraries resort to implementation-
> specific and often incorrect[1] heuristics, like testing for
> __BITS_PER_LONG==32 to see if the <name>64 symbol is needed (which is
> incorrect in modern 32-bit platforms that were defined after Y2038
> support was in place, like x32 or RV32).
>
> Does musl provide, or would consider providing, such a mechanism? Or am
> I approaching this from the wrong angle and libraries shouldn't be
> doing this in the first place?
>
> Many thanks,
> Alba
>
> [1]:
> https://github.com/SELinuxProject/selinux/commit/9395cc03226a0e1a220a37d71d1a4158635c4284

Re: [musl] Making user libraries Y2038 compatible?

[Markus Wichmann]

Am Tue, Mar 18, 2025 at 11:49:43PM +0100 schrieb Alba Mendez:
> Hi,
>
> First of all, sorry if I'm asking in the wrong list or if this has been
> asked already! User libraries understandably want to stay ABI and API
> compatible with the Y2038 proofness and LFS design, just like libc
> does, meaning:
>
> - on systems where time_t (and other types) are 32-bit by default,
>   offer an additional "<name>64" symbol in addition to "<name>".
> - also on these systems, "#define <name>64 <name>" in the header if
>   _TIME_BITS (or the relevant app-defined macro for the type) is 64.
>
> This allows the library to begin providing 64-bit interfaces without
> breaking ABI, and allows it to link with a user application without
> forcing the application to use the same feature test macros as the
> library.
>
> To do this, I think libraries need at a minimum:
>
> - a C type that refers to the original default for that platform (i.e.
>   if no app-defined macros were present). glibc offers __ prefixed
>   types for this purpose (__time_t, __off_t, __ino_t...)
> - a preprocessor define to check if the a platform has a 32-bit default
>   (glibc offers __TIMESIZE for this purpose; this decides the size of
>   all affected types, time or size related)
>

This is what I like to call the "head through the wall" approach. You
attempt to explicitly opt into the only sensible choice for these types,
which massively overcomplicates the problem across the entire codebase.

The more sensible thing would be to find a way for the C library to
declare a 64-bit time_t and off_t and whatever else, and use that
exclusively. Ideally with glibc you should have been using
_FILE_OFFSET_BITS=64 and _TIME_BITS=64 since their inception, then you
wouldn't have these issues today. Because these features were added back
in 1997, if glibc's git log is to be believed.

So what is there to do? For applications, simply always set the above
two macros on glibc, then you get 64-bit types for time_t and off_t and
all the others. On musl, you don't need to do anything. How to tell?
Configure tests. It is pretty easy to statically test the size of types.
You just test if you get the right definitions at the outset, and if not
you try again with these macros defined.

For libraries, you must require this configuration at the outset. You
can put a static assert into your header files, for example. For
ABI compatibility, you may need to use symbol versioning and compat
definitions. Where the types become relevant ABI surface, you can resort
to the stdint.h types.

> In the absence of these mechanisms libraries resort to implementation-
> specific and often incorrect[1] heuristics, like testing for
> __BITS_PER_LONG==32 to see if the <name>64 symbol is needed (which is
> incorrect in modern 32-bit platforms that were defined after Y2038
> support was in place, like x32 or RV32).
>

_FILE_OFFSET_BITS and _TIME_BITS both *are* implementation-specific
heuristics.

Ciao,
Markus

Re: [musl] Making user libraries Y2038 compatible?

[James Y Knight]

On Tue, Mar 18, 2025 at 10:18 PM Khem Raj <raj.khem@gmail.com> wrote:
>
> Musl always had 64bit time_t and off_t regardless of 32bit or 64bit systems
> so I guess we are good here.

"Always"? 2019 wasn't _that_ long ago that everyone's forgotten it
existed already, is it?
https://musl.libc.org/time64.html
https://git.musl-libc.org/cgit/musl/commit/?id=38143339646a4ccce8afe298c34467767c899f51

For musl's time64 transition, the impacted non-libc libraries which
used time_t in their ABIs just took the ABI break. Distros rebuilt
everything that needed to be rebuilt.

Taking an ABI break is also what 99% of libraries are doing with the
glibc transition -- I suppose with libselinux being the exceptional
case.

Re: [musl] Making user libraries Y2038 compatible?

[Rich Felker]

On Tue, Mar 18, 2025 at 11:49:43PM +0100, Alba Mendez wrote:
> Hi,
> 
> First of all, sorry if I'm asking in the wrong list or if this has been
> asked already! User libraries understandably want to stay ABI and API
> compatible with the Y2038 proofness and LFS design, just like libc
> does, meaning:

musl libc does not do the LFS design. Myself and most of the community
generally feel like it was a really bad mistake glibc made.

> - on systems where time_t (and other types) are 32-bit by default,
>   offer an additional "<name>64" symbol in addition to "<name>".
> - also on these systems, "#define <name>64 <name>" in the header if
>   _TIME_BITS (or the relevant app-defined macro for the type) is 64.
> 
> This allows the library to begin providing 64-bit interfaces without
> breaking ABI, and allows it to link with a user application without
> forcing the application to use the same feature test macros as the
> library.

If you really want to do this, *please* don't make the *64() names
public API. Make it so they're automatically remapped if building on a
modern system where time_t is 64-bit or where 64-bit time_t is
available by request, and so the old symbol is just left there for ABI
compatibility with already-linked code using it.

> To do this, I think libraries need at a minimum:
> 
> - a C type that refers to the original default for that platform (i.e.
>   if no app-defined macros were present). glibc offers __ prefixed
>   types for this purpose (__time_t, __off_t, __ino_t...)
> - a preprocessor define to check if the a platform has a 32-bit default
>   (glibc offers __TIMESIZE for this purpose; this decides the size of
>   all affected types, time or size related)
> 
> In the absence of these mechanisms libraries resort to implementation-
> specific and often incorrect[1] heuristics, like testing for
> __BITS_PER_LONG==32 to see if the <name>64 symbol is needed (which is
> incorrect in modern 32-bit platforms that were defined after Y2038
> support was in place, like x32 or RV32).
> 
> Does musl provide, or would consider providing, such a mechanism? Or am
> I approaching this from the wrong angle and libraries shouldn't be
> doing this in the first place?

My leaning would be the latter. That ship really already sailed on
musl a long time ago, unlike on glibc, since 64-bit time_t has been
the only option since we first shipped it, and 32-bit is just there as
ABI-compat with legacy binaries.

If you are making a library that itself wants to do like musl and
provide ABI-compat with existing binaries by using a new remapped
symbol name for the time64 versions of your interfaces, I think the
canonical way to probe is with some kind of test for the symbol
remapping. Maybe something like, if sizeof(long)<sizeof(time_t),
compiling a .c file referencing time() and checking nm(1) for whether
it references a symbol named "time" or something else ("time64").

Rich

Re: [musl] Making user libraries Y2038 compatible?

[A. Wilcox]

On Mar 18, 2025, at 9:18 PM, Khem Raj <raj.khem@gmail.com> wrote:
> Musl always had 64bit time_t and off_t regardless of 32bit or 64bit systems
> so I guess we are good here.

For full disclosure and correctness, no, time_t was extended to 64-bit
in musl 1.2.0.

This was a breaking ABI change that we at Adélie Linux thoroughly
tested, and helped out with where possible.  We also sent many
multitudes of fixes and merge requests upstream when possible.  This
helped fix a lot of issues on platforms other than Linux, too, as some
of the BSDs have done this as well.

It is correct that off_t has been 64-bit always.

Best,
-Anna

Re: [musl] Making user libraries Y2038 compatible?

[Alba Mendez]

Thank you all for your responses! I had indeed missed that musl doesn't
implement the whole feature test macro / dual symbol system, and time_t
/ off_t are always 64-bit non-configurable. Thus libraries don't need
to do anything special for musl, nice!

> If you really want to do this, *please* don't make the *64() names
> public API. Make it so they're automatically remapped if building on
> a modern system where time_t is 64-bit or where 64-bit time_t is
> available by request, and so the old symbol is just left there for
> ABI compatibility with already-linked code using it.

That's the idea, yes. Users are only expected to use the normal names,
which will be conditionally remapped to the *64 versions through the
#define. Users can still use the *64 names in their source code though,
not sure how to prevent that :/

> So what is there to do? For applications, simply always set the above
> two macros on glibc, then you get 64-bit types for time_t and off_t
> and
> all the others. On musl, you don't need to do anything. How to tell?
> Configure tests. It is pretty easy to statically test the size of
> types.
> You just test if you get the right definitions at the outset, and if
> not
> you try again with these macros defined.
> 
> For libraries, you must require this configuration at the outset. You
> can put a static assert into your header files, for example. For
> ABI compatibility, you may need to use symbol versioning and compat
> definitions. Where the types become relevant ABI surface, you can
> resort
> to the stdint.h types.

I am certainly okay with doing this on my libraries, but other projects
may have the requirement for their (same) headers and .so binaries to
work on applications that do not define these macros as well as ones
that do. I'm not sure I'm in a position to tell them to just drop that
requirement ^^'

Alba