time64 abi choices for glibc and musl

time64 abi choices for glibc and musl

[Rich Felker]

Note: this email is to glibc list and particularly seeking feedback
from glibc folks, but CC'd to musl list as well.

As far as I can tell, most time64 work/discussion on the glibc side so
far has been about implementation mechanisms and symbol-binding ABI
aspects, and one aspect that doesn't seem to have been addressed is
making good choices about the actual types involved. Some of them have
been done:

struct timespec (endian-matching padding)
struct timeval (64-bit suseconds_t)
struct itimerspec, itimerval, utimbuf, timeb (obvious definitions)

but I haven't seen a clear proposal with rationale for the choices in
defining:

struct stat
struct msqid_ds, semid_ds, shmid_ds (sysvipc)
struct rusage
struct timex
struct utmp[x]

I'd like to avoid ending up with gratuitous differences between musl's
and glibc's definitions of these types, which is my direct motivation
for asking now, but I'd also like to help ensure glibc makes
well-justified choices rather than just overlooking these and getting
locked into poorly thought-out ones.

For struct stat, I think it's almost necessary to preserve the
existing layout and add new time64 members on the end, and to continue
filling in the time32 members. This is because of how struct stat is
used in callback interfaces -- otherwise, glibc will need a complete
duplicate of ftw/nftw for time32 vs time64. As long as the layouts are
compatible on the initial overlapped part, it's fine for a
time64-aware glibc's time64-aware [n]ftw to callback into a time32
application, passing it pointers to the time64 stat structs. This
issue also affects ABI between pairs of libc consumers. It's common
to pass struct stat pointers across library boundaries, but the party
filling them is almost always the same party allocating them. As long
as the overlapping part of both definitions agrees, this simply works.

For the sysvipc structs, it was tempting to try to use the padding
next to the time32 fields in-place (possibly with endian swapping);
this was what Arnd and the kernel folks intended (except on mips where
padding was omitted). However it's impossible because alignment is
wrong. Almost all of the time_t members are at offsets that are 4 mod
8 on 32-bit archs. So for these musl is adding new time64 members at
the end of the structs, and leaving the old time32 ones in place. I
think glibc should do the same. I spent a lot of time exploring
options here and there just was no other reasonable option.

For rusage, timex, and utmp[x], it's less clear to me what should be
done. I'd like to align what musl does with glibc at least on rusage.
Does anyone from the glibc side have preferences for how it should be
done? I'd rather avoid having a huge gratuitously-padded structure
like x32 has, possibly keeping the time32 members in-place and just
adding new time64 ones at the end, or change the ones at the beginning
to time64 so that the time64 version of rusage can be filled just by
making the time32 syscall with an appropriate offset (8 or 0 depending
on the above choice)then expanding the time members. Either of these
options also allows easy implementation of the time32 compat shim just
by memcpy.

There are probably a few more structs like this I've omitted -- see
https://sourceware.org/glibc/wiki/Y2038ProofnessDesig for a
near-complete list, but it omitted shmid_ds so it should be checked
that nothing else was omitted too. The others are probably not
relevant to musl which is why I haven't looked into them much. (Even
utmp[x] is mostly irrelevant to us at present since musl's utmp is a
stub.)

Rich

Re: time64 abi choices for glibc and musl

[Paul Eggert]

Rich Felker wrote:
> For struct stat, I think it's almost necessary to preserve the
> existing layout and add new time64 members on the end, and to continue
> filling in the time32 members.

This won't be reliable for applications that call 'stat', fiddle with the 
timestamps in the resulting 'struct stat', and then pass the fiddled-with struct 
to some other function. If they fiddle with the 64-bit timestamp the 32-bit copy 
will become wrong, and vice versa.

There are similar issues for applications that create struct stat values by 
hand, without calling a 'stat'-like function.

Re: [musl] Re: time64 abi choices for glibc and musl

[Rich Felker]

On Sat, Aug 10, 2019 at 01:15:34PM -0700, Paul Eggert wrote:
> Rich Felker wrote:
> >For struct stat, I think it's almost necessary to preserve the
> >existing layout and add new time64 members on the end, and to continue
> >filling in the time32 members.
> 
> This won't be reliable for applications that call 'stat', fiddle
> with the timestamps in the resulting 'struct stat', and then pass
> the fiddled-with struct to some other function. If they fiddle with
> the 64-bit timestamp the 32-bit copy will become wrong, and vice
> versa.
> 
> There are similar issues for applications that create struct stat
> values by hand, without calling a 'stat'-like function.

Changing any of the types defined by libc obviously alters the ABI
between any two consumers of these types that uses the changed types
as part of the interface between them. The only thing we can
*guarantee* works is the interface between libc and a single consumer
of libc. Since there's no function in libc which reads from a stat
structure provided by the application, your concern seems to apply
only to the case that's inherently a best-effort thing. Still, here it
is useful to make a best effort, since the only typical way stat
structures are passed around is share (unmodified) results obtained by
performing one of the stat functions on a file.

The [n]ftw matter, by itself, is something entirely different, and
pertains to use of libc by a single consumer. Internally, [n]ftw
performs stat on files and passes the result to a callback provided by
the application. If the stat structure is not such that it can be
accepted by either time32 or time64 callers, then [n]ftw need to be
redirected symbols and there need to be two copies of the
implementation, one that calls the old time32 stat and one that calls
the time64 stat. This is a gratuitous pain on the implementation side.

Now the two are actually related in some sense, because this same
idiom, of passing stat results back to a caller, is fairly common in
library code outside libc that does directory listings, tree walks,
etc. The same condition that makes it [n]ftw painless on the libc
implementation side also makes it so that this kind of library works
with time32 callers when the library was built for time64 (assuming
there are no other points of breakage). Again, this is a best-effort
thing anyway, and can't inherently be expected to work, but the choice
that makes things easy on the libc implementation side is *also* the
choice that makes this work best. Thereby I consider it the most
reasonable choice.

Rich

Re: [musl] Re: time64 abi choices for glibc and musl

[Paul Eggert]

Rich Felker wrote:
> this is a best-effort
> thing anyway, and can't inherently be expected to work, but the choice
> that makes things easy on the libc implementation side is *also* the
> choice that makes this work best.

It doesn't entirely simplify libc, as it enlarges struct stat and (more 
importantly) makes struct stat tricky. This is a judgment call, but I would say 
we're better off in the long run with a simpler struct stat that ordinary 
programmers will understand easily, even if this complicates nftw implementation 
during the transition.

Re: [musl] Re: time64 abi choices for glibc and musl

[Rich Felker]

On Sun, Aug 11, 2019 at 05:31:48PM -0700, Paul Eggert wrote:
> Rich Felker wrote:
> >this is a best-effort
> >thing anyway, and can't inherently be expected to work, but the choice
> >that makes things easy on the libc implementation side is *also* the
> >choice that makes this work best.
> 
> It doesn't entirely simplify libc, as it enlarges struct stat and
> (more importantly) makes struct stat tricky. This is a judgment
> call, but I would say we're better off in the long run with a
> simpler struct stat that ordinary programmers will understand
> easily, even if this complicates nftw implementation during the
> transition.

The various archs already have random junk padding in struct stat.
It's true that the time64 transition would allow the 32-bit archs to
do away with this and invent a uniform definition that they can all
share, but the 64-bit archs would still be stuck with all the legacy
stuff, and struct stat still needs to be an arch-specific bits header
anyway. I don't see much value in making things "prettier" for 32-bit
here when we can't do the same for 64-bit, especially when the vast
majority of users are 64-bit.

The reason I mention "random junk padding" is that, in the long term,
that's what the time32 members end up becoming. If building with
_TIME_BITS=32 is no longer supported at some point, they don't even
have to be called __st_atim32 or whatever anymore; they can be called
__reservedN or whatever, and effectively they're just like the other
padding junk in struct stat. If at some point in the distant, distant
future time32 binaries are no longer supported at all, these reserved
members could even be repurposed for something else if needed.

Aside from time_t and struct timespec themselves, which *have to*
change in an incompatible way, struct stat is probably the most widely
used time_t-dependent data type defined by libc. As stated before,
making things work when time32 and time64 code get linked in the same
program is inherently a best-effort thing at best, but struct stat is
one case where making a best effort will usually be sufficient not to
have any breakage (and only non-serious, invalid-read breakage if
breakage happens).

Both glibc and musl place significant value on stable ABI and
compatibility with users' binaries and libraries. For glibc, every
gratuitous breakage when time32 and time64 code gets mixed is going to
make users hesitant to define _TIME_BITS=64, and thereby delay the
eventual switchover of the default. The switchover very well might get
delayed well past "too late" already, leaving broken stuff all over
the place with 2038 quickly approaching.

For musl, there is not going to be a build option unless you roll your
own by building with old headers. After the time64 release, all newly
built programs will be using time64. As such, distros are going to
have to deal with any breakage that occurs right away, and users
who've built stuff themselves are going to end up dealing with
mismatches. As part of being responsible about our promises of stable
ABI, I do not want to be introducing gratuitous breakage where it
could easily have been avoided. The other types mentioned in my
original email are fairly inconsequential (not going to appear in
interfaces between libraries), but stat is important and does appear.

For what it's worth, as long as glibc is doing the __xstat thing, we
can provide ABI-compat shims through that and not really care if the
layout matches. But I really think glibc would be better off not
breaking things here, so that the transition is as painless as
possible.

Rich

Re: time64 abi choices for glibc and musl

[Zack Weinberg]

On Sat, Aug 10, 2019 at 1:58 PM Rich Felker <dalias@libc.org> wrote:
> As far as I can tell, most time64 work/discussion on the glibc side so
> far has been about implementation mechanisms and symbol-binding ABI
> aspects, and one aspect that doesn't seem to have been addressed is
> making good choices about the actual types involved. Some of them have
> been done:
>
> struct timespec (endian-matching padding)
> struct timeval (64-bit suseconds_t)
> struct itimerspec, itimerval, utimbuf, timeb (obvious definitions)
>
> but I haven't seen a clear proposal with rationale for the choices in
> defining:
>
> struct stat
> struct msqid_ds, semid_ds, shmid_ds (sysvipc)
> struct rusage
> struct timex
> struct utmp[x]

The sysvipc structures, struct rusage, and struct timex are passed
directly from and to the kernel, so this ship has already sailed -- I
don't see that we have any choice but to match the kernel's layouts of
these structures in time64 mode, whether or not that is convenient for
the C library or for call compatibility between procedures compiled
with _TIME_BITS=32 and _TIME_BITS=64.

We do have flex for struct utmp[x], which is entirely controlled by
user space, and for struct stat, because the kernel is providing only
statx() in the 64-bit-time ABI.

Regarding struct stat, I tend to agree with Paul that having both 32-
and 64-bit time fields _visible_ at the same time will cause more
problems than it solves.  What I would suggest is that we add the
64-bit 'struct timespec's at the end, as you suggest, and continue to
write (possibly truncated) valid data to the legacy 32-bit 'struct
timespec's, but the public header file _immediately_ converts the
legacy fields to __reservedN fields.  This should work except for when
an existing _TIME_BITS=32 binary caller changes the 32-bit fields of a
struct stat, passes it to a _TIME_BITS=64 callee, and expects it to
notice.  Is there any practical way for us to find out whether any
such combination actually exists?  It also occurs to me that this
might mean we can share stat/fstat/lstat for _TIME_BITS=32 and =64.

struct utmp[x] describes a format for long-lived files on disk, so not
only do we have to keep the time field where it is, we have to make it
possible for new programs to read and understand old records.  And
there is no padding on either side of the ut_tv field, so we would be
stuck ... except that it uses struct timeval, and it's silly to track
login events to sub-second precision.  So I'm going to suggest that we
redefine that field as a bare time64_t:

#define UT_LINESIZE     32
#define UT_NAMESIZE     32
#define UT_HOSTSIZE     256

struct utmp{,x}
{
  uint16_t ut_type;
  uint16_t __reserved1;
  uint32_t ut_pid;
  char ut_line[UT_LINESIZE];
  char ut_id[4];
  char ut_user[UT_NAMESIZE];
  char ut_host[UT_HOSTSIZE];
  struct exit_status {
    int16_t e_termination;
    int16_t e_exit;
  } ut_exit;
#if __32_BIT_UT_SESSION
  int32_t ut_session;
#else
  int64_t ut_session;
#endif
  union {
    _Alignas(__old_time_t) uint64_t ut_time64;
    struct {
      __old_time_t tv_sec; // historic time_t for this ABI, whatever that was
      int32_t tv_usec;
    } ut_tv;
  };
#if __PAD_AFTER_UT_TV
  int32_t __reserved2;
#endif
  int32_t ut_addr_v6[4];
  char __reserved3[20];
};

We could define a new set of ut_type codes to indicate which of ut_tv
and ut_time64 is meaningful.

Unfortunately no such hack appears to be possible for struct lastlog,
and that has the same problem.  Anyone have an idea for that one?

zw

Re: time64 abi choices for glibc and musl

[Rich Felker]

On Mon, Aug 12, 2019 at 12:29:55PM -0400, Zack Weinberg wrote:
> On Sat, Aug 10, 2019 at 1:58 PM Rich Felker <dalias@libc.org> wrote:
> > As far as I can tell, most time64 work/discussion on the glibc side so
> > far has been about implementation mechanisms and symbol-binding ABI
> > aspects, and one aspect that doesn't seem to have been addressed is
> > making good choices about the actual types involved. Some of them have
> > been done:
> >
> > struct timespec (endian-matching padding)
> > struct timeval (64-bit suseconds_t)
> > struct itimerspec, itimerval, utimbuf, timeb (obvious definitions)
> >
> > but I haven't seen a clear proposal with rationale for the choices in
> > defining:
> >
> > struct stat
> > struct msqid_ds, semid_ds, shmid_ds (sysvipc)
> > struct rusage
> > struct timex
> > struct utmp[x]
> 
> The sysvipc structures, struct rusage, and struct timex are passed
> directly from and to the kernel, so this ship has already sailed -- I
> don't see that we have any choice but to match the kernel's layouts of
> these structures in time64 mode, whether or not that is convenient for
> the C library or for call compatibility between procedures compiled
> with _TIME_BITS=32 and _TIME_BITS=64.

The kernel does not have a format for the sysvipc structures. It gives
you the upper bits of the 64-bit (or, for mips, 48-bit) time_t's
packed into various padding members. This *forces* libc to define a
type and do some sort of translation; the only question is what it
will do.

(Strictly speaking, on i386 you can use the kernel high bits in-place,
but this only works because i386 (1) i386 is little-endian, and (2)
i386 ABI doesn't align 64-bit types to 64-bit boundaries, only to
32-bit boundaries. 32-bit ppc also got lucky and makes it work for a
different reason. But the rest fundamentally *can't* work with the
kernel types in-place. If you're interested in details I can cite my
findings working on this for musl.)

Regarding rusage, there is no SYS_getrusage_time64 syscall at present.
The only kernel type on the existing 32-bit archs (not including RV32
which will use a 64-bit type I think, but I'm not sure) is the 32-bit
one, so some kind of translation is necessary to get a 64-bit struct.
Only 3 functions use this structure (and one, wait3, is just a trivial
wrapper around wait4), so conversion is no big burden.

For timex, it is indeed a kernel interface, and opting to use the
64-bit version (with endian-appropriate padding for the members
documented as long but that use kernel int64's) is a very reasonable
choice. Note that, due to the padding issue, if the kernel reads and
interprets the padding as part of the value, a temp copy of the
structure is needed anyway, in which case it's not really any cheaper
than just translating the whole thing. So it might make sense to
define the libc type without the padding. This is my leaning for musl.

> Regarding struct stat, I tend to agree with Paul that having both 32-
> and 64-bit time fields _visible_ at the same time will cause more
> problems than it solves.  What I would suggest is that we add the
> 64-bit 'struct timespec's at the end, as you suggest, and continue to
> write (possibly truncated) valid data to the legacy 32-bit 'struct
> timespec's, but the public header file _immediately_ converts the
> legacy fields to __reservedN fields.

I'm perfectly fine with this solution, and find it better than
exposing the names, even in the __st_* namespace.

> This should work except for when
> an existing _TIME_BITS=32 binary caller changes the 32-bit fields of a
> struct stat, passes it to a _TIME_BITS=64 callee, and expects it to
> notice.  Is there any practical way for us to find out whether any
> such combination actually exists?

I don't know a good way right off, but it's known, and inherent, that
some things will break under mixing time32 and time64 object
files/shared libs in a single program. What you're describing is a
weird usage and probably only appears either in LD_PRELOAD stuff
that's faking stat results (in which case it has to be aware of all
the glibc __xstat stuff and ABI versions, and use of pre-time64 builds
of it will not work on time64 binaries at all), or perhaps in archival
tools reusing the system's struct stat to move around information
about archive contents (as opposed to stat structures built by calling
one of the libc functions).

> It also occurs to me that this
> might mean we can share stat/fstat/lstat for _TIME_BITS=32 and =64.

You can't share them entirely because the time64 ones will overflow a
time32 caller's buffer. What you can do is make the time32 compat
functions thin shims that just do:

	struct stat st;
	int r = stat(pathname, &st);
	if (!r) memcpy(st32, &st, sizeof *st32);
	return r;

In addition, if you want to catch overflows, you could either check
that the time64 member values fit in 32 bits, or compare them against
the time32 members, and produce EOVERFLOW.

> struct utmp[x] describes a format for long-lived files on disk, so not
> only do we have to keep the time field where it is, we have to make it
> possible for new programs to read and understand old records.  And
> there is no padding on either side of the ut_tv field, so we would be
> stuck ... except that it uses struct timeval, and it's silly to track
> login events to sub-second precision.  So I'm going to suggest that we
> redefine that field as a bare time64_t:
> 
> #define UT_LINESIZE     32
> #define UT_NAMESIZE     32
> #define UT_HOSTSIZE     256
> 
> struct utmp{,x}
> {
>   uint16_t ut_type;
>   uint16_t __reserved1;
>   uint32_t ut_pid;
>   char ut_line[UT_LINESIZE];
>   char ut_id[4];
>   char ut_user[UT_NAMESIZE];
>   char ut_host[UT_HOSTSIZE];
>   struct exit_status {
>     int16_t e_termination;
>     int16_t e_exit;
>   } ut_exit;
> #if __32_BIT_UT_SESSION
>   int32_t ut_session;
> #else
>   int64_t ut_session;
> #endif
>   union {
>     _Alignas(__old_time_t) uint64_t ut_time64;

Per the C standard, _Alignas() does not/cannot produce alignment
smaller than the standard required alignment for the type. You'd need
a GNUC-specific attribute here instead.

>     struct {
>       __old_time_t tv_sec; // historic time_t for this ABI, whatever that was
>       int32_t tv_usec;
>     } ut_tv;
>   };
> #if __PAD_AFTER_UT_TV
>   int32_t __reserved2;
> #endif
>   int32_t ut_addr_v6[4];
>   char __reserved3[20];
> };
> 
> We could define a new set of ut_type codes to indicate which of ut_tv
> and ut_time64 is meaningful.
> 
> Unfortunately no such hack appears to be possible for struct lastlog,
> and that has the same problem.  Anyone have an idea for that one?

Note that struct utmpx is specified by POSIX (XSI option) and required
to contain struct timespec ut_tv. So these changes are non-conforming,
but the x86_64 definition is already non-conforming (using 32-bit
timeval for compat with 32-bit x86 binaries instead of a real
timeval), so presumably glibc doesn't care. I'd like to see this fixed
but it's a huge pain since the structure is used on-disk. Short of
that, any solution glibc adopts is not a significant regression from
the current situation in my eyes, so I don't have much of an opinion
on this. (FWIW I also hate utmp and consider it at best a creepy
stalking tool of 80s/90s era BOFH's, so I'm not the best source of
opinions on this stuff.)

Rich

Re: [musl] time64 abi choices for glibc and musl

[Rich Felker]

On Sat, Aug 10, 2019 at 01:58:08PM -0400, Rich Felker wrote:
> Note: this email is to glibc list and particularly seeking feedback
> from glibc folks, but CC'd to musl list as well.
> 
> As far as I can tell, most time64 work/discussion on the glibc side so
> far has been about implementation mechanisms and symbol-binding ABI
> aspects, and one aspect that doesn't seem to have been addressed is
> making good choices about the actual types involved. Some of them have
> been done:
> 
> struct timespec (endian-matching padding)
> struct timeval (64-bit suseconds_t)
> struct itimerspec, itimerval, utimbuf, timeb (obvious definitions)
> 
> but I haven't seen a clear proposal with rationale for the choices in
> defining:
> 
> struct stat
> struct msqid_ds, semid_ds, shmid_ds (sysvipc)
> struct rusage
> struct timex
> struct utmp[x]
> 
> [...]

The wiki at https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
does not seem to cover any of this issue, and the two branch links
that should shed light on the types seem to be down/deleted:

https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038-2.26
https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038-2.26-rfc-2

I'm guessing the following is the current one that should be linked
instead, no?

https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038

Rich

Re: [musl] Re: time64 abi choices for glibc and musl

[Rich Felker]

On Sun, Aug 11, 2019 at 09:55:14PM -0400, Rich Felker wrote:
> On Sun, Aug 11, 2019 at 05:31:48PM -0700, Paul Eggert wrote:
> > Rich Felker wrote:
> > >this is a best-effort
> > >thing anyway, and can't inherently be expected to work, but the choice
> > >that makes things easy on the libc implementation side is *also* the
> > >choice that makes this work best.
> > 
> > It doesn't entirely simplify libc, as it enlarges struct stat and
> > (more importantly) makes struct stat tricky. This is a judgment
> > call, but I would say we're better off in the long run with a
> > simpler struct stat that ordinary programmers will understand
> > easily, even if this complicates nftw implementation during the
> > transition.
> 
> The various archs already have random junk padding in struct stat.

Apologies; I was under a longstanding mistaken impression that glibc
used the kernel stat64 types, and had per-arch bits headers to provide
them, but apparently it doesn't and always uses its own fixed, fairly
clean layout (sadly without reserved space for expansion), though.

So indeed it would be a little bit of an uglification to add
time64-on-32-bit-arch members here. I still think the benefits to
minimizing breakage of applications (and avoiding the need for
duplicate [n]ftw implementations and symbol redirects, which nobody
has proposed doing yet, likely because nobody even realized they would
have been needed) are worth it, but my assessment was wrong.

This also means whatever glibc decides to do about struct stat is
irrelevant to musl's glibc-ABI-compat goals; what I assumed was
working now is in fact broken, but doesn't matter because the __xstat
shims can fix it up if desired.

Rich

Re: Re: time64 abi choices for glibc and musl

[Rich Felker]

Taking this off CC, just to musl list...

On Mon, Aug 12, 2019 at 02:01:19PM -0400, Rich Felker wrote:
> On Sun, Aug 11, 2019 at 09:55:14PM -0400, Rich Felker wrote:
> > On Sun, Aug 11, 2019 at 05:31:48PM -0700, Paul Eggert wrote:
> > > Rich Felker wrote:
> > > >this is a best-effort
> > > >thing anyway, and can't inherently be expected to work, but the choice
> > > >that makes things easy on the libc implementation side is *also* the
> > > >choice that makes this work best.
> > > 
> > > It doesn't entirely simplify libc, as it enlarges struct stat and
> > > (more importantly) makes struct stat tricky. This is a judgment
> > > call, but I would say we're better off in the long run with a
> > > simpler struct stat that ordinary programmers will understand
> > > easily, even if this complicates nftw implementation during the
> > > transition.
> > 
> > The various archs already have random junk padding in struct stat.
> 
> Apologies; I was under a longstanding mistaken impression that glibc
> used the kernel stat64 types, and had per-arch bits headers to provide
> them, but apparently it doesn't and always uses its own fixed, fairly
> clean layout (sadly without reserved space for expansion), though.
> 
> So indeed it would be a little bit of an uglification to add
> time64-on-32-bit-arch members here. I still think the benefits to
> minimizing breakage of applications (and avoiding the need for
> duplicate [n]ftw implementations and symbol redirects, which nobody
> has proposed doing yet, likely because nobody even realized they would
> have been needed) are worth it, but my assessment was wrong.
> 
> This also means whatever glibc decides to do about struct stat is
> irrelevant to musl's glibc-ABI-compat goals; what I assumed was
> working now is in fact broken, but doesn't matter because the __xstat
> shims can fix it up if desired.

I'm quite surprised this happened, and it's certainly causing some
(likely high) degree of breakage for any attempts to use
glibc-ABI-compat/gcompat stuff. i386 is not affected, precisely
because glibc's unified type is just a copy of the i386 version of the
kernel's struct stat64; them being the same is probably how I wrongly
assumed glibc used the kernel types, back when musl was i386-only. I
vaguely remember some discussion of whether musl should translate stat
back then, so it might have been known but the importance
overlooked...

In any case, this should be fixed. Fortunately for us, the libc stat
functions are never callable from glibc code, because glibc redirects
them through the __xstat functions in libc_nonshared.a. Since glibc
has a unified type for all archs, it would be easy to add code to
convert to the glibc type here, either in musl or gcompat.

Rich

Re: time64 abi choices for glibc and musl

[Joseph Myers]

On Sat, 10 Aug 2019, Rich Felker wrote:

> For struct stat, I think it's almost necessary to preserve the
> existing layout and add new time64 members on the end, and to continue

You mean the existing stat64 layout?  (Since the combination of 64-bit 
time with 32-bit offsets is deliberately not supported by glibc.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: [musl] time64 abi choices for glibc and musl

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 2170 bytes --]

Hi Rich,

> On Sat, Aug 10, 2019 at 01:58:08PM -0400, Rich Felker wrote:
> > Note: this email is to glibc list and particularly seeking feedback
> > from glibc folks, but CC'd to musl list as well.
> > 
> > As far as I can tell, most time64 work/discussion on the glibc side
> > so far has been about implementation mechanisms and symbol-binding
> > ABI aspects, and one aspect that doesn't seem to have been
> > addressed is making good choices about the actual types involved.
> > Some of them have been done:
> > 
> > struct timespec (endian-matching padding)
> > struct timeval (64-bit suseconds_t)
> > struct itimerspec, itimerval, utimbuf, timeb (obvious definitions)
> > 
> > but I haven't seen a clear proposal with rationale for the choices
> > in defining:
> > 
> > struct stat
> > struct msqid_ds, semid_ds, shmid_ds (sysvipc)
> > struct rusage
> > struct timex
> > struct utmp[x]
> > 
> > [...]  
> 
> The wiki at https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
> does not seem to cover any of this issue, 
> and the two branch links
> that should shed light on the types seem to be down/deleted:
> 
> https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038-2.26
> https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038-2.26-rfc-2
> 
> I'm guessing the following is the current one that should be linked
> instead, no?
> 
> https://sourceware.org/git/?p=glibc.git;a=log;h=refs/heads/aaribaud/y2038

The most recent development of this code - i.e. 2.29 glibc working with
warrior Yocto/OE can be found here:

https://github.com/lmajewski/y2038_glibc/commits/Y2038-2.29-glibc-warrior-01-08-2019

(At least on 32 bit system BSP build with Yocto/OE with glibc from above
link it is possible to set / read date beyond Y2038. However, there are
still some parts broken - like ping timestamps.)

> 
> Rich



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: time64 abi choices for glibc and musl

[Rich Felker]

On Mon, Aug 12, 2019 at 08:13:22PM +0000, Joseph Myers wrote:
> On Sat, 10 Aug 2019, Rich Felker wrote:
> 
> > For struct stat, I think it's almost necessary to preserve the
> > existing layout and add new time64 members on the end, and to continue
> 
> You mean the existing stat64 layout?  (Since the combination of 64-bit 
> time with 32-bit offsets is deliberately not supported by glibc.)

Yes. Sorry, I pretend the 32-bit-off_t one doesn't exist. :-)

Rich