From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/14438 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Final (?) time64 proposal Date: Wed, 24 Jul 2019 01:31:42 -0400 Message-ID: <20190724053142.GB1506@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="259457"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.5.21 (2010-09-15) To: musl@lists.openwall.com Original-X-From: musl-return-14454-gllmg-musl=m.gmane.org@lists.openwall.com Wed Jul 24 07:31:58 2019 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1hq9sc-0015O0-AG for gllmg-musl@m.gmane.org; Wed, 24 Jul 2019 07:31:58 +0200 Original-Received: (qmail 5221 invoked by uid 550); 24 Jul 2019 05:31:55 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 5188 invoked from network); 24 Jul 2019 05:31:54 -0000 Content-Disposition: inline Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:14438 Archived-At: OK, doing the preliminaty work for time64 has helped to clarify a lot of the undecideds about it. To summarize, so far I've: - decoupled libc stat struct from kernel stat64 struct - refactored all stat-family functions in terms of fstatat - added support for statx syscall as a backend for fstatat - refactored adjtime-family function in terms of clock_adjtime - written all(?) compat shims for providing old ABI after switchover (out-of-tree, not committed) My plan to go ahead looks like: - time_t 64-bit - suseconds_t 64-bit (to match kernel timeval with no conversion) - timespec's long tv_nsec padded to 64-bit in endian-compatible manner - struct stat layout kept, extended with 64-bit timespecs on the end - struct utmpx layout kept, timeval moved to unused space at end - struct rusage layout kept, timevals moved to unused space at end - struct timex layout kept, timeval moved to unused space at end - struct shmid_ds, msqid_ds, semid_ds layout kept, extended with 64-bit time_t's on the ends - new symbol names for all functions with time_t, timeval, timespec, itimerval, itimerspec, utimbuf, timeb, or stat in their signatures. - keep existing symbol name for functions with utmpx or timex in their signatures, and fill in both old and new time fields. - one of the above two (not sure which yet) for functions with rusage in their signatures. using the reserved space we have without a new symbol name is safe for musl ABI but not for glibc ABI-compat. - compat shims (with the old symbol names) for all functions where a new symbol name is introduced. - symbol redirection of the standard function names to the new symbols via the public headers. - symbol redirection of the compat shim functions to the old symbol names via internal headers. - new definitions of IPC_STAT, SHM_STAT, SHM_STAT_ANY, SEM_STAT, SEM_STAT_ANY, MSG_STAT, and MSG_STAT_ANY to expand the shmid_ds, semid_ds, and msqid_ds upper and lower time bits from their locations in the kernel struct to the new fields at the end. Archs define via alltypes.h.in that they need symbol name redirection and compat shims. this ensures that the knowledge is available to all public headers but still arch-specific, since newly added 32-bit archs (including a possible future ".2 ABI") will *not* have any redirection or shims; they'll just be using 64-bit time_t from the start. This includes even old archs we might add, like sparc. One might wonder whether that affects glibc ABI-compat, e.g. for a hypothetical sparc port. Yes. With the 64-bit time_t transition, and the likelihood that glibc will do all this in a very different way (e.g. different struct layouts, possibly even API-violating ones like using long long for timex fields), I don't think it's going to be practical to keep glibc ABI-compat for all the interfaces defined in musl. Rather, I think it will end up making sense to preserve the functionality via collaboration with the gcompat project, and probably rewiring things so that gcompat or other foreign-ABI-compat libraries get processed as intercepting symbol references from the app/lib needing foreign-ABI-compat, rather than residing in the global symbol namespace. This will also make it possible to fix the regexec/regoff_t mismatch on 64-bit archs. One possibility I'd like to consider is allowing a build with something between the proposal and the ".2 ABI" - omitting the compat shims from libc.a and/or libc.so. This is not the same thing as a ".2 ABI" because it wouldn't overhaul type definitions to be clean and uniform and it wouldn't crete a *conflicting* ABI with the current .1 ABI. The new 64-bit time_t functions would use all their redirected symbol names the same as described above, so binaries built with the above scheme would work with or without this option. The only difference would be that the compat shims would be missing, so that old, 32-bit-time_t binaries and libraries would fail to load. This would allow users concerned about the impact of having a mixed ecosystem, or concerned about non-Y2038-compatible stuff creeping into their systems, to ensure that everything is consistently using 64-bit time_t. With that said, the following are the functions I've identified as still interfacing with the kernel in ways that involve time_t: - clock_gettime ** - clock_settime ** - clock_adjtime ** - clock_nanosleep - timer_gettime - timer_settime - timerfd_gettime - timerfd_settime - utimensat ** - mq_timedsend ** - mq_timedreceive ** - getitimer - setitimer - select - pselect - ppoll - recvmmsg ? - sigtimedwait - semtimedop - clock_getres - sched_rr_get_interval - __timedwait (backend for all timed futex waits) - getrusage - wait4 - wait3 - setsockopt - getsockopt - shmctl *** - semctl *** - msgctl *** - ioctl - recvmsg ? The ones marked with ** absolutely need to use new kernel syscalls when available, falling back to the old ones only if they're not, because they need to accept or return absolute times that don't fit in the legacy 32-bit interfaces. The ones marked with *** need to decode new high bits provided through existing syscalls. The ones marked with ?, recvmsg and recvmmsg, potentially have to rewrite cmsgs if we want SO_TIMESTAMP* to keep working on 32-bit archs without requiring a kernel >=5.1. I'm not sure if this is a sufficiently widely-used feature to warrant such a heavy hammer. We could just wait and see if anything breaks, or ask distros to investigate, or do some code search. Aside from those special cases, the rest of the above only deal with relative times, and can happily continue using the same syscalls they're using; they just need to convert to/from userspace timeval/timespec/etc. The special (**) ones above also need to convert to/from userspace format in the fallback cases when new syscalls are unavailable (ENOSYS). Comments on any of the above are welcome. Especially please speak up if you see any omissions/oversights. Rich