* [musl] no 32bit timestamp compatible stat/lstat/fstat? @ 2021-11-29 13:11 Norbert van Bolhuis 2021-11-29 17:35 ` Rich Felker 0 siblings, 1 reply; 4+ messages in thread From: Norbert van Bolhuis @ 2021-11-29 13:11 UTC (permalink / raw) To: musl Hi All, Why does musl-1.2.x not offer the 32bit timestamp compatible stat/lstat/fstat? We're using a 3rd-party binary (compiled against glibc) on an arm32v7 platform which fails to execute ever since we jumped from musl v1.1.x to 1.2.x, see: / # wl Error relocating /usr/sbin/wl: __fxstat: symbol not found The arm32v7 glibc-2.3x (working with 64bit timestamps) does provide them. Looking at: https://www.openwall.com/lists/musl/2019/08/01/1 it seems this is a known limitation. Any plans on fixing this? I solved my case by preloading a shared library for which the source is: #include <sys/stat.h> extern int __fxstat64(int ver, int fd, struct stat32 *buf); int __fxstat(int ver, int fd, struct stat32 *buf) { return __fxstat64(ver, fd, buf); } Regards, Norbert van Bolhuis Btw. please CC me on replies (as I'm not subscribed to musl@lists.openwall.com) ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] no 32bit timestamp compatible stat/lstat/fstat? 2021-11-29 13:11 [musl] no 32bit timestamp compatible stat/lstat/fstat? Norbert van Bolhuis @ 2021-11-29 17:35 ` Rich Felker 2021-11-29 23:15 ` Norbert van Bolhuis 0 siblings, 1 reply; 4+ messages in thread From: Rich Felker @ 2021-11-29 17:35 UTC (permalink / raw) To: Norbert van Bolhuis; +Cc: musl On Mon, Nov 29, 2021 at 02:11:47PM +0100, Norbert van Bolhuis wrote: > Hi All, > > Why does musl-1.2.x not offer the 32bit timestamp compatible stat/lstat/fstat? > > We're using a 3rd-party binary (compiled against glibc) on an arm32v7 platform > which fails to execute ever since we jumped from musl v1.1.x to 1.2.x, see: > > / # wl > Error relocating /usr/sbin/wl: __fxstat: symbol not found This is not the glibc symbol for 32-bit time_t but for 32-bit off_t, a very bad legacy ABI that musl has never supported and that shouldn't be used. Since it was not compiled with -D_FILE_OFFSET_BITS=64, any calls from this code to functions that take off_t, or structures containing off_t, will be wrong. If the only such functions are the stat family, you can probably make shims to convert and make it work. If it has calls to other functions like lseek, etc., then you may need to also patch the binary to call shims for those. Of course the best and easiest solution if possible would be to get the provider of this library to recompile it properly with -D_FILE_OFFSET_BITS=64, since without that it cannot handle large files or filesystems where inode numbers may not fit in 32 bits. This problem has nothing to do with musl; it's already a problem if you're using their code on glibc. > The arm32v7 glibc-2.3x (working with 64bit timestamps) does provide them. > > Looking at: https://www.openwall.com/lists/musl/2019/08/01/1 > it seems this is a known limitation. No, this was resolved, and it's a different issue: allowing glibc-built code to call the "lfs64" (_FILE_OFFSET_BITS=64) glibc names for the 64-bit off_t functions rather than their unadorned names (which is what musl uses natively since off_t is always 64-bit). > > Any plans on fixing this? > > I solved my case by preloading a shared library for which the source is: > > > #include <sys/stat.h> > > extern int __fxstat64(int ver, int fd, struct stat32 *buf); > > int __fxstat(int ver, int fd, struct stat32 *buf) > { > return __fxstat64(ver, fd, buf); > } That's insufficient because __fxstat is not expected to write the version of the structure __fxstat64 would write. It's expected to write some ancient legacy version of the sturcture (from the 1990s) where the offset and inode fields are only 32-bit. A working shim would require looking up (in the glibc headers) what that format is, and writing the code to convert on success. Rich ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] no 32bit timestamp compatible stat/lstat/fstat? 2021-11-29 17:35 ` Rich Felker @ 2021-11-29 23:15 ` Norbert van Bolhuis 2021-11-30 0:48 ` Rich Felker 0 siblings, 1 reply; 4+ messages in thread From: Norbert van Bolhuis @ 2021-11-29 23:15 UTC (permalink / raw) To: Rich Felker, Norbert van Bolhuis; +Cc: musl Hi Rich, Thanks for your answer! Ah, it's about 32-bit off_t compatibility, but then why does musl-1.1.24 (and the corresponding libc6-compat package on Alpine 3.12) provide this (ancient) __fxstat function? So this can never work properly and the reason why it worked with Alpine 3.12 (musl-1.1.24) is that the 3rd-party binary never really used __fxstat (because there is no way the 64-bit structure is compatible with the 32-bit one), right? Glibc does provide the ancient 32-bit variant, see: nm -D --defined-only /lib/arm-linux-gnueabihf/libc.so.6 | grep __fxstat 00093e68 T __fxstat 00093fdc T __fxstat64 000942a4 T __fxstatat 00094320 T __fxstatat64 I would expect musl libc6-compat package to provide it, why it doesn't? Anyway, I'll certainly ask the provider of the 3rd-party binary to recompile with -D_FILE_OFFSET_BITS=64. I may lookup this ancient 32-bit 'struct stat' and write a shim, another short-term "solution" is to simply return EOVERFLOW. This will tell me if __fxstat is called anywayr. Right now I have no idea, it sure looks like we don't need it for our use-case since we never had an issue with musl-1.1.x. Regards, Norbert On 11/29/21 6:35 PM, Rich Felker wrote: > On Mon, Nov 29, 2021 at 02:11:47PM +0100, Norbert van Bolhuis wrote: >> Hi All, >> >> Why does musl-1.2.x not offer the 32bit timestamp compatible stat/lstat/fstat? >> >> We're using a 3rd-party binary (compiled against glibc) on an arm32v7 platform >> which fails to execute ever since we jumped from musl v1.1.x to 1.2.x, see: >> >> / # wl >> Error relocating /usr/sbin/wl: __fxstat: symbol not found > > This is not the glibc symbol for 32-bit time_t but for 32-bit off_t, a > very bad legacy ABI that musl has never supported and that shouldn't > be used. Since it was not compiled with -D_FILE_OFFSET_BITS=64, any > calls from this code to functions that take off_t, or structures > containing off_t, will be wrong. If the only such functions are the > stat family, you can probably make shims to convert and make it work. > If it has calls to other functions like lseek, etc., then you may need > to also patch the binary to call shims for those. > > Of course the best and easiest solution if possible would be to get > the provider of this library to recompile it properly with > -D_FILE_OFFSET_BITS=64, since without that it cannot handle large > files or filesystems where inode numbers may not fit in 32 bits. This > problem has nothing to do with musl; it's already a problem if you're > using their code on glibc. > >> The arm32v7 glibc-2.3x (working with 64bit timestamps) does provide them. >> >> Looking at: https://www.openwall.com/lists/musl/2019/08/01/1 >> it seems this is a known limitation. > > No, this was resolved, and it's a different issue: allowing > glibc-built code to call the "lfs64" (_FILE_OFFSET_BITS=64) glibc > names for the 64-bit off_t functions rather than their unadorned names > (which is what musl uses natively since off_t is always 64-bit). > >> >> Any plans on fixing this? >> >> I solved my case by preloading a shared library for which the source is: >> >> >> #include <sys/stat.h> >> >> extern int __fxstat64(int ver, int fd, struct stat32 *buf); >> >> int __fxstat(int ver, int fd, struct stat32 *buf) >> { >> return __fxstat64(ver, fd, buf); >> } > > That's insufficient because __fxstat is not expected to write the > version of the structure __fxstat64 would write. It's expected to > write some ancient legacy version of the sturcture (from the 1990s) > where the offset and inode fields are only 32-bit. A working shim > would require looking up (in the glibc headers) what that format is, > and writing the code to convert on success. > > Rich > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [musl] no 32bit timestamp compatible stat/lstat/fstat? 2021-11-29 23:15 ` Norbert van Bolhuis @ 2021-11-30 0:48 ` Rich Felker 0 siblings, 0 replies; 4+ messages in thread From: Rich Felker @ 2021-11-30 0:48 UTC (permalink / raw) To: Norbert van Bolhuis; +Cc: musl On Tue, Nov 30, 2021 at 12:15:59AM +0100, Norbert van Bolhuis wrote: > Hi Rich, > > Thanks for your answer! > > Ah, it's about 32-bit off_t compatibility, but then why > does musl-1.1.24 (and the corresponding libc6-compat package on > Alpine 3.12) provide this (ancient) __fxstat function? The symbol there in 1.1.x didn't behave as needed to make 32-bit-off_t glibc code work. It just happened to be there as an alternate name for the 64-bit-off_t function __fxstat64 as a consequence of 32-bit archs not being special-cased. > So this can never work properly and the reason why it worked with > Alpine 3.12 (musl-1.1.24) is that the 3rd-party binary never really > used __fxstat (because there is no way the 64-bit structure is > compatible with the 32-bit one), right? Right. Well, it might have 'worked' if it didn't actually inspect the fields of the structure, but since the 64-bit structure is larger than the object the caller provided, it was overflowing a buffer and probably clobbering other data. > Glibc does provide the ancient 32-bit variant, see: > nm -D --defined-only /lib/arm-linux-gnueabihf/libc.so.6 | grep __fxstat > 00093e68 T __fxstat > 00093fdc T __fxstat64 > 000942a4 T __fxstatat > 00094320 T __fxstatat64 > I would expect musl libc6-compat package to provide it, why it doesn't? There's no way to support the 32-bit-off_t ABI with just a library, because the symbol names clash. It would require musl's dynamic linker to be specifically aware of foreign glibc libraries and remap their symbol references. This would be something of a cooperative project between musl and the folks working on gcompat to come up with a contract between them for making it all work -- something we've talked about as a vague future goal. Rich ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-11-30 0:48 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-29 13:11 [musl] no 32bit timestamp compatible stat/lstat/fstat? Norbert van Bolhuis 2021-11-29 17:35 ` Rich Felker 2021-11-29 23:15 ` Norbert van Bolhuis 2021-11-30 0:48 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).