[-- Attachment #1: Type: text/plain, Size: 1565 bytes --] Hello, I am trying to build DBMS software (ydb from https://github.com/ydb-platform/ydb) with musl-libc. So far, I have run into certain problems. As YDB uses ioctl.h, it has to include both ioctl.h and certain headers from linux/ itself. This gives me a bunch of conflicts around _IOC macros family being defined by both: The error looks as follows: contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro > redefined [-Werror,-Wmacro-redefined] > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c)) > ^ > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous > definition is here > #define _IOW(type,nr,size) > _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size))) > The following workaround helps, but looks quite ugly: --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35) > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > @@ -1,3 +1,9 @@ > +#undef _IO > +#undef _IOC > +#undef _IOR > +#undef _IOW > +#undef _IOWR > + > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > #define _IOC_NONE 0U > #define _IOC_WRITE 1U > Is it possible to get official solution for the macro conflict? NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined which is missing in sys/ioctl.h. Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux itself (starting from 2.6.4, which is below the minimal version recommended by musl) sets it to 65536. It would be nice to have this value increased in musl itself. Regards, Yuriy. [-- Attachment #2: Type: text/html, Size: 2300 bytes --]
On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote: > Hello, > > I am trying to build DBMS software (ydb from > https://github.com/ydb-platform/ydb) with musl-libc. > So far, I have run into certain problems. > > As YDB uses ioctl.h, it has to include both ioctl.h and certain headers > from linux/ itself. This gives me a bunch of conflicts around _IOC macros > family being defined by both: > > The error looks as follows: > > contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro > > redefined [-Werror,-Wmacro-redefined] > > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c)) > > ^ > > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous > > definition is here > > #define _IOW(type,nr,size) > > _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size))) > > The pathnames in the above error message make it look like you might be using musl in a dubious way, that's likely causing the problem. Particularly, trying to use the headers from a fork of the source tree rather than the installed headers processed by the compiler driver as being the "system include" path where this kind of warning would be suppressed. That's not to say the conflicts aren't a problem, but to warn you that you might hit other places where there's real breakage. Particularly, if you're running your own build of musl not using the build process, it's possible you'll be missing CFLAGS (e.g. -ffreestanding and related things) necessary to suppress transformations that aren't valid when compiling part of the implementation, and other issues like that. > The following workaround helps, but looks quite ugly: > > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35) > > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > > @@ -1,3 +1,9 @@ > > +#undef _IO > > +#undef _IOC > > +#undef _IOR > > +#undef _IOW > > +#undef _IOWR > > + > > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > > #define _IOC_NONE 0U > > #define _IOC_WRITE 1U > > > > Is it possible to get official solution for the macro conflict? It's explicitly unsupported to include linux/* headers that might produce conflicting definitions *before* the libc headers they might conflict with. Does the same problem happen if you put the linux/* headers after? > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined > which is missing in sys/ioctl.h. > > Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux > itself (starting from 2.6.4, which is below the minimal version recommended > by musl) sets it to 65536. > > It would be nice to have this value increased in musl itself. This has been an open issue for a while. At least the initgroups() function is not prepared to handle large values of NGROUPS_MAX. There's been some discussion in the past on how this could be changed. I'll need to dig it up. I think the general leaning was that it should be changed, once we work out a good way to do it. Rich
* Rich Felker <dalias@libc.org> [2022-12-11 00:53:35 -0500]: > On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote: > > The following workaround helps, but looks quite ugly: > > > > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35) > > > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > > > @@ -1,3 +1,9 @@ > > > +#undef _IO > > > +#undef _IOC > > > +#undef _IOR > > > +#undef _IOW > > > +#undef _IOWR > > > + > > > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > > > #define _IOC_NONE 0U > > > #define _IOC_WRITE 1U > > > > > > > Is it possible to get official solution for the macro conflict? > > It's explicitly unsupported to include linux/* headers that might > produce conflicting definitions *before* the libc headers they might > conflict with. Does the same problem happen if you put the linux/* > headers after? i don't think reordering can fix the conflict as linux defines the macros unconditionally. (and glibc relies on the linux definitions) > > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined > > which is missing in sys/ioctl.h. musl defines that in sys/mount.h (just like glibc)
[-- Attachment #1: Type: text/plain, Size: 3807 bytes --] On Sun, 11 Dec 2022 at 06:53, Rich Felker <dalias@libc.org> wrote: > On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote: > > Hello, > > > > I am trying to build DBMS software (ydb from > > https://github.com/ydb-platform/ydb) with musl-libc. > > So far, I have run into certain problems. > > > > As YDB uses ioctl.h, it has to include both ioctl.h and certain headers > > from linux/ itself. This gives me a bunch of conflicts around _IOC macros > > family being defined by both: > > > > The error looks as follows: > > > > contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro > > > redefined [-Werror,-Wmacro-redefined] > > > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c)) > > > ^ > > > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous > > > definition is here > > > #define _IOW(type,nr,size) > > > _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size))) > > > > > The pathnames in the above error message make it look like you might > be using musl in a dubious way, that's likely causing the problem. > Particularly, trying to use the headers from a fork of the source tree > rather than the installed headers processed by the compiler driver as > being the "system include" path where this kind of warning would be > suppressed. > The paths in the errors log were taken from a monorepo which is being build with a proprtietary build system, which does not depend on the host system. We take the original source code and track original compilation in order to get the set of defines and cflags necessary for the compilation. > > That's not to say the conflicts aren't a problem, but to warn you that > you might hit other places where there's real breakage. Particularly, > if you're running your own build of musl not using the build process, > it's possible you'll be missing CFLAGS (e.g. -ffreestanding and > related things) necessary to suppress transformations that aren't > valid when compiling part of the implementation, and other issues like > that. > As we do not depend on host system / glibc, we pass -ffreestanding and -nostdinc to make musl build. I agree that there is a space for improvement, but I do not think it is worth being discussed in this thread. > > The following workaround helps, but looks quite ugly: > > > > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35) > > > +++ arch/generic/bits/ioctl.h > (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > > > @@ -1,3 +1,9 @@ > > > +#undef _IO > > > +#undef _IOC > > > +#undef _IOR > > > +#undef _IOW > > > +#undef _IOWR > > > + > > > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > > > #define _IOC_NONE 0U > > > #define _IOC_WRITE 1U > > > > > > > Is it possible to get official solution for the macro conflict? > > It's explicitly unsupported to include linux/* headers that might > produce conflicting definitions *before* the libc headers they might > conflict with. Does the same problem happen if you put the linux/* > headers after? > > > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant > defined > > which is missing in sys/ioctl.h. > > > > Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux > > itself (starting from 2.6.4, which is below the minimal version > recommended > > by musl) sets it to 65536. > > > > It would be nice to have this value increased in musl itself. > > This has been an open issue for a while. At least the initgroups() > function is not prepared to handle large values of NGROUPS_MAX. > There's been some discussion in the past on how this could be changed. > I'll need to dig it up. I think the general leaning was that it should > be changed, once we work out a good way to do it. > > It would be nice to have this conflict fixed. > Rich > [-- Attachment #2: Type: text/html, Size: 5261 bytes --]
[-- Attachment #1: Type: text/plain, Size: 1703 bytes --] On Sun, 11 Dec 2022 at 12:07, Szabolcs Nagy <nsz@port70.net> wrote: > * Rich Felker <dalias@libc.org> [2022-12-11 00:53:35 -0500]: > > On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote: > > > The following workaround helps, but looks quite ugly: > > > > > > --- arch/generic/bits/ioctl.h > (b4624b83eafbdd5f2e2c37374d62426c27687f35) > > > > +++ arch/generic/bits/ioctl.h > (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > > > > @@ -1,3 +1,9 @@ > > > > +#undef _IO > > > > +#undef _IOC > > > > +#undef _IOR > > > > +#undef _IOW > > > > +#undef _IOWR > > > > + > > > > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > > > > #define _IOC_NONE 0U > > > > #define _IOC_WRITE 1U > > > > > > > > > > Is it possible to get official solution for the macro conflict? > > > > It's explicitly unsupported to include linux/* headers that might > > produce conflicting definitions *before* the libc headers they might > > conflict with. Does the same problem happen if you put the linux/* > > headers after? > > i don't think reordering can fix the conflict as linux defines the > macros unconditionally. (and glibc relies on the linux definitions) > Indeed, the reording was the first thing I have tried. I have failed to find a working solution. > > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant > defined > > > which is missing in sys/ioctl.h. > > musl defines that in sys/mount.h (just like glibc) > This would have been a solution for BLKGETSIZE64, but once I have started to refactor things, I got more problems. Our code also depends on BLKDISCARD, which is not defined by neither musl nor glibc. Is it possible to add this define into musl codebase? [-- Attachment #2: Type: text/html, Size: 2572 bytes --]