mailing list of musl libc
 help / color / mirror / code / Atom feed
* [musl] Various conflicts with linux system headers (ioctl.h)
@ 2022-12-06 10:36 Yuriy Chernyshov
  2022-12-11  5:53 ` Rich Felker
  0 siblings, 1 reply; 5+ messages in thread
From: Yuriy Chernyshov @ 2022-12-06 10:36 UTC (permalink / raw)
  To: musl

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

Hello,

I am trying to build DBMS software (ydb from
https://github.com/ydb-platform/ydb) with musl-libc.
So far, I have run into certain problems.

As YDB uses ioctl.h, it has to include both ioctl.h and certain headers
from linux/ itself. This gives me a bunch of conflicts around _IOC macros
family being defined by both:

The error looks as follows:

contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro
> redefined [-Werror,-Wmacro-redefined]
> #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c))
>         ^
> contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous
> definition is here
> #define _IOW(type,nr,size)
>  _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size)))
>

The following workaround helps, but looks quite ugly:

--- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35)
> +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063)
> @@ -1,3 +1,9 @@
> +#undef _IO
> +#undef _IOC
> +#undef _IOR
> +#undef _IOW
> +#undef _IOWR
> +
>  #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) )
>  #define _IOC_NONE  0U
>  #define _IOC_WRITE 1U
>

Is it possible to get official solution for the macro conflict?

NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined
which is missing in sys/ioctl.h.

Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux
itself (starting from 2.6.4, which is below the minimal version recommended
by musl) sets it to 65536.

It would be nice to have this value increased in musl itself.


Regards,
Yuriy.

[-- Attachment #2: Type: text/html, Size: 2300 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [musl] Various conflicts with linux system headers (ioctl.h)
  2022-12-06 10:36 [musl] Various conflicts with linux system headers (ioctl.h) Yuriy Chernyshov
@ 2022-12-11  5:53 ` Rich Felker
  2022-12-11 11:07   ` Szabolcs Nagy
  2022-12-11 12:13   ` Yuriy Chernyshov
  0 siblings, 2 replies; 5+ messages in thread
From: Rich Felker @ 2022-12-11  5:53 UTC (permalink / raw)
  To: Yuriy Chernyshov; +Cc: musl

On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote:
> Hello,
> 
> I am trying to build DBMS software (ydb from
> https://github.com/ydb-platform/ydb) with musl-libc.
> So far, I have run into certain problems.
> 
> As YDB uses ioctl.h, it has to include both ioctl.h and certain headers
> from linux/ itself. This gives me a bunch of conflicts around _IOC macros
> family being defined by both:
> 
> The error looks as follows:
> 
> contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro
> > redefined [-Werror,-Wmacro-redefined]
> > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c))
> >         ^
> > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous
> > definition is here
> > #define _IOW(type,nr,size)
> >  _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size)))
> >

The pathnames in the above error message make it look like you might
be using musl in a dubious way, that's likely causing the problem.
Particularly, trying to use the headers from a fork of the source tree
rather than the installed headers processed by the compiler driver as
being the "system include" path where this kind of warning would be
suppressed.

That's not to say the conflicts aren't a problem, but to warn you that
you might hit other places where there's real breakage. Particularly,
if you're running your own build of musl not using the build process,
it's possible you'll be missing CFLAGS (e.g. -ffreestanding and
related things) necessary to suppress transformations that aren't
valid when compiling part of the implementation, and other issues like
that.

> The following workaround helps, but looks quite ugly:
> 
> --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35)
> > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063)
> > @@ -1,3 +1,9 @@
> > +#undef _IO
> > +#undef _IOC
> > +#undef _IOR
> > +#undef _IOW
> > +#undef _IOWR
> > +
> >  #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) )
> >  #define _IOC_NONE  0U
> >  #define _IOC_WRITE 1U
> >
> 
> Is it possible to get official solution for the macro conflict?

It's explicitly unsupported to include linux/* headers that might
produce conflicting definitions *before* the libc headers they might
conflict with. Does the same problem happen if you put the linux/*
headers after?

> NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined
> which is missing in sys/ioctl.h.
> 
> Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux
> itself (starting from 2.6.4, which is below the minimal version recommended
> by musl) sets it to 65536.
> 
> It would be nice to have this value increased in musl itself.

This has been an open issue for a while. At least the initgroups()
function is not prepared to handle large values of NGROUPS_MAX.
There's been some discussion in the past on how this could be changed.
I'll need to dig it up. I think the general leaning was that it should
be changed, once we work out a good way to do it.

Rich

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [musl] Various conflicts with linux system headers (ioctl.h)
  2022-12-11  5:53 ` Rich Felker
@ 2022-12-11 11:07   ` Szabolcs Nagy
  2022-12-11 12:26     ` Yuriy Chernyshov
  2022-12-11 12:13   ` Yuriy Chernyshov
  1 sibling, 1 reply; 5+ messages in thread
From: Szabolcs Nagy @ 2022-12-11 11:07 UTC (permalink / raw)
  To: Rich Felker; +Cc: Yuriy Chernyshov, musl

* Rich Felker <dalias@libc.org> [2022-12-11 00:53:35 -0500]:
> On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote:
> > The following workaround helps, but looks quite ugly:
> > 
> > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35)
> > > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c9d8063)
> > > @@ -1,3 +1,9 @@
> > > +#undef _IO
> > > +#undef _IOC
> > > +#undef _IOR
> > > +#undef _IOW
> > > +#undef _IOWR
> > > +
> > >  #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) )
> > >  #define _IOC_NONE  0U
> > >  #define _IOC_WRITE 1U
> > >
> > 
> > Is it possible to get official solution for the macro conflict?
> 
> It's explicitly unsupported to include linux/* headers that might
> produce conflicting definitions *before* the libc headers they might
> conflict with. Does the same problem happen if you put the linux/*
> headers after?

i don't think reordering can fix the conflict as linux defines the
macros unconditionally. (and glibc relies on the linux definitions)

> > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant defined
> > which is missing in sys/ioctl.h.

musl defines that in sys/mount.h (just like glibc)

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [musl] Various conflicts with linux system headers (ioctl.h)
  2022-12-11  5:53 ` Rich Felker
  2022-12-11 11:07   ` Szabolcs Nagy
@ 2022-12-11 12:13   ` Yuriy Chernyshov
  1 sibling, 0 replies; 5+ messages in thread
From: Yuriy Chernyshov @ 2022-12-11 12:13 UTC (permalink / raw)
  To: Rich Felker; +Cc: musl

[-- Attachment #1: Type: text/plain, Size: 3807 bytes --]

On Sun, 11 Dec 2022 at 06:53, Rich Felker <dalias@libc.org> wrote:

> On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote:
> > Hello,
> >
> > I am trying to build DBMS software (ydb from
> > https://github.com/ydb-platform/ydb) with musl-libc.
> > So far, I have run into certain problems.
> >
> > As YDB uses ioctl.h, it has to include both ioctl.h and certain headers
> > from linux/ itself. This gives me a bunch of conflicts around _IOC macros
> > family being defined by both:
> >
> > The error looks as follows:
> >
> > contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro
> > > redefined [-Werror,-Wmacro-redefined]
> > > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c))
> > >         ^
> > > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous
> > > definition is here
> > > #define _IOW(type,nr,size)
> > >  _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size)))
> > >
>
> The pathnames in the above error message make it look like you might
> be using musl in a dubious way, that's likely causing the problem.
> Particularly, trying to use the headers from a fork of the source tree
> rather than the installed headers processed by the compiler driver as
> being the "system include" path where this kind of warning would be
> suppressed.
>

The paths in the errors log were taken from a monorepo which is being build
with a proprtietary build system, which does not depend on the host system.
We take the original source code and track original compilation in order to
get the set of defines and cflags necessary for the compilation.


>
> That's not to say the conflicts aren't a problem, but to warn you that
> you might hit other places where there's real breakage. Particularly,
> if you're running your own build of musl not using the build process,
> it's possible you'll be missing CFLAGS (e.g. -ffreestanding and
> related things) necessary to suppress transformations that aren't
> valid when compiling part of the implementation, and other issues like
> that.
>

As we do not depend on host system / glibc, we pass -ffreestanding and
-nostdinc to make musl build.
I agree that there is a space for improvement, but I do not think it is
worth being discussed in this thread.


> > The following workaround helps, but looks quite ugly:
> >
> > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35)
> > > +++ arch/generic/bits/ioctl.h
> (d545cbc1ae3f5c9132eb26b176bef3638c9d8063)
> > > @@ -1,3 +1,9 @@
> > > +#undef _IO
> > > +#undef _IOC
> > > +#undef _IOR
> > > +#undef _IOW
> > > +#undef _IOWR
> > > +
> > >  #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) )
> > >  #define _IOC_NONE  0U
> > >  #define _IOC_WRITE 1U
> > >
> >
> > Is it possible to get official solution for the macro conflict?
>
> It's explicitly unsupported to include linux/* headers that might
> produce conflicting definitions *before* the libc headers they might
> conflict with. Does the same problem happen if you put the linux/*
> headers after?
>
> > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant
> defined
> > which is missing in sys/ioctl.h.
> >
> > Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux
> > itself (starting from 2.6.4, which is below the minimal version
> recommended
> > by musl) sets it to 65536.
> >
> > It would be nice to have this value increased in musl itself.
>
> This has been an open issue for a while. At least the initgroups()
> function is not prepared to handle large values of NGROUPS_MAX.
> There's been some discussion in the past on how this could be changed.
> I'll need to dig it up. I think the general leaning was that it should
> be changed, once we work out a good way to do it.
>
>
It would be nice to have this conflict fixed.


> Rich
>

[-- Attachment #2: Type: text/html, Size: 5261 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [musl] Various conflicts with linux system headers (ioctl.h)
  2022-12-11 11:07   ` Szabolcs Nagy
@ 2022-12-11 12:26     ` Yuriy Chernyshov
  0 siblings, 0 replies; 5+ messages in thread
From: Yuriy Chernyshov @ 2022-12-11 12:26 UTC (permalink / raw)
  To: Rich Felker, Yuriy Chernyshov, musl

[-- Attachment #1: Type: text/plain, Size: 1703 bytes --]

On Sun, 11 Dec 2022 at 12:07, Szabolcs Nagy <nsz@port70.net> wrote:

> * Rich Felker <dalias@libc.org> [2022-12-11 00:53:35 -0500]:
> > On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote:
> > > The following workaround helps, but looks quite ugly:
> > >
> > > --- arch/generic/bits/ioctl.h
> (b4624b83eafbdd5f2e2c37374d62426c27687f35)
> > > > +++ arch/generic/bits/ioctl.h
> (d545cbc1ae3f5c9132eb26b176bef3638c9d8063)
> > > > @@ -1,3 +1,9 @@
> > > > +#undef _IO
> > > > +#undef _IOC
> > > > +#undef _IOR
> > > > +#undef _IOW
> > > > +#undef _IOWR
> > > > +
> > > >  #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) )
> > > >  #define _IOC_NONE  0U
> > > >  #define _IOC_WRITE 1U
> > > >
> > >
> > > Is it possible to get official solution for the macro conflict?
> >
> > It's explicitly unsupported to include linux/* headers that might
> > produce conflicting definitions *before* the libc headers they might
> > conflict with. Does the same problem happen if you put the linux/*
> > headers after?
>
> i don't think reordering can fix the conflict as linux defines the
> macros unconditionally. (and glibc relies on the linux definitions)
>

Indeed, the reording was the first thing I have tried.
I have failed to find a working solution.

> > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant
> defined
> > > which is missing in sys/ioctl.h.
>
> musl defines that in sys/mount.h (just like glibc)
>

This would have been a solution for BLKGETSIZE64, but once I have started
to refactor things, I got more problems.
Our code also depends on BLKDISCARD, which is not defined by neither musl
nor glibc.
Is it possible to add this define into musl codebase?

[-- Attachment #2: Type: text/html, Size: 2572 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2022-12-11 15:17 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-06 10:36 [musl] Various conflicts with linux system headers (ioctl.h) Yuriy Chernyshov
2022-12-11  5:53 ` Rich Felker
2022-12-11 11:07   ` Szabolcs Nagy
2022-12-11 12:26     ` Yuriy Chernyshov
2022-12-11 12:13   ` Yuriy Chernyshov

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).