From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,HTML_MESSAGE,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 30725 invoked from network); 11 Dec 2022 15:17:14 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 11 Dec 2022 15:17:14 -0000 Received: (qmail 32092 invoked by uid 550); 11 Dec 2022 15:17:10 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 13654 invoked from network); 11 Dec 2022 12:14:02 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=7AxBo8VSUi5NmJNP0p1Y7nipPYSDOE8s6utR2IKE1S0=; b=btnDH3byMkJPLUsDKR+mJdvpJP1yBSvG5NcqCdsLEuiVDhG+Rjp8QpJljCbI6GY6Ql tsdWiUq7GwT3O2f6eL+UhrJViYYkzT3HnT7/oKWc2RlgKccU4SWYmaADyDTs2TRubhP3 69+xqsE9F6FA49n+iotp0NL0602EjhN30wSJaqoA3Lp1g3uLTaJVOS1Qr14Z1ds9vmGM zC3WAR9ppoxAQnsUseUSORr2L2nxDNx0ZEeud3c9dKI9xFGimZ5qeEc9/0D18Zo9/SO9 i+3XquzLA12vPF4olJH6a3QHBiGnYjbM8gULcomjxc1tp80fha45fViD3voJN9Fml3Qr XJYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7AxBo8VSUi5NmJNP0p1Y7nipPYSDOE8s6utR2IKE1S0=; b=Oin4XJcvwS4Hyp/PhXwF9rO15kfjZGthbNCGWcSECdgc48NcJBNCPAEi2r+i4QkR0F 9No5vUu6mTR2h4/NFXTJnQJf01lUJOlyAZQZMcuTolb45H4DH0if+6Z9r4Enh0vp4kwh m4laDw6LP7uuf0LUccdB7z7KgkTREXiRQT0VuMUw7nkwUXMaY04bLK6/fP3bQIPN4nB+ +Ovz7eQzJmlR9CnNW9mar53AT4dOpWFEQCQAOOBq6i71HTIOE0srF8G3Ai/IdGUbouEB UJkzfreZQDjaOh8j0lz+NtvEahRmFBPsR3l5j3Y86xFNpAjINd7vPrFob/yRTdihhl3/ 9Gpw== X-Gm-Message-State: ANoB5pkUYywZdN4L9PO4lScf5QWXSiYLF6yERBZBvj2OuCWoareWRgkG eNmbslvdd2zU5GZs7R1R4nGz+NxPvQFZNvi+0+g= X-Google-Smtp-Source: AA0mqf4TKwBHGXaQwXwuj5aNEc42DyhYR+IACWxcgH+K2Nh7a6bkFrqMoJkG8DXAkqD9a+a/GJPdLA6Z42ykK8JYUe8= X-Received: by 2002:a05:622a:181a:b0:3a5:26e0:f60a with SMTP id t26-20020a05622a181a00b003a526e0f60amr68750337qtc.372.1670760830437; Sun, 11 Dec 2022 04:13:50 -0800 (PST) MIME-Version: 1.0 References: <20221211055335.GX29905@brightrain.aerifal.cx> In-Reply-To: <20221211055335.GX29905@brightrain.aerifal.cx> From: Yuriy Chernyshov Date: Sun, 11 Dec 2022 13:13:39 +0100 Message-ID: To: Rich Felker Cc: musl@lists.openwall.com Content-Type: multipart/alternative; boundary="000000000000cd8eb605ef8c51e6" Subject: Re: [musl] Various conflicts with linux system headers (ioctl.h) --000000000000cd8eb605ef8c51e6 Content-Type: text/plain; charset="UTF-8" On Sun, 11 Dec 2022 at 06:53, Rich Felker wrote: > On Tue, Dec 06, 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote: > > Hello, > > > > I am trying to build DBMS software (ydb from > > https://github.com/ydb-platform/ydb) with musl-libc. > > So far, I have run into certain problems. > > > > As YDB uses ioctl.h, it has to include both ioctl.h and certain headers > > from linux/ itself. This gives me a bunch of conflicts around _IOC macros > > family being defined by both: > > > > The error looks as follows: > > > > contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW' macro > > > redefined [-Werror,-Wmacro-redefined] > > > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c)) > > > ^ > > > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previous > > > definition is here > > > #define _IOW(type,nr,size) > > > _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size))) > > > > > The pathnames in the above error message make it look like you might > be using musl in a dubious way, that's likely causing the problem. > Particularly, trying to use the headers from a fork of the source tree > rather than the installed headers processed by the compiler driver as > being the "system include" path where this kind of warning would be > suppressed. > The paths in the errors log were taken from a monorepo which is being build with a proprtietary build system, which does not depend on the host system. We take the original source code and track original compilation in order to get the set of defines and cflags necessary for the compilation. > > That's not to say the conflicts aren't a problem, but to warn you that > you might hit other places where there's real breakage. Particularly, > if you're running your own build of musl not using the build process, > it's possible you'll be missing CFLAGS (e.g. -ffreestanding and > related things) necessary to suppress transformations that aren't > valid when compiling part of the implementation, and other issues like > that. > As we do not depend on host system / glibc, we pass -ffreestanding and -nostdinc to make musl build. I agree that there is a space for improvement, but I do not think it is worth being discussed in this thread. > > The following workaround helps, but looks quite ugly: > > > > --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f35) > > > +++ arch/generic/bits/ioctl.h > (d545cbc1ae3f5c9132eb26b176bef3638c9d8063) > > > @@ -1,3 +1,9 @@ > > > +#undef _IO > > > +#undef _IOC > > > +#undef _IOR > > > +#undef _IOW > > > +#undef _IOWR > > > + > > > #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | (c) | ((d)<<16) ) > > > #define _IOC_NONE 0U > > > #define _IOC_WRITE 1U > > > > > > > Is it possible to get official solution for the macro conflict? > > It's explicitly unsupported to include linux/* headers that might > produce conflicting definitions *before* the libc headers they might > conflict with. Does the same problem happen if you put the linux/* > headers after? > > > NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant > defined > > which is missing in sys/ioctl.h. > > > > Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Linux > > itself (starting from 2.6.4, which is below the minimal version > recommended > > by musl) sets it to 65536. > > > > It would be nice to have this value increased in musl itself. > > This has been an open issue for a while. At least the initgroups() > function is not prepared to handle large values of NGROUPS_MAX. > There's been some discussion in the past on how this could be changed. > I'll need to dig it up. I think the general leaning was that it should > be changed, once we work out a good way to do it. > > It would be nice to have this conflict fixed. > Rich > --000000000000cd8eb605ef8c51e6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sun, 11 Dec 2022 at 06:53, Rich Fe= lker <dalias@libc.org> wrote:<= br>
On Tue, Dec 06, = 2022 at 11:36:24AM +0100, Yuriy Chernyshov wrote:
> Hello,
>
> I am trying to build DBMS software (ydb from
> https://github.com/ydb-platform/ydb) with musl-libc.
> So far, I have run into certain problems.
>
> As YDB uses ioctl.h, it has to include both ioctl.h and certain header= s
> from linux/ itself. This gives me a bunch of conflicts around _IOC mac= ros
> family being defined by both:
>
> The error looks as follows:
>
> contrib/libs/musl/arch/generic/bits/ioctl.h:7:9: error: '_IOW'= macro
> > redefined [-Werror,-Wmacro-redefined]
> > #define _IOW(a,b,c) _IOC(_IOC_WRITE,(a),(b),sizeof(c))
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0^
> > contrib/libs/linux-headers/asm-generic/ioctl.h:90:9: note: previo= us
> > definition is here
> > #define _IOW(type,nr,size)
> >=C2=A0 _IOC(_IOC_WRITE,(type),(nr),(_IOC_TYPECHECK(size)))
> >

The pathnames in the above error message make it look like you might
be using musl in a dubious way, that's likely causing the problem.
Particularly, trying to use the headers from a fork of the source tree
rather than the installed headers processed by the compiler driver as
being the "system include" path where this kind of warning would = be
suppressed.
=C2=A0
The paths in the errors log were taken from a monorepo which is=20 being build with a proprtietary build system, which does not depend on=20 the host system.
We take the original source code and track original c= ompilation in order to get the set of defines and cflags necessary for the = compilation.
=C2=A0

That's not to say the conflicts aren't a problem, but to warn you t= hat
you might hit other places where there's real breakage. Particularly, if you're running your own build of musl not using the build process, it's possible you'll be missing CFLAGS (e.g. -ffreestanding and
related things) necessary to suppress transformations that aren't
valid when compiling part of the implementation, and other issues like
that.

As we do not depend on host syste= m / glibc, we pass -ffreestanding and -nostdinc to make musl build.
I agree that there is a space for improvement, but I do not think it is = worth being discussed in this thread.


> The following workaround helps, but looks quite ugly:
>
> --- arch/generic/bits/ioctl.h (b4624b83eafbdd5f2e2c37374d62426c27687f3= 5)
> > +++ arch/generic/bits/ioctl.h (d545cbc1ae3f5c9132eb26b176bef3638c= 9d8063)
> > @@ -1,3 +1,9 @@
> > +#undef _IO
> > +#undef _IOC
> > +#undef _IOR
> > +#undef _IOW
> > +#undef _IOWR
> > +
> >=C2=A0 #define _IOC(a,b,c,d) ( ((a)<<30) | ((b)<<8) | = (c) | ((d)<<16) )
> >=C2=A0 #define _IOC_NONE=C2=A0 0U
> >=C2=A0 #define _IOC_WRITE 1U
> >
>
> Is it possible to get official solution for the macro conflict?

It's explicitly unsupported to include linux/* headers that might
produce conflicting definitions *before* the libc headers they might
conflict with. Does the same problem happen if you put the linux/*
headers after?

> NB: we have to use linux/fs.h in order to get BLKGETSIZE64 constant de= fined
> which is missing in sys/ioctl.h.
>
> Another conflict is in NGROUPS_MAX value: musl sets it to 32, while Li= nux
> itself (starting from 2.6.4, which is below the minimal version recomm= ended
> by musl) sets it to 65536.
>
> It would be nice to have this value increased in musl itself.

This has been an open issue for a while. At least the initgroups()
function is not prepared to handle large values of NGROUPS_MAX.
There's been some discussion in the past on how this could be changed.<= br> I'll need to dig it up. I think the general leaning was that it should<= br> be changed, once we work out a good way to do it.


It would be nice to have this conflict= fixed.
=C2=A0
Rich
--000000000000cd8eb605ef8c51e6--