Issues building gdbserver with musl

Issues building gdbserver with musl

[Thomas Petazzoni]

Hello,

We are building a number of toolchains, some of them using the musl C
library, and are seeing that gdbserver fails to build with musl on a
number of architectures. We have used musl 1.1.16, i.e the latest
version.

Here are the failures that we see:

 - On SuperH:

   usr/sh4-buildroot-linux-musl/sysroot/usr/include/asm/ptrace_32.h:44:8: error: redefinition of 'struct pt_regs'

 - On PowerPC64 and PowerPC64le:

   usr/powerpc64-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'
   usr/powerpc64le-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'

 - On PowerPC:

   usr/powerpc-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'

 - On Microblaze, little endian and big endian

   usr/microblaze-buildroot-linux-musl/sysroot/usr/include/sys/procfs.h:30:2: error: unknown type name 'elf_gregset_t'
   error: 'Elf64_auxv_t' undeclared (first use in this function)
   error: 'Elf32_auxv_t' undeclared (first use in this function)
   ...

On all those architectures, gdbserver builds fine with other C libraries.

I had a brief look at the commits in musl since 1.1.16 and couldn't
spot anything that looked related.

Are you interested in fixing those issues? If needed, I can provide an
easy way of reproducing those build failures.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Issues building gdbserver with musl

[Khem Raj]

Hi Thomas



On Sat, Jun 3, 2017 at 7:14 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> We are building a number of toolchains, some of them using the musl C
> library, and are seeing that gdbserver fails to build with musl on a
> number of architectures. We have used musl 1.1.16, i.e the latest
> version.
>
> Here are the failures that we see:
>
>  - On SuperH:
>
>    usr/sh4-buildroot-linux-musl/sysroot/usr/include/asm/ptrace_32.h:44:8: error: redefinition of 'struct pt_regs'
>
>  - On PowerPC64 and PowerPC64le:
>
>    usr/powerpc64-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'
>    usr/powerpc64le-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'
>
>  - On PowerPC:
>
>    usr/powerpc-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'


for ppe issues you can use something like
http://git.yoctoproject.org/clean/cgit.cgi/poky/tree/meta/recipes-devtools/gdb/gdb/0004-ppc-ptrace-Define-pt_regs-uapi_pt_regs-on-GLIBC-syst.patch

>
>  - On Microblaze, little endian and big endian
>
>    usr/microblaze-buildroot-linux-musl/sysroot/usr/include/sys/procfs.h:30:2: error: unknown type name 'elf_gregset_t'
>    error: 'Elf64_auxv_t' undeclared (first use in this function)
>    error: 'Elf32_auxv_t' undeclared (first use in this function)
>    ...
>
> On all those architectures, gdbserver builds fine with other C libraries.
>
> I had a brief look at the commits in musl since 1.1.16 and couldn't
> spot anything that looked related.
>
> Are you interested in fixing those issues? If needed, I can provide an
> easy way of reproducing those build failures.
>
> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux and Kernel engineering
> http://free-electrons.com

Re: Issues building gdbserver with musl

[mzpqnxow]

[-- Attachment #1: Type: text/plain, Size: 3070 bytes --]

Which version of gdb are you using? I've found I had to patch several
versions of gdb when building gdbserver for non-x86 platforms, possibly due
to less QA testing of these "obscure" platforms on the gdb dev side.

I at least encountered issues with ARM and MIPS on some of the 6 and/or 7
branch of gdb. Patches were only about 20-30 lines max.

I'm maintaining a collection of statically linked gdbserver executables[1]
for various embedded platforms so I'd be happy to write a patch to address
the issues in gdb and send them to you and upstream if they can be fixed on
the gdb side of things. I need to add those platforms anyway so it's no
bother.

I'll take a look today to see if I can patch any errors in gdb to get a
functional build but if it's in fact a musl issue I'll have to defer.

Have you had success building the same gdbserver versions with glibc or
uClibc for the same architectures?

Adam

1.
https://github.com/mzpqnxow/embedded-toolkit/tree/master/prebuilt_static_bins/gdbserver

On Sat, Jun 3, 2017 at 10:35 Khem Raj <raj.khem@gmail.com> wrote:

> Hi Thomas
>
>
>
> On Sat, Jun 3, 2017 at 7:14 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Hello,
> >
> > We are building a number of toolchains, some of them using the musl C
> > library, and are seeing that gdbserver fails to build with musl on a
> > number of architectures. We have used musl 1.1.16, i.e the latest
> > version.
> >
> > Here are the failures that we see:
> >
> >  - On SuperH:
> >
> >
> usr/sh4-buildroot-linux-musl/sysroot/usr/include/asm/ptrace_32.h:44:8:
> error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC64 and PowerPC64le:
> >
> >
> usr/powerpc64-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
> >
> usr/powerpc64le-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC:
> >
> >
> usr/powerpc-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
>
>
> for ppe issues you can use something like
>
> http://git.yoctoproject.org/clean/cgit.cgi/poky/tree/meta/recipes-devtools/gdb/gdb/0004-ppc-ptrace-Define-pt_regs-uapi_pt_regs-on-GLIBC-syst.patch
>
> >
> >  - On Microblaze, little endian and big endian
> >
> >
> usr/microblaze-buildroot-linux-musl/sysroot/usr/include/sys/procfs.h:30:2:
> error: unknown type name 'elf_gregset_t'
> >    error: 'Elf64_auxv_t' undeclared (first use in this function)
> >    error: 'Elf32_auxv_t' undeclared (first use in this function)
> >    ...
> >
> > On all those architectures, gdbserver builds fine with other C libraries.
> >
> > I had a brief look at the commits in musl since 1.1.16 and couldn't
> > spot anything that looked related.
> >
> > Are you interested in fixing those issues? If needed, I can provide an
> > easy way of reproducing those build failures.
> >
> > Thanks!
> >
> > Thomas
> > --
> > Thomas Petazzoni, CTO, Free Electrons
> > Embedded Linux and Kernel engineering
> > http://free-electrons.com
>

[-- Attachment #2: Type: text/html, Size: 4306 bytes --]

Re: Issues building gdbserver with musl

[Thomas Petazzoni]

Hello,

On Sat, 3 Jun 2017 07:34:14 -0700, Khem Raj wrote:

> >  - On SuperH:
> >
> >    usr/sh4-buildroot-linux-musl/sysroot/usr/include/asm/ptrace_32.h:44:8: error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC64 and PowerPC64le:
> >
> >    usr/powerpc64-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'
> >    usr/powerpc64le-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC:
> >
> >    usr/powerpc-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8: error: redefinition of 'struct pt_regs'  
> 
> for ppe issues you can use something like
> http://git.yoctoproject.org/clean/cgit.cgi/poky/tree/meta/recipes-devtools/gdb/gdb/0004-ppc-ptrace-Define-pt_regs-uapi_pt_regs-on-GLIBC-syst.patch

Thanks. This fixes the build on ppc and pp64. A slightly modified
version also fixed the build on SuperH.

However, it feels a lot like a hack. Your patch says it has been
submitted upstream. What was the feedback ?

> >  - On Microblaze, little endian and big endian
> >
> >    usr/microblaze-buildroot-linux-musl/sysroot/usr/include/sys/procfs.h:30:2: error: unknown type name 'elf_gregset_t'
> >    error: 'Elf64_auxv_t' undeclared (first use in this function)
> >    error: 'Elf32_auxv_t' undeclared (first use in this function)

Any idea about these? <asm/user.h> for Microblaze in musl is empty,
with just:

/* FIXME: missing in kernel? */

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Issues building gdbserver with musl

[Thomas Petazzoni]

Hello,

On Sat, 03 Jun 2017 17:19:39 +0000, mzpqnxow wrote:
> Which version of gdb are you using?

Presently, I was testing with gdb 7.11.1

> I've found I had to patch several
> versions of gdb when building gdbserver for non-x86 platforms, possibly due
> to less QA testing of these "obscure" platforms on the gdb dev side.

For all the reports I made, gdbserver was building fine on the same
architectures with other C libraries (glibc or uclibc). So there are no
architecture problems per-se, only C library problems.

> Have you had success building the same gdbserver versions with glibc or
> uClibc for the same architectures?

See above: yes.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Issues building gdbserver with musl

[mzpqnxow]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

FWIW, the following is the patch I came up with for PPC- it was very, very
short:

diff -Naur gdb-7.12/gdb/gdbserver/linux-ppc-low.c
gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c
--- gdb-7.12/gdb/gdbserver/linux-ppc-low.c 2016-08-01 11:50:20.000000000
-0400
+++ gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c 2017-05-15
13:40:22.073884258 -0400
@@ -21,7 +21,9 @@
 #include "linux-low.h"

 #include <elf.h>
+#define __ASSEMBLY__
 #include <asm/ptrace.h>
+#undef __ASSEMBLY__

 #include "nat/ppc-linux.h"
 #include "linux-ppc-tdesc.h"


On Sat, Jun 3, 2017 at 10:34 AM, Khem Raj <raj.khem@gmail.com> wrote:

> Hi Thomas
>
>
>
> On Sat, Jun 3, 2017 at 7:14 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Hello,
> >
> > We are building a number of toolchains, some of them using the musl C
> > library, and are seeing that gdbserver fails to build with musl on a
> > number of architectures. We have used musl 1.1.16, i.e the latest
> > version.
> >
> > Here are the failures that we see:
> >
> >  - On SuperH:
> >
> >    usr/sh4-buildroot-linux-musl/sysroot/usr/include/asm/ptrace_32.h:44:8:
> error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC64 and PowerPC64le:
> >
> >    usr/powerpc64-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
> >    usr/powerpc64le-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
> >
> >  - On PowerPC:
> >
> >    usr/powerpc-buildroot-linux-musl/sysroot/usr/include/asm/ptrace.h:31:8:
> error: redefinition of 'struct pt_regs'
>
>
> for ppe issues you can use something like
> http://git.yoctoproject.org/clean/cgit.cgi/poky/tree/meta/
> recipes-devtools/gdb/gdb/0004-ppc-ptrace-Define-pt_regs-
> uapi_pt_regs-on-GLIBC-syst.patch
>
> >
> >  - On Microblaze, little endian and big endian
> >
> >    usr/microblaze-buildroot-linux-musl/sysroot/usr/include/sys/procfs.h:30:2:
> error: unknown type name 'elf_gregset_t'
> >    error: 'Elf64_auxv_t' undeclared (first use in this function)
> >    error: 'Elf32_auxv_t' undeclared (first use in this function)
> >    ...
> >
> > On all those architectures, gdbserver builds fine with other C libraries.
> >
> > I had a brief look at the commits in musl since 1.1.16 and couldn't
> > spot anything that looked related.
> >
> > Are you interested in fixing those issues? If needed, I can provide an
> > easy way of reproducing those build failures.
> >
> > Thanks!
> >
> > Thomas
> > --
> > Thomas Petazzoni, CTO, Free Electrons
> > Embedded Linux and Kernel engineering
> > http://free-electrons.com
>

[-- Attachment #2: Type: text/html, Size: 4083 bytes --]

Re: Issues building gdbserver with musl

[Thomas Petazzoni]

Hello,

On Sat, 3 Jun 2017 18:50:59 -0400, mzpqnxow wrote:
> FWIW, the following is the patch I came up with for PPC- it was very, very
> short:
> 
> diff -Naur gdb-7.12/gdb/gdbserver/linux-ppc-low.c
> gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c
> --- gdb-7.12/gdb/gdbserver/linux-ppc-low.c 2016-08-01 11:50:20.000000000
> -0400
> +++ gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c 2017-05-15
> 13:40:22.073884258 -0400
> @@ -21,7 +21,9 @@
>  #include "linux-low.h"
> 
>  #include <elf.h>
> +#define __ASSEMBLY__
>  #include <asm/ptrace.h>
> +#undef __ASSEMBLY__

Can we upstream something like this? What is the explanation why musl
needs such a patch and not glibc/uclibc?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: Issues building gdbserver with musl

[Rich Felker]

On Mon, Jun 05, 2017 at 11:31:27AM +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Sat, 3 Jun 2017 18:50:59 -0400, mzpqnxow wrote:
> > FWIW, the following is the patch I came up with for PPC- it was very, very
> > short:
> > 
> > diff -Naur gdb-7.12/gdb/gdbserver/linux-ppc-low.c
> > gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c
> > --- gdb-7.12/gdb/gdbserver/linux-ppc-low.c 2016-08-01 11:50:20.000000000
> > -0400
> > +++ gdb-7.12-ppc-patched/gdb/gdbserver/linux-ppc-low.c 2017-05-15
> > 13:40:22.073884258 -0400
> > @@ -21,7 +21,9 @@
> >  #include "linux-low.h"
> > 
> >  #include <elf.h>
> > +#define __ASSEMBLY__
> >  #include <asm/ptrace.h>
> > +#undef __ASSEMBLY__
> 
> Can we upstream something like this? What is the explanation why musl
> needs such a patch and not glibc/uclibc?

Sorry for the delayed response. The reason is a historical mess. The
original intent of glibc was not to use kernel headers to define
things the libc headers need to provide, but to do their own
definitions for everything (like musl always does). But some poorly
maintained archs (that were originally maintained out-of-tree) ignored
this rule/intent and did things the old (libc5) way, using the kernel
headers.

In order to provide an API matching what glibc exposes, musl defines
pt_regs etc. in the powerpc sys/user.h. (Compare this to something
like the arm version, which has non-overlapping names.) But this
breaks if you also include the kernel header that defines pt_regs.

I'm not sure what the right solution is; there really is no good one.
Ideally musl and glibc would agree the problem needs to be fixed and
change user.h to provide definitions with new names that don't clash
with the kernel, and fix glibc's use of kernel headers. But then
applications (basically just gdb) would need to be fixed to use the
new names.

Another possible solution would be removing sys/user.h and other "just
for gdb cruft" from musl and providing it in a separate package that
depends (for some archs) on kernel headers. Or we could do something
really nasty and just say "on broken archs X, Y, and Z, sys/user.h and
other cruft headers depend on kernel headers, just like on glibc".

I'm not really sure what's best to do; that's why there's been no
progress on this issue, despite it being known and longstanding.

Rich

Re: Issues building gdbserver with musl

[A. Wilcox]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 10/06/17 21:46, Rich Felker wrote:
> In order to provide an API matching what glibc exposes, musl
> defines pt_regs etc. in the powerpc sys/user.h. (Compare this to
> something like the arm version, which has non-overlapping names.)
> But this breaks if you also include the kernel header that defines
> pt_regs.
> 
> I'm not sure what the right solution is; there really is no good
> one. Ideally musl and glibc would agree the problem needs to be
> fixed and change user.h to provide definitions with new names that
> don't clash with the kernel, and fix glibc's use of kernel headers.
> But then applications (basically just gdb) would need to be fixed
> to use the new names.


Have they been contacted to see if they'd be willing to do that?  They
seem to be doing a lot of cleanups and breaking changes in their
current dev cycle, so this would be the time to ask if I'm reading
their lists correctly.


> Another possible solution would be removing sys/user.h and other
> "just for gdb cruft" from musl and providing it in a separate
> package that depends (for some archs) on kernel headers. Or we
> could do something really nasty and just say "on broken archs X, Y,
> and Z, sys/user.h and other cruft headers depend on kernel headers,
> just like on glibc".


Is <sys/user.h> really only used by gdb?  There are no other users?  I
might actually put a #warning or #error in the file when I do the
Adélie mass tree rebuild to see if that really is the only package.
If that is the case, then this really seems like a problem that should
be solved out-of-tree (i.e. with a musl-gdb-support package or similar).

Do you have a list of the so-called "other just for gdb cruft"
headers?  I can put #warning / #error in them too, and we can test the
feasibility of moving these headers out of musl.


> I'm not really sure what's best to do; that's why there's been no 
> progress on this issue, despite it being known and longstanding.


As you might be able to tell, I am highly motivated to find a way to
fix it :)  That may be because I'm one of a few actually developing
for musl on ppc32...


> 
> Rich
> 

Best,
- --arw

- -- 
A. Wilcox (awilfox)
Project Lead, Adélie Linux
http://adelielinux.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJZPcYeAAoJEMspy1GSK50UC/QP/RjMtCnB983w+t9qiNZydKrY
dP9GZtd/3HeAZy88Fkgm9VpSCsO4/ztqnLLi9m/nOUHGuLIEEUB+4mYMc4IdEpLZ
/mV52NZqIIN/iUDH/KFAb/ejYXLreXw4LXTz3SfcsVAVT9V6fZ7CLFUdy0OPV/VA
CYz10bMZTNJJ4pRJiZuoE+mZxxkZUTMzedPvMlpicJ8bfSDKokwXHudLrM/8PiTb
i2hDyYTEKypuqOHSYxufba2yie/03EWrKWl6O9K3KbEW9MHiTo/MrpmEQGpRM/gt
/3wV6nW5Dv4YEekgg77L25HF3Q4kA7p0REoFlSppr4PvM7VvR2QbWx17inDgNV3m
vR9uT9vJE1+uxVOThZb9DAqhRzZ0PMTptf7Lyumui/5O+X5WTUzH/Xv/hdkUNJud
ol3C0Orq3wCGWf23JQJ43vhAB+RyLgMseUv802i60FCEYA1ybCg0pnON/WaQJ1x6
NCa5upvnAnQgopFAQFtxy7e2cvo+Er50IP68Lys+wiISoK4kCSEHOvtZB3Vwukrz
/sMWM8gIlT7h49sBPgh7f963TvGkHAXKWGShuKXfG/GKOb8zEq9nuaD0eGPa84Et
moVK2y7Gx4HDcXtMPba6ng5mVwEpp00W5YDjRQB66PQPeVQ9P6qAyhXMiqtE1vCl
4n04tUPD98XgQ+apO3ln
=/ktx
-----END PGP SIGNATURE-----