compiling musl on x86_64 linux with ppc

compiling musl on x86_64 linux with ppc

[u-igbb]

Hello,

I am trying to compile musl 1.1.4 on x86_64 with ppc (latest from cvs).

Unfortunately the compilation breaks with

-----------
 ...
pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/aio/aio_readwrite.o src/aio/aio_readwrite.c
./arch/x86_64/syscall_arch.h, line 37: warning: no register assignment (yet)
./arch/x86_64/syscall_arch.h, line 46: warning: no register assignment (yet)
./arch/x86_64/syscall_arch.h, line 47: warning: no register assignment (yet)
./arch/x86_64/syscall_arch.h, line 56: warning: no register assignment (yet)
./arch/x86_64/syscall_arch.h, line 57: warning: no register assignment (yet)
./arch/x86_64/syscall_arch.h, line 58: warning: no register assignment (yet)
src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
error: XXXXX/libexec/ccom terminated with status 1
make: *** [src/aio/aio_readwrite.o] Error 1
-----------

Is there any way around this, short of adding the register assignment
functionality to the compiler?

Regards,
Rune

"pcc" of course ([musl] compiling musl on x86_64 linux with ppc)

[u-igbb]

On Wed, Aug 13, 2014 at 11:18:43AM +0200, u-igbb@aetey.se wrote:
> I am trying to compile musl 1.1.4 on x86_64 with ppc (latest from cvs).

apologies for the typo, pcc (the Portable C Compiler) and nothing else,
as below:

> Unfortunately the compilation breaks with
> 
> -----------
>  ...
> pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/aio/aio_readwrite.o src/aio/aio_readwrite.c
> ./arch/x86_64/syscall_arch.h, line 37: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 46: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 47: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 56: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 57: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 58: warning: no register assignment (yet)
> src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
> error: XXXXX/libexec/ccom terminated with status 1
> make: *** [src/aio/aio_readwrite.o] Error 1
> -----------
> 
> Is there any way around this, short of adding the register assignment
> functionality to the compiler?
> 
> Regards,
> Rune

Re: compiling musl on x86_64 linux with ppc

[Szabolcs Nagy]

* u-igbb@aetey.se <u-igbb@aetey.se> [2014-08-13 11:18:43 +0200]:
> pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/aio/aio_readwrite.o src/aio/aio_readwrite.c
> ./arch/x86_64/syscall_arch.h, line 37: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 46: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 47: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 56: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 57: warning: no register assignment (yet)
> ./arch/x86_64/syscall_arch.h, line 58: warning: no register assignment (yet)
> src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
> error: XXXXX/libexec/ccom terminated with status 1
> make: *** [src/aio/aio_readwrite.o] Error 1
> -----------
> 
> Is there any way around this, short of adding the register assignment
> functionality to the compiler?
> 

there is a way around:

instead of the inline functions in syscall_arch.h use extern
asm implementations or just call the generic __syscall

(it will be less efficient than inline asm syscall, but
pcc is not very good at optimizations anyway)

eg. in syscall_arch.h change

static __inline long __syscall1(long n, long a1)
{
	unsigned long ret;
	__asm__ __volatile__ ("syscall" : "=a"(ret) : "a"(n), "D"(a1) : "rcx", "r11", "memory");
	return ret;
}

into

static __inline long __syscall1(long n, long a1)
{
	return (__syscall)(n, a1);
}

etc

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 11:49:10AM +0200, Szabolcs Nagy wrote:
> > ./arch/x86_64/syscall_arch.h, line 58: warning: no register assignment (yet)
> > src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11

> there is a way around:

> static __inline long __syscall1(long n, long a1)
> {
> 	return (__syscall)(n, a1);
> }
> 
> etc

Thanks Szabolcs, this made it!

Unfortunately I hit further problems:

 ...
pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/env/__init_tls.o src/env/__init_tls.c
src/env/__init_tls.c:112: error: #else in non-conditional section
error: XXXXXX/libexec/cpp terminated with status 1
 ...

I'll look further for the reason and a fix but if somebody has suggestions,
they will be appreciated.

Rune

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 12:25:01PM +0200, u-igbb@aetey.se wrote:
> pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/env/__init_tls.o src/env/__init_tls.c
> src/env/__init_tls.c:112: error: #else in non-conditional section
> error: XXXXXX/libexec/cpp terminated with status 1

FWIIW, looks like a limitation of the preprocessor in pcc:

   __syscall(
#ifdef SYS_mmap2

becomes after -E

   __syscall(#ifdef SYS_mmap2 ....

works around easily by moving the conditional out of the parentheses:
------
--- src/env/__init_tls.c.ori    2014-08-13 12:56:35.389032321 +0200
+++ src/env/__init_tls.c        2014-08-13 12:58:49.852815255 +0200
@@ -91,14 +91,17 @@
        libc.tls_size = 2*sizeof(void *)+T.size+T.align+sizeof(struct pthread);
 
        if (libc.tls_size > sizeof builtin_tls) {
-               mem = (void *)__syscall(
 #ifdef SYS_mmap2
+               mem = (void *)__syscall(
                        SYS_mmap2,
+                       0, libc.tls_size, PROT_READ|PROT_WRITE,
+                       MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
 #else
+               mem = (void *)__syscall(
                        SYS_mmap,
-#endif
                        0, libc.tls_size, PROT_READ|PROT_WRITE,
                        MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+#endif
                /* -4095...-1 cast to void * will crash on dereference anyway,
                 * so don't bloat the init code checking for error codes and
                 * explicitly calling a_crash(). */
------

Othwerwise, pcc does not look robust against arithmetic expressions:

pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/math/asinhl.o src/math/asinhl.c
src/math/asinhl.c, line 25: compiler error: Cannot generate code, node 0x5be9e0 op TEMP
error: XXXXXXXX/libexec/ccom terminated with status 1

src/math/asinhl.c:
24:             /* |x| >= 2 */
25:             x = logl(2*x + 1/(sqrtl(x*x+1)+x));

I will check this one on the pcc list.

Regards,
Rune

summary: [musl] compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 01:22:07PM +0200, u-igbb@aetey.se wrote:
> src/math/asinhl.c, line 25: compiler error: Cannot generate code, node 0x5be9e0 op TEMP
> error: XXXXXXXX/libexec/ccom terminated with status 1
> 
> src/math/asinhl.c:
> 24:             /* |x| >= 2 */
> 25:             x = logl(2*x + 1/(sqrtl(x*x+1)+x));

Trial and error showed that it is enough to split the evaluation
into two expressions like
   x = 2*x + 1/(sqrtl(x*x+1)+x);
   x = logl(x);
to make the compiler happy. A similar change is necessary on another
line in the same file too.

Given the tweaks in

 arch/x86_64/syscall_arch.h     // let all __syscallN() call __syscall()
 src/env/__init_tls.c           // move #if conditional outside ()
 src/math/asinhl.c              // split log*() expressions

the compilation went through.

Rune

Re: compiling musl on x86_64 linux with ppc

[Rich Felker]

On Wed, Aug 13, 2014 at 11:18:43AM +0200, u-igbb@aetey.se wrote:
> Hello,
> 
> I am trying to compile musl 1.1.4 on x86_64 with ppc (latest from cvs).

Are you sure it's latest? If you're using a snapshot there are
multiple branches (iirc 1.0.DEVEL and 1.1.DEVEL or something like
that) and the old one is very outdated and broken.

> Unfortunately the compilation breaks with
> 
> -----------
>  ...
> pcc -std=c99 -nostdinc -ffreestanding -fexcess-precision=standard -frounding-math -D_XOPEN_SOURCE=700 -I./arch/x86_64 -I./src/internal -I./include  -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -Wa,--noexecstack -Werror=implicit-function-declaration -Werror=implicit-int -Werror=pointer-sign -Werror=pointer-arith -fno-stack-protector  -c -o src/aio/aio_readwrite.o src/aio/aio_readwrite.c
> ../arch/x86_64/syscall_arch.h, line 37: warning: no register assignment (yet)
> ../arch/x86_64/syscall_arch.h, line 46: warning: no register assignment (yet)
> ../arch/x86_64/syscall_arch.h, line 47: warning: no register assignment (yet)
> ../arch/x86_64/syscall_arch.h, line 56: warning: no register assignment (yet)
> ../arch/x86_64/syscall_arch.h, line 57: warning: no register assignment (yet)
> ../arch/x86_64/syscall_arch.h, line 58: warning: no register assignment (yet)
> src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
> error: XXXXX/libexec/ccom terminated with status 1
> make: *** [src/aio/aio_readwrite.o] Error 1
> -----------
> 
> Is there any way around this, short of adding the register assignment
> functionality to the compiler?

Just use the #ifdef __clang__ branch of the code which handles the
same brokenness in clang. :-)

Rich

Re: compiling musl on x86_64 linux with ppc

[u-igbb]

On Wed, Aug 13, 2014 at 08:38:32AM -0400, Rich Felker wrote:
> > I am trying to compile musl 1.1.4 on x86_64 with ppc (latest from cvs).
> 
> Are you sure it's latest? If you're using a snapshot there are
> multiple branches (iirc 1.0.DEVEL and 1.1.DEVEL or something like
> that) and the old one is very outdated and broken.

I did my best to locate the latest one,
it seems to present itself as 1.1.0.DEVEL

> > src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
> > error: XXXXX/libexec/ccom terminated with status 1
> > make: *** [src/aio/aio_readwrite.o] Error 1
> > -----------
> > 
> > Is there any way around this, short of adding the register assignment
> > functionality to the compiler?
> 
> Just use the #ifdef __clang__ branch of the code which handles the
> same brokenness in clang. :-)

According to Anders Magnusson (he replied on pcc-list) it is sufficient
to refer to r11 as %r11 to make it work. He thinks otherwise of adding
the alias r11 for %r11 in pcc.

Indeed, with a change of all "r11" to "%r11" in
 arch/x86_64/syscall_arch.h
the compilation went through without disabling the separate __syscallN().

My goal is a musl-based selfcontained pcc toolchain.
I'd like to also have a tcc-based one but musl seems to be too hard
a target for tcc yet :( sigh.

Rune

Re: compiling musl on x86_64 linux with ppc

[Rich Felker]

On Wed, Aug 13, 2014 at 02:56:08PM +0200, u-igbb@aetey.se wrote:
> > > src/aio/aio_readwrite.c, line 23: compiler error: unsupported xasm constraint r11
> > > error: XXXXX/libexec/ccom terminated with status 1
> > > make: *** [src/aio/aio_readwrite.o] Error 1
> > > -----------
> > > 
> > > Is there any way around this, short of adding the register assignment
> > > functionality to the compiler?
> > 
> > Just use the #ifdef __clang__ branch of the code which handles the
> > same brokenness in clang. :-)
> 
> According to Anders Magnusson (he replied on pcc-list) it is sufficient
> to refer to r11 as %r11 to make it work. He thinks otherwise of adding
> the alias r11 for %r11 in pcc.
> 
> Indeed, with a change of all "r11" to "%r11" in
>  arch/x86_64/syscall_arch.h
> the compilation went through without disabling the separate __syscallN().

Just because it compiled doesn't mean it works. Those warnings about
register assignment definitely should be hard errors because the code
will do horribly wrong things if the requested constraints are not
met.

> My goal is a musl-based selfcontained pcc toolchain.
> I'd like to also have a tcc-based one but musl seems to be too hard
> a target for tcc yet :( sigh.

I think tcc has a long ways to go... But pcc was working in the past
(at least on 32-bit) and should work.

BTW about the mmap2 preprocessor issue, did you raise that with ppc
developers too? I think it's a bug on their side but I'm not 100%
sure. If the code in musl is not valid C I'll change it, but if it is,
I think we should just wait for pcc to get fixed.

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 10:23:32AM -0400, Rich Felker wrote:
> > Indeed, with a change of all "r11" to "%r11" in
> >  arch/x86_64/syscall_arch.h
> > the compilation went through without disabling the separate __syscallN().
> 
> Just because it compiled doesn't mean it works. Those warnings about
> register assignment definitely should be hard errors because the code
> will do horribly wrong things if the requested constraints are not
> met.

I suspected this, did not try to run that code.

> I think tcc has a long ways to go... But pcc was working in the past
> (at least on 32-bit) and should work.

I'll report the progress when I come further.

> BTW about the mmap2 preprocessor issue, did you raise that with ppc
> developers too? I think it's a bug on their side but I'm not 100%
> sure. If the code in musl is not valid C I'll change it, but if it is,
> I think we should just wait for pcc to get fixed.

Anders wrote:

>>     __syscall(
>> #ifdef SYS_mmap2
>>
>> becomes after -E
>>
>>     __syscall(#ifdef SYS_mmap2 ....
>Having conditionals inside macro lists is undefined, see 6.10.3 clause 11.
>Here I recommend fixing the code instead.

I think this means the code should be changed.

Regards,
Rune

Re: compiling musl on x86_64 linux with ppc

[Szabolcs Nagy]

* Rich Felker <dalias@libc.org> [2014-08-13 10:23:32 -0400]:
> BTW about the mmap2 preprocessor issue, did you raise that with ppc
> developers too? I think it's a bug on their side but I'm not 100%
> sure. If the code in musl is not valid C I'll change it, but if it is,

it is not valid c

http://port70.net/~nsz/c/c99/n1256.html#6.10.3p11

i'd fix it like

+#ifndef SYS_mmap2
+#define SYS_mmap2 SYS_mmap
+#endif
		mem = (void *)__syscall(
-#ifdef SYS_mmap2
			SYS_mmap2,
-#else
-			SYS_mmap,
-#endif

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 10:23:32AM -0400, Rich Felker wrote:
> > I'd like to also have a tcc-based one but musl seems to be too hard
> > a target for tcc yet :( sigh.
> 
> I think tcc has a long ways to go... But pcc was working in the past
> (at least on 32-bit) and should work.

AFAICT tcc can not even _use_ musl (i.e. build programs to be linked to
musl) as musl relies on __builtin_va_list.

That's a pity. Wonder whether it would cost too much to make
__builtin_va_list non-mandatory?

Rune

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Thu, Aug 14, 2014 at 09:10:55AM +0200, u-igbb@aetey.se wrote:
> On Wed, Aug 13, 2014 at 10:23:32AM -0400, Rich Felker wrote:
> > > I'd like to also have a tcc-based one but musl seems to be too hard
> > > a target for tcc yet :( sigh.
> > 
> > I think tcc has a long ways to go... But pcc was working in the past
> > (at least on 32-bit) and should work.
> 
> AFAICT tcc can not even _use_ musl (i.e. build programs to be linked to
> musl) as musl relies on __builtin_va_list.
> 
> That's a pity. Wonder whether it would cost too much to make
> __builtin_va_list non-mandatory?

How else would you do it? There's no real alternative.

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Thu, Aug 14, 2014 at 10:20:56AM -0400, Rich Felker wrote:
> > AFAICT tcc can not even _use_ musl (i.e. build programs to be linked to
> > musl) as musl relies on __builtin_va_list.
> > 
> > That's a pity. Wonder whether it would cost too much to make
> > __builtin_va_list non-mandatory?
> 
> How else would you do it? There's no real alternative.

Well, tcc works e.g. with uclibc, which means this is possible,
even though I did not look at "how".

Rune

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Thu, Aug 14, 2014 at 04:38:38PM +0200, u-igbb@aetey.se wrote:
> On Thu, Aug 14, 2014 at 10:20:56AM -0400, Rich Felker wrote:
> > > AFAICT tcc can not even _use_ musl (i.e. build programs to be linked to
> > > musl) as musl relies on __builtin_va_list.
> > > 
> > > That's a pity. Wonder whether it would cost too much to make
> > > __builtin_va_list non-mandatory?
> > 
> > How else would you do it? There's no real alternative.
> 
> Well, tcc works e.g. with uclibc, which means this is possible,
> even though I did not look at "how".

Apparently tcc has its own stdarg.h which uses the nonsense legacy
definition for i386 where the arguments are assumed to lie at fixed
locations on the stack; this assumption is invalid because the
compiler need not reuse the passed-in storage as the local object, and
in the case of inlining there won't even be any passed-in storage. But
it probably works for tcc since tcc sucks. See the __GNUC__>=3
condition in musl's stdarg.h. Disabling that (perhaps adding
&&!__TCC__ or similar) should make tcc work with musl on i386, but
x86_64 is harder since there is no legacy definition and tcc did its
own craziness that relies on external functions named __va_*.

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Thu, Aug 14, 2014 at 10:47:02AM -0400, Rich Felker wrote:
> > > > __builtin_va_list non-mandatory?
> > > 
> > > How else would you do it? There's no real alternative.
> > 
> > Well, tcc works e.g. with uclibc, which means this is possible,
> > even though I did not look at "how".
> 
> Apparently tcc has its own stdarg.h which uses the nonsense legacy
> definition for i386 where the arguments are assumed to lie at fixed
> locations on the stack; this assumption is invalid because the
> compiler need not reuse the passed-in storage as the local object, and
> in the case of inlining there won't even be any passed-in storage. But
> it probably works for tcc since tcc sucks.

:)

> See the __GNUC__>=3
> condition in musl's stdarg.h. Disabling that (perhaps adding
> &&!__TCC__ or similar) should make tcc work with musl on i386, but

This is a good news (modulo the limitations) and at least some benefit
from the legacy stuff - tcc is invaluable for rapid prototyping.

> x86_64 is harder since there is no legacy definition and tcc did its
> own craziness that relies on external functions named __va_*.

This is a bad news.

Thanks for the explanation Rich,

Rune

va_list (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

On Thu, Aug 14, 2014 at 10:47:02AM -0400, Rich Felker wrote:
> > > > That's a pity. Wonder whether it would cost too much to make
> > > > __builtin_va_list non-mandatory?
> > > 
> > > How else would you do it? There's no real alternative.

To be able to mix object files and/or libraries compiled with different
compilers apparently there must be a standard ABI.

This seems also to be reflected in

 https://stackoverflow.com/questions/4958384/what-is-the-format-of-the-x86-64-va-list-structure

and (referred from there)

 http://www.x86-64.org/documentation/abi.pdf

I may be missing something but it looks like this ABI can not be an opaque
fully "compiler's internal business". Compilers may implement as much
optimizations as they wish but they must be able to produce interoperable
object files, don't they?

I would appreciate to be enlightened on this matter, what is the obstacle
to adding a compiler-independent definition of va_list (or otherwise
possibly even using the compiler-specific non-builtin definition, like
the one of tcc)?

This of course does not have to be used when the presumably more efficient
builtin is available.

Rune

Re: va_list (was: [musl] compiling musl on x86_64 linux with pcc)

[Rich Felker]

On Fri, Aug 15, 2014 at 12:49:36PM +0200, u-igbb@aetey.se wrote:
> On Thu, Aug 14, 2014 at 10:47:02AM -0400, Rich Felker wrote:
> > > > > That's a pity. Wonder whether it would cost too much to make
> > > > > __builtin_va_list non-mandatory?
> > > > 
> > > > How else would you do it? There's no real alternative.
> 
> To be able to mix object files and/or libraries compiled with different
> compilers apparently there must be a standard ABI.

C code has absolutely no access to the ABI. There is no way to write,
in C, "offset X from the stack pointer", and even if there were (e.g.
via inline asm), there is no way to know the current offset of the
stack pointer relative to what it was at function entry.

> This seems also to be reflected in
> 
>  https://stackoverflow.com/questions/4958384/what-is-the-format-of-the-x86-64-va-list-structure
> 
> and (referred from there)
> 
>  http://www.x86-64.org/documentation/abi.pdf
> 
> I may be missing something but it looks like this ABI can not be an opaque
> fully "compiler's internal business". Compilers may implement as much
> optimizations as they wish but they must be able to produce interoperable
> object files, don't they?

Yes, but this all pertains to calls between (external or potentially
externally visible via function pointer) functions. It has nothing to
do with what happens inside the function once its called.

I'll try to explain with the (wrong) legacy stdarg.h macro definitions
for i386 (much simpler) as an example. In

void foo(int a, ...)
{
	va_list ap;
	va_start(ap, a);
	int b = va_arg(ap, int);
	...
}

the legacy macros are taking the address of a, adding 4 to it, and
using the result to access the first variadic argument which is then
stored in b. However, there is utterly no relationship between the
address of a and the location where the variadic argument is stored!

There is a relationship between the addresses where the _value_ which
is stored in the _local, automatic variable_ a is passed on the stack,
and where the value for the first variadic argument is passed. But
there is no "is" relationship between the object a and the storage
where this value is passed. E.g. in the following function:

int bar(int n)
{
	int n0 = n;
	...
	/* maybe use &n ? */
	...
	return n-n0;
}

a very reasonable compiler would choose to use the original storage on
the stack where the argument was passed as n0, and create new storage
for n, possibly even keeping it in a register and never creating space
for it on the stack unless &n is actually used and possibly
dereferenced.

Other compilers might _always_ create new storage for arguments just
like other automatic objects, and simply use the values passed on the
stack to initialize them.

> I would appreciate to be enlightened on this matter, what is the obstacle
> to adding a compiler-independent definition of va_list (or otherwise
> possibly even using the compiler-specific non-builtin definition, like
> the one of tcc)?

So there is fundamentally no way to write a "portable" stdarg.h or
even a "compiler independent" one for a specific ABI, because it has
nothing to do with the ABI. The correct implementation has to do with
how the compiler generates code that's entirely _internal_ to the
function being compiled.

Further, variadic functions need not be external, externally callable,
or even existant as independent functions. If the above example foo is
inlined, the object a need not even have an address; if it does, it
would necessarily be created at some new location in the caller's
stack frame, not as part of a list of incoming argument values on the
stack.

> This of course does not have to be used when the presumably more efficient
> builtin is available.

It's not a matter of efficiency but correctness. The __builtin version
is compatible with any compiler that implements it, and any non-crap
(e.g. supporting inline, and making any nontrivial optimizations)
compiler must implement it or something equivalent with a gratuitously
different name. The legacy definition for i386 only works for utterly
idiotic compilers like tcc (and maybe gcc 2.x) that use a fixed
mechanical transformation to machine code for the _internals_ of the
variadic function being compiled.

Rich

Re: va_list (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

Hello Rich,

On Fri, Aug 15, 2014 at 09:44:34AM -0400, Rich Felker wrote:
> >  http://www.x86-64.org/documentation/abi.pdf
> > 
> > I may be missing something but it looks like this ABI can not be an opaque
> > fully "compiler's internal business". Compilers may implement as much
> > optimizations as they wish but they must be able to produce interoperable
> > object files, don't they?
> 
> Yes, but this all pertains to calls between (external or potentially
> externally visible via function pointer) functions. It has nothing to
> do with what happens inside the function once its called.

> I'll try to explain with the (wrong) legacy stdarg.h macro definitions
> for i386 (much simpler) as an example. In

Thanks for the detailed explanation.

> void foo(int a, ...)
> {
> 	va_list ap;
> 	va_start(ap, a);
> 	int b = va_arg(ap, int);
> 	...
> }
> 
> the legacy macros are taking the address of a, adding 4 to it, and
> using the result to access the first variadic argument which is then
> stored in b. However, there is utterly no relationship between the
> address of a and the location where the variadic argument is stored!

> There is a relationship between the addresses where the _value_ which
> is stored in the _local, automatic variable_ a is passed on the stack,

I see. This is certainly the internal business of the compiler.
Nevertheless, as soon as a compiler offers an implementation
of finding the location of variadic variables, then it is bound
by its promises and has to generate code which corresponds to
this implementation.

More advanced compilers implement this as builtins, less advanced
still may choose to implement this "in C" in stdarg.h without breaking
compatibility with other compilers (as long as the actual passing of
arguments conforms to the ABI).

So I guess nothing bad should happen and it could "just work"
if tcc finds its stdarg.h instead of the musl one.

Testing... I can compile with tcc a file calling printf, link
with musl and successfully run it. Nice!

(Hmm, bits/alltypes defines ...va_list "instead of including stdarg.h",
I guess it could be made to include, guarded by some #if defined() ?

Besides this detail, it was apparently just a matter of wrapping tcc
with "-I<where-the-tcc-stdarg.h-alone-lives> \
      -D__DEFINED_va_list \
      -D__isoc_va_list=va_list \
      -D__DEFINED___isoc_va_list"
(this part is of course not of concern for musl, besides preserving the
possibility to externally define the types, in a compiler-specific stdarg.h)

I think this is a correct approach which makes musl usable with
more compilers than otherwise.

Regards,
Rune

Re: va_list (was: [musl] compiling musl on x86_64 linux with pcc)

[Rich Felker]

On Fri, Aug 15, 2014 at 05:07:23PM +0200, u-igbb@aetey.se wrote:
> Hello Rich,
> 
> On Fri, Aug 15, 2014 at 09:44:34AM -0400, Rich Felker wrote:
> > >  http://www.x86-64.org/documentation/abi.pdf
> > > 
> > > I may be missing something but it looks like this ABI can not be an opaque
> > > fully "compiler's internal business". Compilers may implement as much
> > > optimizations as they wish but they must be able to produce interoperable
> > > object files, don't they?
> > 
> > Yes, but this all pertains to calls between (external or potentially
> > externally visible via function pointer) functions. It has nothing to
> > do with what happens inside the function once its called.
> 
> > I'll try to explain with the (wrong) legacy stdarg.h macro definitions
> > for i386 (much simpler) as an example. In
> 
> Thanks for the detailed explanation.
> 
> > void foo(int a, ...)
> > {
> > 	va_list ap;
> > 	va_start(ap, a);
> > 	int b = va_arg(ap, int);
> > 	...
> > }
> > 
> > the legacy macros are taking the address of a, adding 4 to it, and
> > using the result to access the first variadic argument which is then
> > stored in b. However, there is utterly no relationship between the
> > address of a and the location where the variadic argument is stored!
> 
> > There is a relationship between the addresses where the _value_ which
> > is stored in the _local, automatic variable_ a is passed on the stack,
> 
> I see. This is certainly the internal business of the compiler.
> Nevertheless, as soon as a compiler offers an implementation
> of finding the location of variadic variables, then it is bound
> by its promises and has to generate code which corresponds to
> this implementation.
> 
> More advanced compilers implement this as builtins, less advanced
> still may choose to implement this "in C" in stdarg.h without breaking
> compatibility with other compilers (as long as the actual passing of
> arguments conforms to the ABI).
> 
> So I guess nothing bad should happen and it could "just work"
> if tcc finds its stdarg.h instead of the musl one.
> 
> Testing... I can compile with tcc a file calling printf, link
> with musl and successfully run it. Nice!
> 
> (Hmm, bits/alltypes defines ...va_list "instead of including stdarg.h",
> I guess it could be made to include, guarded by some #if defined() ?

No, this is the nasty gcc way which precludes the compiler from
optimizing out multiple inclusions of the same header file, and which
requires the libc headers and headers provided by the compiler to be
aware of each other's implementation details, which is not really
appropriate.

> Besides this detail, it was apparently just a matter of wrapping tcc
> with "-I<where-the-tcc-stdarg.h-alone-lives> \
>       -D__DEFINED_va_list \
>       -D__isoc_va_list=va_list \
>       -D__DEFINED___isoc_va_list"
> (this part is of course not of concern for musl, besides preserving the
> possibility to externally define the types, in a compiler-specific stdarg.h)

This is not supported usage with musl.

> I think this is a correct approach which makes musl usable with
> more compilers than otherwise.

Only tcc, and tcc really just needs to be fixed in this regard. All
the other compilers do it right already. It would not be hard to add
the builtins and have them do the right thing, and this is essentially
needed for x86_64 anyway -- the current approach of calling external
functions is totally inappropriate since the generated .o files are
not compatible with the ABI (which does not define such external
functions) and cannot be successfully linked by non-tcc toolchains.

Rich

Re: va_list (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

On Fri, Aug 15, 2014 at 11:56:56AM -0400, Rich Felker wrote:
> > Testing... I can compile with tcc a file calling printf, link
> > with musl and successfully run it. Nice!
> > 
> > (Hmm, bits/alltypes defines ...va_list "instead of including stdarg.h",
> > I guess it could be made to include, guarded by some #if defined() ?
> 
> No, this is the nasty gcc way which precludes the compiler from
> optimizing out multiple inclusions of the same header file, and which
> requires the libc headers and headers provided by the compiler to be
> aware of each other's implementation details, which is not really
> appropriate.

Fine with me, it is just a matter to teach tcc to implicitly include its
stdarg.h.

> > Besides this detail, it was apparently just a matter of wrapping tcc
> > with "-I<where-the-tcc-stdarg.h-alone-lives> \
> >       -D__DEFINED_va_list \
> >       -D__isoc_va_list=va_list \
> >       -D__DEFINED___isoc_va_list"
> > (this part is of course not of concern for musl, besides preserving the
> > possibility to externally define the types, in a compiler-specific stdarg.h)
> 
> This is not supported usage with musl.

Surely, I am using the internal details which is bad - it would be
nice if musl would allow a cleaner way of doing a corresponding
hook.

> > I think this is a correct approach which makes musl usable with
> > more compilers than otherwise.
> 
> Only tcc, and tcc really just needs to be fixed in this regard. All

This of course would be the best solution - but tcc is as it is, with
the limitations and the nice properties. I'd like to be able to use it,
or if anybody hacks a differently nice / differently bad compiler,
to be able to use it too.

This does not look like putting any constraints on musl efficienly or
correctness - the only thing which is necessary is a (possibly even
musl-version-dependent) way to skip/replace the va-definitions. I think
it is no practical problem, good enough, unless you decide to radically
change the implementation.

> the other compilers do it right already. It would not be hard to add
> the builtins and have them do the right thing, and this is essentially
> needed for x86_64 anyway -- the current approach of calling external
> functions is totally inappropriate since the generated .o files are
> not compatible with the ABI (which does not define such external
> functions) and cannot be successfully linked by non-tcc toolchains.

Hmm. Yes that's right (non-tcc-"aware" toolchains that is - no problem
to arrange non-tcc linking to work, but you can not give such object files
to others to link...). To be compatible, a compiler-specific implementation
must be effectively inlined in every .o file, which tcc does not do
(and which builtins of course do, but they are nevertheless not the only
valid solution - even though they seem to be a good one :)

Thanks for clearing the matter!

Rune

Re: compiling musl on x86_64 linux with ppc

[u-igbb]

On Wed, Aug 13, 2014 at 10:23:32AM -0400, Rich Felker wrote:
> pcc was working in the past
> (at least on 32-bit) and should work.

Unfortunately since then pcc acquired a bug which is a showstopper
until fixed - "xxx\0yyy" (\0 bytes in string literals) leads to
badly distorted string data. The developers are aware of this and
hopefully in several days pcc and musl will be able to coexist again.
(it was the introduction of support for utf-8 which caused the regression)

Rune

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Wed, Aug 13, 2014 at 10:23:32AM -0400, Rich Felker wrote:
 ...
> pcc was working in the past
> (at least on 32-bit) and should work.

Unfortunately (even after the fix in pcc for "\0") musl seems to be
miscompiled by pcc of 2014-08-24, tried static on x86_64 and dynamic on
i386, with musl 1.1.4.

Do you have a reference to which versions of musl and pcc
did work together? (May be the musl compilation went through, like
it does now, but the resulting library was not fully healthy either?)

Rune

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Mon, Aug 25, 2014 at 10:28:08AM +0200, u-igbb@aetey.se wrote:
> Do you have a reference to which versions of musl and pcc
> did work together? (May be the musl compilation went through, like
> it does now, but the resulting library was not fully healthy either?)

Found http://www.etalabs.net/musl/oldversions.html which says
that musl-0.7.10.tar.gz - May 17, 2011 was good with pcc (presumably
of the same time). I guess too much happened since then on both sides.

:(

Rune

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Mon, Aug 25, 2014 at 10:34:57AM +0200, u-igbb@aetey.se wrote:
> On Mon, Aug 25, 2014 at 10:28:08AM +0200, u-igbb@aetey.se wrote:
> > Do you have a reference to which versions of musl and pcc
> > did work together? (May be the musl compilation went through, like
> > it does now, but the resulting library was not fully healthy either?)
> 
> Found http://www.etalabs.net/musl/oldversions.html which says
> that musl-0.7.10.tar.gz - May 17, 2011 was good with pcc (presumably
> of the same time). I guess too much happened since then on both sides.

I have pcc 1.1.0.DEVEL 20130227 with pcc-libs-20130721 and it seems to
still work fine with current musl. Note that some fixes are needed for
pcc-libs:

- You need to remove the broken csu files and replace them with
  symlinks to the musl files.

- You need to add __attribute__((__visibility__("hidden"))) to all
  external functions in libpcc.

- You need to add -fPIC to the CFLAGS for building libpcc or you get a
  broken libpcc (containing textrels) that can't be used for building
  libc.so and that's harmful for use with any other shared libs.

I just ran libc-test with libc.so built by pcc and confirmed that
there are no non-math regressions versus gcc-built libc.so. (I did not
check math in detail because there are lots of known math failures
that are not serious.)

Let me know if you have any further questions about my setup.

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

Hello Rich,

On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> I have pcc 1.1.0.DEVEL 20130227 with pcc-libs-20130721 and it seems to
> still work fine with current musl. Note that some fixes are needed for
> pcc-libs:
> 
> - You need to remove the broken csu files and replace them with
>   symlinks to the musl files.

I do not even generate crt[01in].o in pcclibs and of course use
the musl ones.

> - You need to add __attribute__((__visibility__("hidden"))) to all
>   external functions in libpcc.

This does not seem to make any difference (and probably would not 
be noticeable with simple tests anyway?).

> - You need to add -fPIC to the CFLAGS for building libpcc or you get a
>   broken libpcc (containing textrels) that can't be used for building
>   libc.so and that's harmful for use with any other shared libs.

I do.

Anyway, the brokenness is present with both static and dynamic linking.

(testing on i486-pc-linux-gnu)

I narrowed down what looks broken:
*printf() with "%something" seems to pick wrong arguments,
iow it looks like malfunctioning variadic args. (?)

> I just ran libc-test with libc.so built by pcc and confirmed that
> there are no non-math regressions versus gcc-built libc.so. (I did not
> check math in detail because there are lots of known math failures
> that are not serious.)
> 
> Let me know if you have any further questions about my setup.

Is it possible to try pcc from 2014-08-24 in your environment?

If it is hard, I can test the old versions instead (when I can, right now
my time slice is over).

Regards,
Rune

PS
(this variadic business reminds me again that stdarg.h should be provided
by the compiler who knows what it can do to the generated functions and
what it must do with the public ones - not by the library which can be
used with different compilers and should not boldly assume the level of
their intelligence)

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Tue, Aug 26, 2014 at 09:34:39PM +0200, u-igbb@aetey.se wrote:
> Hello Rich,
> 
> On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> > I have pcc 1.1.0.DEVEL 20130227 with pcc-libs-20130721 and it seems to
> > still work fine with current musl. Note that some fixes are needed for
> > pcc-libs:
> > 
> > - You need to remove the broken csu files and replace them with
> >   symlinks to the musl files.
> 
> I do not even generate crt[01in].o in pcclibs and of course use
> the musl ones.

OK, just checking.

> > - You need to add __attribute__((__visibility__("hidden"))) to all
> >   external functions in libpcc.
> 
> This does not seem to make any difference

It does. Failure to fix this will result in these symbols becoming
part of the public ABI of any .so file that pulls them in, and will
thereby prevent them from being linked into apps that use the .so file
(since the app will already have its need for them satisfied). If the
.so file later changes such that it no longer needs the functions (or
if it's upgraded to a version that's correctly linked with a libpcc.a
where these symbols are hidden, then such apps will break.

> (and probably would not 
> be noticeable with simple tests anyway?).

I agree that it would not be noticed right away, but it is important.

> > - You need to add -fPIC to the CFLAGS for building libpcc or you get a
> >   broken libpcc (containing textrels) that can't be used for building
> >   libc.so and that's harmful for use with any other shared libs.
> 
> I do.
> 
> Anyway, the brokenness is present with both static and dynamic linking.
> 
> (testing on i486-pc-linux-gnu)
> 
> I narrowed down what looks broken:
> *printf() with "%something" seems to pick wrong arguments,
> iow it looks like malfunctioning variadic args. (?)

I have not seen anything like that happening.

> > I just ran libc-test with libc.so built by pcc and confirmed that
> > there are no non-math regressions versus gcc-built libc.so. (I did not
> > check math in detail because there are lots of known math failures
> > that are not serious.)
> > 
> > Let me know if you have any further questions about my setup.
> 
> Is it possible to try pcc from 2014-08-24 in your environment?
> 
> If it is hard, I can test the old versions instead (when I can, right now
> my time slice is over).

I can, but at at later time. I've got lots of other things I'm trying
to get done without adding pcc testing right now. So if you want to go
ahead and try it, go ahead.

> PS
> (this variadic business reminds me again that stdarg.h should be provided
> by the compiler who knows what it can do to the generated functions and
> what it must do with the public ones - not by the library which can be
> used with different compilers and should not boldly assume the level of
> their intelligence)

This is unrelatd. pcc's version would be exactly the same thing, and
in fact, as I explained before, there is only one correct
implementation: builtin functions like __builtin_va_arg, etc. but
possibly with gratuitously different names or argument orders. In any
case there is no way to simply "let the compiler do it" because
certain other headers need to access or expose the va_list type, and
doing this with a compiler-provided stdarg.h requires knowledge of the
compiler versions internals in the libc headers, which is much worse
than the current situation.

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Tue, Aug 26, 2014 at 03:54:07PM -0400, Rich Felker wrote:
> > > - You need to add __attribute__((__visibility__("hidden"))) to all
> > >   external functions in libpcc.
> > 
> > This does not seem to make any difference
> 
> It does. Failure to fix this will result in these symbols becoming
> part of the public ABI of any .so file that pulls them in, and will
> thereby prevent them from being linked into apps that use the .so file
> (since the app will already have its need for them satisfied). If the
> .so file later changes such that it no longer needs the functions (or
> if it's upgraded to a version that's correctly linked with a libpcc.a
> where these symbols are hidden, then such apps will break.
> 
> > (and probably would not 
> > be noticeable with simple tests anyway?).
> 
> I agree that it would not be noticed right away, but it is important.

I see, I meant only that this change did not improve the situation, not
that it was unnecessary. The compiler-related libraries are logically
just a part of the generated code which is chosen not to be inlined, iow
either the ABI to them should be stable (and the library available as
both static and shared) or the referred funtions should be implicitly
inserted into each object file. I guess the "hidden" magic approximates
this well enough (ensuring a single copy of the statically linked
functions in dynamically linked executables).

> > I narrowed down what looks broken:
> > *printf() with "%something" seems to pick wrong arguments,
> > iow it looks like malfunctioning variadic args. (?)
> 
> I have not seen anything like that happening.

Ok.

> > (this variadic business reminds me again that stdarg.h should be provided
> > by the compiler who knows what it can do to the generated functions and
> > what it must do with the public ones - not by the library which can be
> > used with different compilers and should not boldly assume the level of
> > their intelligence)
> 
> This is unrelatd. pcc's version would be exactly the same thing, and

Yes it is more general but when I thought about troubleshooting I realized
I have to account for what the library / the compiler believe about the
variadic arguments and whether these views are compatible.

> in fact, as I explained before, there is only one correct
> implementation: builtin functions like __builtin_va_arg, etc. but
> possibly with gratuitously different names or argument orders. In any
> case there is no way to simply "let the compiler do it" because
> certain other headers need to access or expose the va_list type, and
> doing this with a compiler-provided stdarg.h requires knowledge of the
> compiler versions internals in the libc headers, which is much worse
> than the current situation.

The current situation is (!) actually using knowledge of the compiler
in the library headers (checking for gcc version, as you noted when we
were talking about tcc), which shouldn't be needed if the library would
skip referring to the intimate knowledge of those details.

I am possibly not realizing some crucial bit but I do not see why the
implementation of stdarg.h must be the business of the library. If the
compiler *building* the library wants the va_arg to be __builtin_va_arg,
then you get the same functionality / efficiency as now, otherwise you
get a less efficient but still fully compatible library, don't you?
A compiler used *together* with the library may lack the builtins but it
still can be fully compatible even if in some situations less efficient.

No compiler (if ABI-compatible) is allowed to produce code unusable when
linked to code compiled by another compiler. How allowing the compiler
to pick its own stdarg.h would lead to a less efficient code than otherwise?

Rune

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Wed, Aug 27, 2014 at 09:40:36AM +0200, u-igbb@aetey.se wrote:
> > in fact, as I explained before, there is only one correct
> > implementation: builtin functions like __builtin_va_arg, etc. but
> > possibly with gratuitously different names or argument orders. In any
> > case there is no way to simply "let the compiler do it" because
> > certain other headers need to access or expose the va_list type, and
> > doing this with a compiler-provided stdarg.h requires knowledge of the
> > compiler versions internals in the libc headers, which is much worse
> > than the current situation.
> 
> The current situation is (!) actually using knowledge of the compiler
> in the library headers (checking for gcc version, as you noted when we
> were talking about tcc), which shouldn't be needed if the library would
> skip referring to the intimate knowledge of those details.

This is only a hack that's left in the i386 headers for supporting
ancient gcc versions that probably have not worked right for more than
a decade. It should probably be removed, and bits/stdarg.h removed.

> I am possibly not realizing some crucial bit but I do not see why the
> implementation of stdarg.h must be the business of the library. If the

Because other headers need to know how to get va_list defined for use
in function prototypes, and they can't include stdarg.h because it
would expose more than they want. glibc handles this by making use of
a special inclusion protocol between the glibc headers and the gcc
headers: it defines a special macro before including stdarg.h telling
gcc that it just wants the definition of va_list and nothing else.

> compiler *building* the library wants the va_arg to be __builtin_va_arg,
> then you get the same functionality / efficiency as now, otherwise you
> get a less efficient but still fully compatible library, don't you?

No. This has absolutely nothing to do with efficiency.

> A compiler used *together* with the library may lack the builtins but it
> still can be fully compatible even if in some situations less efficient.

No, it can't. There is simply no way to express, in C, the concept of
"get the next variadic argument" without __builtin_va_arg or something
100% equivalent to it. If you have a compiler than chooses to be
gratuitously incompatible and use a different name for its builtin,
you can always add -D'__builtin_va_arg(x)=__foo(x)' etc. to your
CFLAGS when using that compiler. However no such compiler seems to
exist.

> No compiler (if ABI-compatible) is allowed to produce code unusable when
> linked to code compiled by another compiler. How allowing the compiler
> to pick its own stdarg.h would lead to a less efficient code than otherwise?

Again, this has nothing to do with ABI between external variadic
functions. It's a matter of the inability to express to the compiler
what you want it to do (get the next variadic argument) in plain C.

Rich

variadic args (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

On Wed, Aug 27, 2014 at 03:54:07AM -0400, Rich Felker wrote:
> > I am possibly not realizing some crucial bit but I do not see why the
> > implementation of stdarg.h must be the business of the library. If the
> 
> Because other headers need to know how to get va_list defined for use
> in function prototypes, and they can't include stdarg.h because it
> would expose more than they want. glibc handles this by making use of

I guess this is the matter which everything revolves around.
What is it that stdarg.h would expose too much?

> a special inclusion protocol between the glibc headers and the gcc
> headers: it defines a special macro before including stdarg.h telling
> gcc that it just wants the definition of va_list and nothing else.

Wonder whether it would be reasonable to say that musl expects from
every compiler to be used with the library either
a stdarg.h with following properties: ...
 <which ones?>
otherwise expecting the compiler to implement the following builtins:
 <the current contents of musl's stdarg.h>

I do not see how this would hurt in any way.

By the way, this - in any case - *is* an interface against the compiler.

As a matter of fact, the bultins naming differs between compilers.

So the current musl approach is to just boldly assume a *certain*
relatively arbitrary interface, not "the only possible" nor "the only
reasonable one".

> > A compiler used *together* with the library may lack the builtins but it
> > still can be fully compatible even if in some situations less efficient.
> 
> No, it can't. There is simply no way to express, in C, the concept of
> "get the next variadic argument" without __builtin_va_arg or something
> 100% equivalent to it. If you have a compiler than chooses to be

Wait, this is not a library busyness, this is something which happens
_inside_ a function. Passing variadic arguments between internal functions
can also be left to the compiler and this should not have to be reflected
in the library's public headers.

I will when I can make an extra round of reading the musl source to
try to see the implications - for the moment the point is evading me.

> Again, this has nothing to do with ABI between external variadic
> functions. It's a matter of the inability to express to the compiler
> what you want it to do (get the next variadic argument) in plain C.

It is of course only the compiler who knows how to do va_arg, but its
internal knowledge can be expressed *either* by __builtin_va_arg *or*
by an include file defining va_arg in a compiler-specific fashion
*or even* by implementing va_arg as a function like tcc does.

I do not argue that not-builtins are "good" but frankly I do not see
how these non-builtins-ways to express the compiler specific details
are contradicting standards or compatibility.

The difference from a builtin is that a definition in a header is "less
opaque", but the preprocessor macros are a natural abstraction layer in
C and are not supposed to be looked behind.

Rune

Re: variadic args (was: [musl] compiling musl on x86_64 linux with pcc)

[Rich Felker]

On Wed, Aug 27, 2014 at 10:52:54AM +0200, u-igbb@aetey.se wrote:
> On Wed, Aug 27, 2014 at 03:54:07AM -0400, Rich Felker wrote:
> > > I am possibly not realizing some crucial bit but I do not see why the
> > > implementation of stdarg.h must be the business of the library. If the
> > 
> > Because other headers need to know how to get va_list defined for use
> > in function prototypes, and they can't include stdarg.h because it
> > would expose more than they want. glibc handles this by making use of
> 
> I guess this is the matter which everything revolves around.
> What is it that stdarg.h would expose too much?

va_arg, va_start, va_end, va_copy.

> > a special inclusion protocol between the glibc headers and the gcc
> > headers: it defines a special macro before including stdarg.h telling
> > gcc that it just wants the definition of va_list and nothing else.
> 
> Wonder whether it would be reasonable to say that musl expects from
> every compiler to be used with the library either
> a stdarg.h with following properties: ...
>  <which ones?>
> otherwise expecting the compiler to implement the following builtins:
>  <the current contents of musl's stdarg.h>

This would require changes from the compiler anyway (and then the
compiler would be using musl-header-internal interfaces that we do not
define as stable and do not intend for outside use), or require
encoding assumptions about specific compilers' headers in several
places in musl's headers. It would also

> I do not see how this would hurt in any way.
> 
> By the way, this - in any case - *is* an interface against the compiler.
> 
> As a matter of fact, the bultins naming differs between compilers.

Could you provide an example where the naming differs? I have not seen
it. (It should be a working C compiler that could otherwise reasonably
be used with musl, not something like MSVC, or a 16-bit x86 compiler,
or anything else that's definitely not usable. :)

> So the current musl approach is to just boldly assume a *certain*
> relatively arbitrary interface, not "the only possible" nor "the only
> reasonable one".

The same is necessary for things like INF/NAN, complex.h I, etc.;
there is no standard way to express them in C, so we choose the
universally-agreed-upon way to do it.

> > > A compiler used *together* with the library may lack the builtins but it
> > > still can be fully compatible even if in some situations less efficient.
> > 
> > No, it can't. There is simply no way to express, in C, the concept of
> > "get the next variadic argument" without __builtin_va_arg or something
> > 100% equivalent to it. If you have a compiler than chooses to be
> 
> Wait, this is not a library busyness, this is something which happens
> _inside_ a function. Passing variadic arguments between internal functions
> can also be left to the compiler and this should not have to be reflected
> in the library's public headers.

It does matter to the library because the types in the headers have to
match. va_list is not used only in stdarg.h.

> > Again, this has nothing to do with ABI between external variadic
> > functions. It's a matter of the inability to express to the compiler
> > what you want it to do (get the next variadic argument) in plain C.
> 
> It is of course only the compiler who knows how to do va_arg, but its
> internal knowledge can be expressed *either* by __builtin_va_arg *or*
> by an include file defining va_arg in a compiler-specific fashion
> *or even* by implementing va_arg as a function like tcc does.

Setting up your compiler to target musl already requires some specific
tuning like -Wl,-dynamic-linker,... if you're using dynamic linking
(GCC requires more tweaking to remove some glibc assumptions); ideally
such tweaking is added in the compiler's profile for musl, or in
something like spec files or whatever. There's no reason you can't use
'-D__builtin_va_XXX(x)=...' in such a context, unless the compiler
reserves this exact same name but uses it for something completely
different that's impossible to override, and in that case the same
argument applies to lots of musl-internal stuff beginning with __ in
the public headers. This is because __builtin_va_list, etc. are in the
reserved namespace.

> I do not argue that not-builtins are "good" but frankly I do not see
> how these non-builtins-ways to express the compiler specific details
> are contradicting standards or compatibility.

There is simply no way a decent compiler could use the non-builtin
definitions. As I've stated before, they preclude optimization, and
since the way they are expressed overlaps with highly invalid pointer
arithmetic for which a good compiler would want to produce warnings,
they also preclude warning about buggy code.

Note that the nonsense legacy definition would not even be in musl's
i386 bits/stdarg.h if I were writing it today. That header was written
something like a decade ago when i was using it with a much older
toolchain and system, and basically left and forgotten.

> The difference from a builtin is that a definition in a header is "less
> opaque", but the preprocessor macros are a natural abstraction layer in
> C and are not supposed to be looked behind.

I really don't want to argue about this anymore. It's not achieving
anything productive. Your proposed changes on musl's side would not be
likely to help anyone achieve anything they can't already achieve in
simpler ways (-D) and would uglify the headers and make new
cross-component (compiler and libc) dependencies between internal
headers inclusion-control conventions -- one of the original things
musl was intended to avoid.

Also, in the time you've spent arguing for supporting something that's
obviously broken (using illegal pointer arithmetic to represent
variadic args) you could probably have gotten a patch adding
__builtin_va_* into tcc, using predefined macros with the current bad
definitions if nothing else.

Rich

Re: variadic args (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

> I really don't want to argue about this anymore.

I respect this and will not bother you and the list on this topic.

Thanks for going out of your way to explain your reasons for the choices.

Unfortunately in some cases I have to work around the choices, for reasons
legitimate in my eyes. This does not imply that you (or me) are wrong,
but merely confirms that no approach is good for every situation.

> Also, in the time you've spent arguing for supporting something that's
> obviously broken (using illegal pointer arithmetic to represent
> variadic args) you could probably have gotten a patch adding
> __builtin_va_* into tcc, using predefined macros with the current bad
> definitions if nothing else.

Sorry, this is an argument which I can not accept (skipping the technical
statements which I do not agree with but we are not to talk about).

I am not in a position to fix every compiler I might need, unless I am
forced to.

I try otherwise to fix what I work with and what I possibly know better
than other parties, namely how to make the pieces (compilers, libraries
and applications) work _together_ in a most general fashion.

Musl happens to be an elegantly designed and efficiently usable libc,
very useful in such a context. That's why I care about it.
Thanks Rich!

Rune

Re: variadic args (was: [musl] compiling musl on x86_64 linux with pcc)

[Rich Felker]

On Thu, Aug 28, 2014 at 10:13:16AM +0200, u-igbb@aetey.se wrote:
> > Also, in the time you've spent arguing for supporting something that's
> > obviously broken (using illegal pointer arithmetic to represent
> > variadic args) you could probably have gotten a patch adding
> > __builtin_va_* into tcc, using predefined macros with the current bad
> > definitions if nothing else.
> 
> Sorry, this is an argument which I can not accept (skipping the technical
> statements which I do not agree with but we are not to talk about).
> 
> I am not in a position to fix every compiler I might need, unless I am
> forced to.

And likewise we're not in a position to support interacting with the
compiler-provided stdarg.h from every possible compiler.

Note that this is actually complex in practice (there are lots of
variants!) while the __builtin_va_* stuff is not complex in practice
(tcc is the only example anyone's given where the uniform
__builtin_va_* stuff doesn't work).

Rich

Re: variadic args (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

On Thu, Aug 28, 2014 at 10:40:32AM -0400, Rich Felker wrote:
> compiler-provided stdarg.h from every possible compiler.
> 
> Note that this is actually complex in practice (there are lots of
> variants!) while the __builtin_va_* stuff is not complex in practice
> (tcc is the only example anyone's given where the uniform
> __builtin_va_* stuff doesn't work).

Indeed, you actually convinced me that e.g. fulfilling the symbol
visibility constraints (to be fully standard compliant) is not easily
provided with a "simplistic" approach.

So, reasoning in the practical terms, the reliance on builtins is the
best fit for efficient and consistent musl development.

I am otherwise satisfied with workarounds (working and good enough
for my practical needs - the applications usually disregard a lot more
standards than my setup :) for the sometimes "weird" compilers I happen
to deal with, so this issue is in no way a showstopper here.

Thanks for your time in the discussion Rich.

Best regards,
Rune

compiling musl with pcc (i486-pc-linux-gnu) (was: [musl] compiling musl on x86_64 linux with pcc)

[u-igbb]

On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> I have pcc 1.1.0.DEVEL 20130227 with pcc-libs-20130721 and it seems to
> still work fine with current musl. Note that some fixes are needed for

I have got a musl instance compiled with pcc which seems to be usable.
A pcc compiled and linked against it reproduces itself and then
reproduces the same code for musl.

The catch:

Using --enable-optimize=no leads to a broken library (at least
concerning the variadic arguments).

> I just ran libc-test with libc.so built by pcc and confirmed that
> there are no non-math regressions versus gcc-built libc.so. (I did not
> check math in detail because there are lots of known math failures
> that are not serious.)

Now we know that this is possible with the current version as well,
but not without optimization :)

Regards,
Rune

Re: compiling musl with pcc (i486-pc-linux-gnu) (was: [musl] compiling musl on x86_64 linux with pcc)

[Rich Felker]

On Wed, Aug 27, 2014 at 04:38:58PM +0200, u-igbb@aetey.se wrote:
> On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> > I have pcc 1.1.0.DEVEL 20130227 with pcc-libs-20130721 and it seems to
> > still work fine with current musl. Note that some fixes are needed for
> 
> I have got a musl instance compiled with pcc which seems to be usable.
> A pcc compiled and linked against it reproduces itself and then
> reproduces the same code for musl.
> 
> The catch:
> 
> Using --enable-optimize=no leads to a broken library (at least
> concerning the variadic arguments).

When I get a chance I'll see if I can reproduce this with my pcc
toolchain or a new one. It won't necessarily be right away though. If
so, I can probably track down the cause pretty quickly. Let me know if
you find anything else about the cause.

> > I just ran libc-test with libc.so built by pcc and confirmed that
> > there are no non-math regressions versus gcc-built libc.so. (I did not
> > check math in detail because there are lots of known math failures
> > that are not serious.)
> 
> Now we know that this is possible with the current version as well,
> but not without optimization :)

Yes. Thanks for reporting!

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> Note that some fixes are needed for
> pcc-libs:
> 
> - You need to remove the broken csu files and replace them with
>   symlinks to the musl files.
> 
> - You need to add __attribute__((__visibility__("hidden"))) to all
>   external functions in libpcc.
> 
> - You need to add -fPIC to the CFLAGS for building libpcc or you get a
>   broken libpcc (containing textrels) that can't be used for building
>   libc.so and that's harmful for use with any other shared libs.

I followed these directions and musl with pcc play along very well for me
(on i386).

One strange thing I notice though, which is probably more pcc-related
than musl (even though this concerns a musl-based toolchain).

If I build a shared library with pcc without supplying -Wl,-Bsymbolic
(or -Wl,-Bsymbolic-functions as musl does) then a(ny) binary linked to that
library segfaults at start even if it does not actually need/use the
library. I noticed this with libpcap and wonder what is the reason. Adding
-Wl,-Bsymbolic makes everything work so the code is not exactly broken.

Wonder if you happen to guess a possible reason for such kind of trouble,
something that pcc gets wrong? This looks related to runtime linking
behaviour so possibly not fully offtopic here.

Rune

Re: compiling musl on x86_64 linux with pcc

[Rich Felker]

On Tue, Sep 23, 2014 at 09:05:25PM +0200, u-igbb@aetey.se wrote:
> On Mon, Aug 25, 2014 at 11:46:17AM -0400, Rich Felker wrote:
> > Note that some fixes are needed for
> > pcc-libs:
> > 
> > - You need to remove the broken csu files and replace them with
> >   symlinks to the musl files.
> > 
> > - You need to add __attribute__((__visibility__("hidden"))) to all
> >   external functions in libpcc.
> > 
> > - You need to add -fPIC to the CFLAGS for building libpcc or you get a
> >   broken libpcc (containing textrels) that can't be used for building
> >   libc.so and that's harmful for use with any other shared libs.
> 
> I followed these directions and musl with pcc play along very well for me
> (on i386).
> 
> One strange thing I notice though, which is probably more pcc-related
> than musl (even though this concerns a musl-based toolchain).
> 
> If I build a shared library with pcc without supplying -Wl,-Bsymbolic
> (or -Wl,-Bsymbolic-functions as musl does) then a(ny) binary linked to that
> library segfaults at start even if it does not actually need/use the
> library. I noticed this with libpcap and wonder what is the reason. Adding
> -Wl,-Bsymbolic makes everything work so the code is not exactly broken.
> 
> Wonder if you happen to guess a possible reason for such kind of trouble,
> something that pcc gets wrong? This looks related to runtime linking
> behaviour so possibly not fully offtopic here.

Have you checked that this happens with all libs, or did you only
observe it with libpcap? If you have a minimal example .so file that
causes the crash when loaded, can you post it somewhere I could
download a copy to analyze, and also post the commands you used to
produce it?

Rich

Re: compiling musl on x86_64 linux with pcc

[u-igbb]

[-- Attachment #1: Type: text/plain, Size: 784 bytes --]

On Tue, Sep 23, 2014 at 03:31:34PM -0400, Rich Felker wrote:
> > If I build a shared library with pcc without supplying -Wl,-Bsymbolic
> > (or -Wl,-Bsymbolic-functions as musl does) then a(ny) binary linked to that
> > library segfaults at start even if it does not actually need/use the
> > library. I noticed this with libpcap and wonder what is the reason. Adding
> > -Wl,-Bsymbolic makes everything work so the code is not exactly broken.

> Have you checked that this happens with all libs, or did you only
> observe it with libpcap? If you have a minimal example .so file that
> causes the crash when loaded, can you post it somewhere I could
> download a copy to analyze, and also post the commands you used to
> produce it?

Attaching the test case, it is small enough.

Rune

[-- Attachment #2: libfoo.tar.gz --]
[-- Type: application/x-tar-gz, Size: 2795 bytes --]

Re: compiling musl on x86_64 linux with ppc

[Isaac Dunham]

On Wed, Aug 13, 2014 at 02:56:08PM +0200, u-igbb@aetey.se wrote:
> My goal is a musl-based selfcontained pcc toolchain.
> I'd like to also have a tcc-based one but musl seems to be too hard
> a target for tcc yet :( sigh.

You will need to patch pcc to get the right ld.so; if I'm remembering
right, there's no reliable way to change it on the command line.
(Just a heads up just in case.)

HTH,
Isaac Dunham

Re: compiling musl on x86_64 linux with ppc

[Rich Felker]

On Wed, Aug 13, 2014 at 04:39:29PM -0700, Isaac Dunham wrote:
> On Wed, Aug 13, 2014 at 02:56:08PM +0200, u-igbb@aetey.se wrote:
> > My goal is a musl-based selfcontained pcc toolchain.
> > I'd like to also have a tcc-based one but musl seems to be too hard
> > a target for tcc yet :( sigh.
> 
> You will need to patch pcc to get the right ld.so; if I'm remembering
> right, there's no reliable way to change it on the command line.
> (Just a heads up just in case.)

-Wl,-dynamic-linker should work fine, but for changing the default you
probably need to patch (unless there's the equivalent of gcc's specs
file).

Rich