mips port working! & remaining issues

mips port working! & remaining issues

[Rich Felker]

Hi all,

According to libc-testsuite, the new mips port of musl is working. If
remaining issues exist, I suspect they're in the bits/*.h files'
definitions of kernel constants and structures, which I could use some
help reviewing. Threads, including cancellation, are known to work,
and mipsel (little endian) mode presumably works but is untested.

One major omission that remains is the dynamic linker. Due to mips
o32's pathologically bad PIC ABI which has no PC-relative addressing,
the current dynamic linker cannot work as-is. Rather than replacing it
with an ugly mess of fragile early-stage code that depends on
non-optimization by the compiler (the way traditional dynamic linkers
work), I'd rather simply require the dynamic linker and libc to be
loaded at a fixed address chosen at build time -- this is basically
what prelink does. We may be able to use the existing prelink tools,
or just write a small special-purpose tool to prepare libc.so for
loading at a fixed address.

Another major omission shared with arm is that setjmp/longjmp cannot
preserve floating point registers. This is because I don't yet know
how to determine at runtime whether hard-float is available, and the
registers can't be saved unless they can be determined to exist. Help
solving this problem would also be appreciated.

Rich

Re: mips port working! & remaining issues

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-07-13 01:23:13 -0400]:
> Another major omission shared with arm is that setjmp/longjmp cannot
> preserve floating point registers. This is because I don't yet know
> how to determine at runtime whether hard-float is available, and the
> registers can't be saved unless they can be determined to exist. Help
> solving this problem would also be appreciated.
> 

then use compile time check:
if libc is compiled with -msoft-float then dont save the registers
otherwise save them

i guess gcc has some predefined macro so you can check soft float
at compile time

when gcc compiles with hard float for mips there will be a
a .gnu.attributes section in the object file which is dumped
by readelf as
  Tag_GNU_MIPS_ABI_FP: Hard float (-mdouble-float)
and binutils ld warns when such .o is linked with a soft-float .o

so i think we shouldn't worry about applications using hard float
while the libc is soft float

as for runtime check
on mips you may grep for '^  FPU' in /proc/cpuinfo
on arm '^Features' tells the fpu variant
but i guess these are not very robust
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=arch/mips/kernel/proc.c
http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=blob;f=arch/arm/kernel/setup.c

Re: mips port working! & remaining issues

[Justin Cormack]

On Fri, Jul 13, 2012 at 9:15 AM, Szabolcs Nagy <nsz@port70.net> wrote:

> so i think we shouldn't worry about applications using hard float
> while the libc is soft float

Agreed. For ARM the assumption is you don't use fp at all in
softfloat, use the armhf ABI if you wan't real floats and have a more
recent CPU. Most of the Linux distros are now splitting into two for
ARM, hardfloat and softfloat, with the different ABI.

Justin

Re: mips port working! & remaining issues

[Rich Felker]

On Fri, Jul 13, 2012 at 09:18:01AM +0100, Justin Cormack wrote:
> On Fri, Jul 13, 2012 at 9:15 AM, Szabolcs Nagy <nsz@port70.net> wrote:
> 
> > so i think we shouldn't worry about applications using hard float
> > while the libc is soft float
> 
> Agreed. For ARM the assumption is you don't use fp at all in
> softfloat, use the armhf ABI if you wan't real floats and have a more
> recent CPU. Most of the Linux distros are now splitting into two for
> ARM, hardfloat and softfloat, with the different ABI.

My understanding is that the EABI provides conventions that ensure
that code can be linked together whether it's written to use hard or
soft float. But unless all floating point registers are temp (not
callee-saved), I can't find a way to make that work with setjmp...

Rich

Re: mips port working! & remaining issues

[Luca Barbato]

On 07/13/2012 03:08 PM, Rich Felker wrote:

> My understanding is that the EABI provides conventions that ensure
> that code can be linked together whether it's written to use hard or
> soft float. But unless all floating point registers are temp (not
> callee-saved), I can't find a way to make that work with setjmp...

Currently hardfloat just pass the registers instead of doing some copy
over in a way or another and it is what people will use.

lu

-- 

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero

Re: mips port working! & remaining issues

[Rich Felker]

On Fri, Jul 13, 2012 at 03:36:03PM +0200, Luca Barbato wrote:
> On 07/13/2012 03:08 PM, Rich Felker wrote:
> 
> > My understanding is that the EABI provides conventions that ensure
> > that code can be linked together whether it's written to use hard or
> > soft float. But unless all floating point registers are temp (not
> > callee-saved), I can't find a way to make that work with setjmp...
> 
> Currently hardfloat just pass the registers instead of doing some copy
> over in a way or another and it is what people will use.

That's the hardfloat ABI variant of EABI, but there's also base EABI
that can use hard float behind the scenes (in the soft float
functions) just by calling the __aeabi functions and having them
implemented with hard-float. Although I suppose this usage does not
require the registers to be preserved.

Now I just need to work out a nice way to conditionally compile
different ASM for each variant. Or I could have setjmp and longjmp
just read a global var with the hardfloat flag in it, and jump over
the float register code if it's false. Opinions on what's best?

Rich

Re: mips port working! & remaining issues

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-07-13 10:25:21 -0400]:
> On Fri, Jul 13, 2012 at 03:36:03PM +0200, Luca Barbato wrote:
> > On 07/13/2012 03:08 PM, Rich Felker wrote:
> > 
> > > My understanding is that the EABI provides conventions that ensure
> > > that code can be linked together whether it's written to use hard or
> > > soft float. But unless all floating point registers are temp (not
> > > callee-saved), I can't find a way to make that work with setjmp...
> > 
> > Currently hardfloat just pass the registers instead of doing some copy
> > over in a way or another and it is what people will use.
> 
> That's the hardfloat ABI variant of EABI, but there's also base EABI
> that can use hard float behind the scenes (in the soft float
> functions) just by calling the __aeabi functions and having them
> implemented with hard-float. Although I suppose this usage does not
> require the registers to be preserved.
> 
> Now I just need to work out a nice way to conditionally compile
> different ASM for each variant. Or I could have setjmp and longjmp
> just read a global var with the hardfloat flag in it, and jump over
> the float register code if it's false. Opinions on what's best?
> 
> Rich

glibc seems to do runtime check for vfp and iwmmxt fpu on arm

http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/arm/setjmp.S

and do compile time check on mips

http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/mips/setjmp_aux.c

Re: mips port working! & remaining issues

[Rich Felker]

On Fri, Jul 13, 2012 at 06:10:12PM +0200, Szabolcs Nagy wrote:
> glibc seems to do runtime check for vfp and iwmmxt fpu on arm
> 
> http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/arm/setjmp.S
> 
> and do compile time check on mips
> 
> http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/mips/setjmp_aux.c

Thanks for digging up these for reference. I wonder if the auxv
contains hw caps on mips too...?

On a side note, it's really unbelievable that mips setjmp is written
in C on glibc and uclibc (which just copied from glibc). The compiler
is completely free to clobber any of the registers that code is
attempting to save (as long as the original value gets restored before
the return statement) and as such their implementation is completely
invalid unless you assume the compiler behaves in a particular way.

Rich

Re: mips port working! & remaining issues

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-07-13 13:34:11 -0400]:
> On Fri, Jul 13, 2012 at 06:10:12PM +0200, Szabolcs Nagy wrote:
> > glibc seems to do runtime check for vfp and iwmmxt fpu on arm
> > 
> > http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/arm/setjmp.S
> > 
> > and do compile time check on mips
> > 
> > http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/mips/setjmp_aux.c
> 
> Thanks for digging up these for reference. I wonder if the auxv
> contains hw caps on mips too...?
> 

i don't know, but i guess then mips code would use it as well

> On a side note, it's really unbelievable that mips setjmp is written
> in C on glibc and uclibc (which just copied from glibc). The compiler
> is completely free to clobber any of the registers that code is
> attempting to save (as long as the original value gets restored before
> the return statement) and as such their implementation is completely
> invalid unless you assume the compiler behaves in a particular way.

there is another place where register saving is implemented (getcontext)
and that is asm for mips as well

http://sourceware.org/git/?p=glibc.git;a=blob;f=ports/sysdeps/unix/sysv/linux/mips/getcontext.S

Re: mips port working! & remaining issues

[idunham]

> On Fri, Jul 13, 2012 at 03:36:03PM +0200, Luca Barbato wrote:
>> Currently hardfloat just pass the registers instead of doing some copy
>> over in a way or another and it is what people will use.
>
> That's the hardfloat ABI variant of EABI, but there's also base EABI
> that can use hard float behind the scenes (in the soft float
> functions) just by calling the __aeabi functions and having them
> implemented with hard-float. Although I suppose this usage does not
> require the registers to be preserved.
>
> Now I just need to work out a nice way to conditionally compile
> different ASM for each variant. Or I could have setjmp and longjmp
> just read a global var with the hardfloat flag in it, and jump over
> the float register code if it's false. Opinions on what's best?
>
Seems to me that the hardfloat flag (while slightly more bloated,
because you have code for both options) has room for adding runtime
checks: eventually, it might be possible to have that as a static
variable (I'm thinking char or something; at least 2 bits would be
needed for this method...), and if it is uninitialized (0x00?), try
saving the registers or something that will raise a SIGILL on pure
softfloat systems; the handler would set it to indicate softfloat
(0xFF?), while otherwise it sets it to indicate hardfloat (any
intermediate value?).
That particular approach might not work (can you install handlers in
*jmp safely?), but a runtime check would require using the hardfloat
flag.
In fact, you could use the flag to specify which FPU is present,
which is potentially useful for some other purposes down the road.
It would still be possible to set the flag at compile time.

Isaac Dunham

Re: mips port working! & remaining issues

[Rich Felker]

On Fri, Jul 13, 2012 at 04:58:39PM -0400, idunham@lavabit.com wrote:
> Seems to me that the hardfloat flag (while slightly more bloated,
> because you have code for both options) has room for adding runtime
> checks: eventually, it might be possible to have that as a static
> variable (I'm thinking char or something; at least 2 bits would be
> needed for this method...), and if it is uninitialized (0x00?), try
> saving the registers or something that will raise a SIGILL on pure
> softfloat systems; the handler would set it to indicate softfloat
> (0xFF?), while otherwise it sets it to indicate hardfloat (any
> intermediate value?).
> That particular approach might not work (can you install handlers in
> *jmp safely?), but a runtime check would require using the hardfloat
> flag.

Test-and-trap is not a valid approach for runtime detection in
userspace, at least not at the library level. In principle it could be
if the libc did heavy control of signal handling, installed its own
SIGILL handler, and faked sigaction to redirect the application's
signal handler (if any) through the libc one rather than setting it
directly in kernelspace. But this is a lot more invasive than most
people would like, I think.

Proper runtime detection is performed via cpu self-identification
(e.g. the cpuid instruction on x86) or via the AT_HWCAP auxv entry
passed to the program by the kernel at startup.

Rich

Re: mips port working! & remaining issues

[Gregor Richards]

New cross compiler tarballs, particularly for those wishing for soft 
float, since that's not workable with the normal tarball:

Hard float: 
https://bitbucket.org/GregorR/musl-cross/downloads/crossx86-mips-linux-musl-2012-07-13-649cec5f.tar.xz

Soft float: 
https://bitbucket.org/GregorR/musl-cross/downloads/crossx86-mips-sf-linux-musl-2012-07-13-649cec5f.tar.xz

With valediction,
  - Gregor Richards

On 07/13/12 01:23, Rich Felker wrote:
> Hi all,
>
> According to libc-testsuite, the new mips port of musl is working. If
> remaining issues exist, I suspect they're in the bits/*.h files'
> definitions of kernel constants and structures, which I could use some
> help reviewing. Threads, including cancellation, are known to work,
> and mipsel (little endian) mode presumably works but is untested.
>
> One major omission that remains is the dynamic linker. Due to mips
> o32's pathologically bad PIC ABI which has no PC-relative addressing,
> the current dynamic linker cannot work as-is. Rather than replacing it
> with an ugly mess of fragile early-stage code that depends on
> non-optimization by the compiler (the way traditional dynamic linkers
> work), I'd rather simply require the dynamic linker and libc to be
> loaded at a fixed address chosen at build time -- this is basically
> what prelink does. We may be able to use the existing prelink tools,
> or just write a small special-purpose tool to prepare libc.so for
> loading at a fixed address.
>
> Another major omission shared with arm is that setjmp/longjmp cannot
> preserve floating point registers. This is because I don't yet know
> how to determine at runtime whether hard-float is available, and the
> registers can't be saved unless they can be determined to exist. Help
> solving this problem would also be appreciated.
>
> Rich