Re: SIGILL in setjmp() on ARM

["Felix Hädicke" <felixhaedicke@web.de>]

On 10/09/17 07:06, Rich Felker wrote:
> On Sun, Oct 08, 2017 at 08:57:36PM +0200, Felix Hädicke wrote:
>> Hello,
>>
>> For the XCSoar open source software (https://xcsoar.org), we have a
>> version for Kobo eReader devices. We are currently using glibc for this,
>> but are trying to switch to musl.
>>
>> However, for optimized builds, the program crashes with SIGILL in
>> setjmp(), which is called by the freetype library, which is used in our
>> program.
>>
>> The following CFLAGS were used:
>> -march=armv7-a -mcpu=cortex-a8 -mfpu=neon -mfloat-abi=hard -mthumb
>> -fvisibility=hidden -Os -g
>>
>> Everything (including musl and freetype) is linked statically.
>>
>> Debian's "arm-linux-gnueabihf" toolchain is used for compiling, which
>> was configured with
>> --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb
>>
>> The Kobo Mini eReader which I use for testing, has a Freescale i.MX 5
>> (Cortex-A8). But the crash is reproducible on i.MX 6 based Kobo devices,
>> and even on QEMU, as well.
>>
>> Disabling optimisation (flag "-O0" instead of "-Os"), or disabling Thumb
>> (flag "-marm") for musl solves the problem. However, doing this for
>> compiling the setjmp.s file only does not make a difference. And using
>> "-O1" instead of "-Os" does not help.
>>
>> Find some GDB crash dump analysis below.
>>
>> Regards,
>> Felix
>>
>>
>>
>> Core was generated by `/mnt/onboard/xcsoar'.
>> Program terminated with signal SIGILL, Illegal instruction.
>> #0  setjmp () at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/musl-1.1.16/src/setjmp/arm/setjmp.s:35
>> 35              stcl p1, cr10, [ip], #8
>> [Current thread is 1 (LWP 820)]
>>
>> (gdb) bt
>> #0  setjmp () at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/musl-1.1.16/src/setjmp/arm/setjmp.s:35
>> #1  0x00135c04 in tt_face_build_cmaps (face=face@entry=0x545880) at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/sfnt/ttcmap.c:3753
>> #2  0x00135f34 in sfnt_load_face (stream=<optimized out>, face=0x545880,
>> face_instance_index=<optimized out>, num_params=<optimized out>, params=0x0)
>>     at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/sfnt/sfobjs.c:1470
>> #3  0x0012e7cc in tt_face_init (stream=<optimized out>, ttface=0x545880,
>> face_index=0, num_params=0, params=0x0) at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/truetype/ttobjs.c:596
>> #4  0x0012983e in open_face (driver=driver@entry=0x5443b0,
>> astream=astream@entry=0x7efdaaec,
>> external_stream=external_stream@entry=0 '\000',
>> face_index=face_index@entry=0, num_params=0, params=0x0,
>>     aface=aface@entry=0x7efdaaf0) at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/base/ftobjs.c:1195
>> #5  0x0012a34a in ft_open_face_internal (library=0x5442e0,
>> args=args@entry=0x7efdac10, face_index=face_index@entry=0,
>> aface=aface@entry=0x7efdac3c, test_mac_fonts=test_mac_fonts@entry=1 '\001')
>>     at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/base/ftobjs.c:2267
>> #6  0x0012a686 in FT_New_Face (library=<optimized out>,
>> pathname=pathname@entry=0x544710 "/opt/xcsoar/share/fonts/Vera.ttf",
>> face_index=face_index@entry=0, aface=aface@entry=0x7efdac3c)
>>     at
>> /home/felix/.tmp/xcsoar-merge-musl/output/src/freetype-2.8/src/base/ftobjs.c:1258
>> #7  0x000c7a52 in FreeType::Load (path=path@entry=0x544710
>> "/opt/xcsoar/share/fonts/Vera.ttf") at src/Screen/FreeType/Init.cpp:63
>> #8  0x000c757a in Font::LoadFile (this=0x52a27c <Fonts::map>,
>> file=0x544710 "/opt/xcsoar/share/fonts/Vera.ttf", ptsize=18,
>> bold=<optimized out>, italic=false) at src/Screen/FreeType/Font.cpp:157
>> #9  0x000c765a in Font::Load (this=this@entry=0x52a27c <Fonts::map>,
>> d=...) at src/Screen/FreeType/Font.cpp:213
>> #10 0x00068eb8 in Fonts::Load (settings=...) at src/Look/GlobalFonts.cpp:36
>> #11 0x00068fe0 in Fonts::Initialize () at src/Look/DefaultFonts.cpp:52
>> #12 0x0006ae1c in MainWindow::Initialise (this=this@entry=0x5451e0) at
>> src/MainWindow.cpp:196
>> #13 0x0006b8e4 in Startup () at src/Startup.cpp:231
>> #14 0x0001a800 in Main () at src/XCSoar.cpp:121
>> #15 main (argc=<optimized out>, argv=<optimized out>) at src/XCSoar.cpp:170
>>
>> (gdb) disassemble
>> Dump of assembler code for function setjmp:
>>    0x001968ce <+0>:     mov     r12, r0
>>    0x001968d0 <+2>:     stmia.w r12!, {r4, r5, r6, r7, r8, r9, r10, r11}
>>    0x001968d4 <+6>:     mov     r2, sp
>>    0x001968d6 <+8>:     stmia.w r12!, {r2, lr}
>>    0x001968da <+12>:    mov.w   r0, #0
>>    0x001968de <+16>:    add     r1, pc, #60     ; (adr r1, 0x19691c
>> <setjmp+78>)
>>    0x001968e0 <+18>:    ldr     r2, [pc, #60]   ; (0x196920 <setjmp+82>)
>>    0x001968e2 <+20>:    ldr     r1, [r1, r2]
>>    0x001968e4 <+22>:    tst.w   r1, #608        ; 0x260
>>    0x001968e8 <+26>:    beq.n   0x19691c <setjmp+78>
>>    0x001968ea <+28>:    tst.w   r1, #32
>>    0x001968ee <+32>:    beq.n   0x1968f4 <setjmp+38>
>>    0x001968f0 <+34>:    sfm     f4, 4, [r12], #48       ; 0x30
>>    0x001968f4 <+38>:    tst.w   r1, #64 ; 0x40
>>    0x001968f8 <+42>:    beq.n   0x1968fe <setjmp+48>
>>    0x001968fa <+44>:    vstmia  r12!, {d8-d15}
>>    0x001968fe <+48>:    tst.w   r1, #512        ; 0x200
>>    0x00196902 <+52>:    beq.n   0x19691c <setjmp+78>
>> => 0x00196904 <+54>:    stfp    f2, [r12], #8
>>    0x00196908 <+58>:    stfp    f3, [r12], #8
>>    0x0019690c <+62>:    stfp    f4, [r12], #8
>>    0x00196910 <+66>:    stfp    f5, [r12], #8
>>    0x00196914 <+70>:    stfp    f6, [r12], #8
>>    0x00196918 <+74>:    stfp    f7, [r12], #8
>>    0x0019691c <+78>:    bx      lr
>>    0x0019691e <+80>:    eorseq  r7, r10, r2, ror #20
>>    0x00196922 <+84>:    stmdbmi r4, {}  ; <UNPREDICTABLE>
>> End of assembler dump.
>>
>> (gdb) info registers
>> r0             0x0      0
>> r1             0x27af4601       665798145
>> r2             0x3a     58
>> r3             0x10c    268
>> r4             0x7efda7c0       2130552768
>> r5             0xa59    2649
>> r6             0x1      1
>> r7             0x0      0
>> r8             0x0      0
>> r9             0x0      0
>> r10            0x0      0
>> r11            0x219cf4 2202868
>> r12            0x7efda7e8       2130552808
>> sp             0x7efda778       0x7efda778
>> lr             0x135c05 1268741
>> pc             0x196904 0x196904 <setjmp+54>
>> cpsr           0x30     48
> OK, this is a very subtle issue with the assembler and/or linker. For
> the adr pseudo-instruction above (setjmp+16), the assembler generates
> a 16-bit thumb add instruction which can only represent word-aligned
> addresses, despite not knowing the alignment of the label. When the
> setjmp function is assigned a non-multiple-of-4 address at link time
> (which is perturbed by your -O level, etc. of *other* object files,
> not setjmp itself) the load then loads from the wrong address
> (setjmp+78 rather than setjmp+80) and ends up reading nonsense instead
> of the value of __hwcap, thereby thinking you have IWMMX extensions
> and need to save/restore those registers.
>
> Try the attached patch and let me know if it fixes it for you.
>
> Rich

Your patch solves the problem! Tried with GCC5 and GCC7, with -fno-PIE
and with -fPIE. Everything compiled with -Os and -mthumb.

Regards,
Felix