Memory management problem?

Memory management problem?

[Tobias Koch]

Hi,

I have (cross) compiled a chroot out of Debian testing sources but with musl and busybox at the core. The target is x86_64-cross-linux-musl. Now that I got to the point where I can actually run some builds inside the chroot itself, I noticed that flex segfaults on startup. In the flex sources this snippet is executed:

    num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */
    (yy_buffer_stack) = (struct yy_buffer_state**) yyalloc(num_to_alloc * sizeof(struct yy_buffer_state*));

    if ( ! (yy_buffer_stack) )
        YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" );

    memset((*yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*));

The memset is optimized away, because GCC understands that the pointer target is just one native word and does a

    => 0x0000000000401c7e <+64>:64movq   $0x0,(%rax)

instead. Strace shows the following output:

    execve("/tools/bin/flex", ["flex"], [/* 18 vars */]) = 0
    arch_prctl(ARCH_SET_FS, 0x7f87ee4d5b28) = 0
    set_tid_address(0x7f87ee4d5b60)         = 18855
    mprotect(0x7f87ee4d2000, 4096, PROT_READ) = 0
    mprotect(0x63f000, 4096, PROT_READ)     = 0
    brk(NULL)                               = 0x1cd4000
    brk(0x1cd6000)                          = 0x1cd6000
    brk(0x1cd7000)                          = 0x1cd7000
    brk(0x1cd8000)                          = 0x1cd8000
    brk(0x1cda000)                          = 0x1cda000
    brk(0x1cdc000)                          = 0x1cdc000
    brk(0x1cde000)                          = 0x1cde000
    brk(0x1ce0000)                          = 0x1ce0000
    brk(0x1ce2000)                          = 0x1ce2000
    brk(0x1ce4000)                          = 0x1ce4000
    brk(0x1ce6000)                          = 0x1ce6000
    brk(0x1ce8000)                          = 0x1ce8000
    brk(0x1cea000)                          = 0x1cea000
    brk(0x1cec000)                          = 0x1cec000
    brk(0x1cef000)                          = 0x1cef000
    brk(0x1cf0000)                          = 0x1cf0000
    brk(0x1cf1000)                          = 0x1cf1000
    brk(0x1cf2000)                          = 0x1cf2000
    brk(0x1cf3000)                          = 0x1cf3000
    brk(0x1cf4000)                          = 0x1cf4000
    brk(0x1cf6000)                          = 0x1cf6000
    brk(0x1cf8000)                          = 0x1cf8000
    --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xffffffffee4d6d60} ---
    +++ killed by SIGSEGV +++
    Segmentation fault

I'm not sure, if there is a problem with musl or some sort of optimization problem with GCC. If I compile either musl or flex without optimizations, the problem goes away. I have tried version 1.1.16 and git master.

Any hints on how I could get to the bottom of this, would be greatly appreciated.

Tobias

Re: Memory management problem?

[Rich Felker]

On Fri, Feb 17, 2017 at 05:48:08PM +0000, Tobias Koch wrote:
> Hi,
> 
> I have (cross) compiled a chroot out of Debian testing sources but
> with musl and busybox at the core. The target is
> x86_64-cross-linux-musl. Now that I got to the point where I can
> actually run some builds inside the chroot itself, I noticed that
> flex segfaults on startup. In the flex sources this snippet is
> executed:
> 
>     num_to_alloc = 1; /* After all that talk, this was set to 1 anyways.... */
>     (yy_buffer_stack) = (struct yy_buffer_state**) yyalloc(num_to_alloc * sizeof(struct yy_buffer_state*));
> 
>     if ( ! (yy_buffer_stack) )
>         YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" );
> 
>     memset((*yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*));
> 
> The memset is optimized away, because GCC understands that the pointer target is just one native word and does a
> 
>     => 0x0000000000401c7e <+64>:64movq   $0x0,(%rax)
> 
> instead. Strace shows the following output:
> 
>     execve("/tools/bin/flex", ["flex"], [/* 18 vars */]) = 0
>     arch_prctl(ARCH_SET_FS, 0x7f87ee4d5b28) = 0
>     set_tid_address(0x7f87ee4d5b60)         = 18855
>     mprotect(0x7f87ee4d2000, 4096, PROT_READ) = 0
>     mprotect(0x63f000, 4096, PROT_READ)     = 0
>     brk(NULL)                               = 0x1cd4000
>     brk(0x1cd6000)                          = 0x1cd6000
>     brk(0x1cd7000)                          = 0x1cd7000
>     brk(0x1cd8000)                          = 0x1cd8000
>     brk(0x1cda000)                          = 0x1cda000
>     brk(0x1cdc000)                          = 0x1cdc000
>     brk(0x1cde000)                          = 0x1cde000
>     brk(0x1ce0000)                          = 0x1ce0000
>     brk(0x1ce2000)                          = 0x1ce2000
>     brk(0x1ce4000)                          = 0x1ce4000
>     brk(0x1ce6000)                          = 0x1ce6000
>     brk(0x1ce8000)                          = 0x1ce8000
>     brk(0x1cea000)                          = 0x1cea000
>     brk(0x1cec000)                          = 0x1cec000
>     brk(0x1cef000)                          = 0x1cef000
>     brk(0x1cf0000)                          = 0x1cf0000
>     brk(0x1cf1000)                          = 0x1cf1000
>     brk(0x1cf2000)                          = 0x1cf2000
>     brk(0x1cf3000)                          = 0x1cf3000
>     brk(0x1cf4000)                          = 0x1cf4000
>     brk(0x1cf6000)                          = 0x1cf6000
>     brk(0x1cf8000)                          = 0x1cf8000
>     --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xffffffffee4d6d60} ---
>     +++ killed by SIGSEGV +++
>     Segmentation fault
> 
> I'm not sure, if there is a problem with musl or some sort of
> optimization problem with GCC. If I compile either musl or flex
> without optimizations, the problem goes away. I have tried version
> 1.1.16 and git master.
> 
> Any hints on how I could get to the bottom of this, would be greatly
> appreciated.

Judging from the address 0xffffffffee4d6d60, which is in kernel
address range, it looks like a pointer was truncated to 32 bits then
"sign-extended" back to 64. I suspect you have a missing declaration
(possibly due to missing include file) for some function that returns
a pointer and gcc is idiotically assuming it returns int and still
compiling rather than producing an error. Fix this by adding
-Werror=implicit-function-declaration to your CFLAGS; it should
pinpoint the location of the error.

Rich

Re: Memory management problem?

[Tobias Koch]

[-- Attachment #1: Type: text/plain, Size: 30 bytes --]

Spot on. Thanks a lot, Rich.

[-- Attachment #2: Type: text/html, Size: 447 bytes --]