Re: MUSCL + MIPS + threads + debugging/stack traces

[Rich Felker <dalias@libc.org>]

On Mon, Jan 18, 2016 at 05:34:27PM -0800, Ward Willats wrote:
> Hello.
> 
> We're working on a consumer product using an OpenWRT-based MIPS
> Linux platform and linking our C++11 std::thread (really pthread)
> app to MUSL.
> 
> When we try to get a backtrace with the toolchain gdb/gdbserver (or
> the latest one we cross compiled ourselves from source) we get these
> kind of pathetic, truncated stack traces.
> 
> Here, for example, is a sample:
> 
> (gdb) thread apply all backtrace
> 
> Thread 4 (Thread 8009.9567):
> #0  0x77fcb904 in __cp_end () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> #1  0x77fc6f38 in __syscall_cp_c () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> #2  0x00000000 in ?? ()
> Backtrace stopped: frame did not save the PC
> 
> Thread 3 (Thread 8009.9566):
> #0  0x77fcb904 in __cp_end () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> #1  0x77fc6f38 in __syscall_cp_c () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> #2  0x00000000 in ?? ()
> Backtrace stopped: frame did not save the PC
> 
> Thread 2 (Thread 8009.9565):
> #0  0x77fcb904 in __cp_end () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> #1  0x77fc6f38 in __syscall_cp_c () from /Volumes/OpenWrt/fsp-openwrt/staging_dir/target-mipsel_24kec+dsp_musl-1.1.11/root-UberFSP/lib/ld-musl-mipsel-sf.so.1
> warning: GDB can't find the start of the function at 0x77b8da46.
> 
>     GDB is unable to find the start of the function at 0x77b8da46
> and thus can't determine the size of that function's stack frame.
> This means that GDB may be unable to access that stack frame, or
> the frames below it.
>     This problem is most likely caused by an invalid program counter or
> stack pointer.
>     However, if you think GDB should simply search farther back
> from 0x77b8da46 for code which looks like the beginning of a
> function, you can increase the range of the search using the `set
> heuristic-fence-post' command.
> #2  0x77b8da48 in ?? ()

It looks like your libc build lacks debug info and thus gdb is
attempting to use heuristics to make sense of the call stack. Do you
know if this is the case, and if so, whether there's a way to get
OpenWRT to build libc with -g? It's also possible that missing CFI in
our asm files it the cause. I'll ask and see if anyone else familiar
with MIPS has run into this and has ideas.

Rich