Re: recvmsg/sendmsg broken on mips64

[Rich Felker <dalias@libc.org>]

On Mon, Apr 11, 2016 at 12:24:49AM +0200, Sebastian Gottschall wrote:
> >I think what nsz was asking for, and what I'd like to see, is a way to
> >reproduce the bug. I'm going to try building iproute2 for mips64 and
> >running it on a prebuilt kernel from Aboriginal Linux under
> >qemu-system-mips64, but I don't know what specific commands are needed
> >to hit the affected code path.
> any command since all is netlink based
> ip add add 192.168.1.1/24  dev eth0
> 
> yo will see that nothing will happen. ip will just return a error
> message (i wrote this message already in the first entry on this
> mailinglist)
> "EOF on netlink" is the error which is shown

OK, I'll try this.

> >>its all resulting in the same failing recvmsg / sendmsg call.. so
> >>yes libnetlink.c does not work with musl on mips64 (it does work on
> >>x64 and everything else, just not mips64) unless the hack i offered
> >>was applied which again fixed all.
> >>before you ask again for a problem description, just read again. it
> >>wont change the description if you ask again and just makes people
> >>tired on this list.
> >Both versions of the struct (musl's and your modified one that matches
> >the kernel) have the exact same layout, but due to having a member
> >with 64-bit type, yours has 8-byte alignment and musl's only has
> >4-byte alignment. This means, at least:
> >
> >1. When musl's sendmsg.c makes its copy to zero out the padding, the
> >    copy may not be correctly aligned for 64-bit writes, and the kernel
> >    faults or manually produces an error for this case, causing the
> >    whole operation to fail. However, I don't see where iproute2 is
> >    actually passing control messages to sendmsg, so while this is a
> >    problem, I don't think it's the cause. Maybe I'm missing the
> >    affected call point; this is why I'd like steps to reproduce the
> >    issue so I can see it.
> >
> >2. iproute2's libnetlink.c's rtnl_listen function does not properly
> >    declare its cmsgbuf with the alignment of cmsghdr; it has type
> >    char[] so the compiler is free not to align it at all. This is
> >    presumably a bug in iproute2, but I can't find any good
> >    documentation (in the standards or Linux-specific) for how you're
> >    supposed to allocate this space, so maybe the kernel is able to
> >    handle aligning the buffer itself. I don't see any way the
> >    alignment of musl's cmsghdr type affects recvmsg though.
> >
> >Maybe there are other effects I'm missing? I'll follow up again once I
> >get a test build/run of iproute2 and let you know whether I can see
> >the problem.
> okay. if you need a remote access to a octeon system using musl (my
> fixed variant), just tell me.

That would be really helpful. Something's wrong with the userspace for
the Aboriginal mips64 binaries (SIGBUS in init) and debugging that
would be a big distraction.

BTW do you have gdb and strace available?

Rich