Re: recvmsg/sendmsg broken on mips64

[Sebastian Gottschall <s.gottschall@dd-wrt.com>]

Am 11.04.2016 um 00:29 schrieb Rich Felker:
> On Mon, Apr 11, 2016 at 12:24:49AM +0200, Sebastian Gottschall wrote:
>>> I think what nsz was asking for, and what I'd like to see, is a way to
>>> reproduce the bug. I'm going to try building iproute2 for mips64 and
>>> running it on a prebuilt kernel from Aboriginal Linux under
>>> qemu-system-mips64, but I don't know what specific commands are needed
>>> to hit the affected code path.
>> any command since all is netlink based
>> ip add add 192.168.1.1/24  dev eth0
>>
>> yo will see that nothing will happen. ip will just return a error
>> message (i wrote this message already in the first entry on this
>> mailinglist)
>> "EOF on netlink" is the error which is shown
> OK, I'll try this.
>
>>>> its all resulting in the same failing recvmsg / sendmsg call.. so
>>>> yes libnetlink.c does not work with musl on mips64 (it does work on
>>>> x64 and everything else, just not mips64) unless the hack i offered
>>>> was applied which again fixed all.
>>>> before you ask again for a problem description, just read again. it
>>>> wont change the description if you ask again and just makes people
>>>> tired on this list.
>>> Both versions of the struct (musl's and your modified one that matches
>>> the kernel) have the exact same layout, but due to having a member
>>> with 64-bit type, yours has 8-byte alignment and musl's only has
>>> 4-byte alignment. This means, at least:
>>>
>>> 1. When musl's sendmsg.c makes its copy to zero out the padding, the
>>>     copy may not be correctly aligned for 64-bit writes, and the kernel
>>>     faults or manually produces an error for this case, causing the
>>>     whole operation to fail. However, I don't see where iproute2 is
>>>     actually passing control messages to sendmsg, so while this is a
>>>     problem, I don't think it's the cause. Maybe I'm missing the
>>>     affected call point; this is why I'd like steps to reproduce the
>>>     issue so I can see it.
>>>
>>> 2. iproute2's libnetlink.c's rtnl_listen function does not properly
>>>     declare its cmsgbuf with the alignment of cmsghdr; it has type
>>>     char[] so the compiler is free not to align it at all. This is
>>>     presumably a bug in iproute2, but I can't find any good
>>>     documentation (in the standards or Linux-specific) for how you're
>>>     supposed to allocate this space, so maybe the kernel is able to
>>>     handle aligning the buffer itself. I don't see any way the
>>>     alignment of musl's cmsghdr type affects recvmsg though.
>>>
>>> Maybe there are other effects I'm missing? I'll follow up again once I
>>> get a test build/run of iproute2 and let you know whether I can see
>>> the problem.
>> okay. if you need a remote access to a octeon system using musl (my
>> fixed variant), just tell me.
> That would be really helpful. Something's wrong with the userspace for
> the Aboriginal mips64 binaries (SIGBUS in init) and debugging that
> would be a big distraction.
>
> BTW do you have gdb and strace available?
not on the system itself. i'm not sure if strace works on mips64. never 
tried it.
but you're free to copy any binary to the /tmp dir. it has 2 gb ram. so 
enough space for static binaries if you want to play with.
i will send you the ssh data in a private email
>
> Rich
>