From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/9882 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: recvmsg/sendmsg broken on mips64 Date: Sun, 10 Apr 2016 18:18:13 -0400 Message-ID: <20160410221812.GP21636@brightrain.aerifal.cx> References: <20160401113146.GU9862@port70.net> <91bfe81c-73c4-9b25-6d9b-a97d4ee54e89@dd-wrt.com> <20160401131712.GV9862@port70.net> <4445e7a7-19f3-aa7c-04dc-3e329ef7fdac@dd-wrt.com> <20160407094806.GE9862@port70.net> <20160407184643.GI9862@port70.net> <2656e404-f225-cd95-3989-a48df486d914@dd-wrt.com> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1460326719 8940 80.91.229.3 (10 Apr 2016 22:18:39 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 10 Apr 2016 22:18:39 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-9895-gllmg-musl=m.gmane.org@lists.openwall.com Mon Apr 11 00:18:34 2016 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1apNgc-0000UT-5Y for gllmg-musl@m.gmane.org; Mon, 11 Apr 2016 00:18:30 +0200 Original-Received: (qmail 19592 invoked by uid 550); 10 Apr 2016 22:18:26 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 19574 invoked from network); 10 Apr 2016 22:18:25 -0000 Content-Disposition: inline In-Reply-To: <2656e404-f225-cd95-3989-a48df486d914@dd-wrt.com> User-Agent: Mutt/1.5.21 (2010-09-15) Original-Sender: Rich Felker Xref: news.gmane.org gmane.linux.lib.musl.general:9882 Archived-At: On Fri, Apr 08, 2016 at 01:33:51AM +0200, Sebastian Gottschall wrote: > Am 07.04.2016 um 20:46 schrieb Szabolcs Nagy: > >* Sebastian Gottschall [2016-04-07 13:42:17 +0200]: > >>>ok so the failure is in sendmsg and in the msg_control copy. > >>> > >>>does the call fail with ENOMEM (because >1024 bytes of ancillary data)? > >>>that would be easy to fix.. > >>> > >>>(libc has to make a copy, the struct is const and might be in > >>>readonly memory. a detailed bug report of the failure would > >>>be more useful than speculations about broken compilers.. > >>>e.g. strace log with and without the msg_control copying.) > >>how to make a more detailed report than just that all netlink operations in > >>iproute2 fail. so the whole ip command doesnt work. > >there are only two places where msg->msg_control > >is used in iproute2: bpf_scm.h and libnetlink.c, > >they both use a fixed char[1024] buffer, which > >should work with musl. > > > >one thing i noticed is that iproute2 fails to > >take cmsghdr alignment requirements into account, > >so it only works by accident. > > > >i think the musl struct has different alignment > >(4 byte instead of 8 byte) which may cause problems > >because the copy uses the musl alignment, i'm > >not sure if this can cause what you observed. > > > >so we still don't know what your problem was > >and what fails exactly. > easy again. iproute2 doesnt work on mips64 targets since recvmsg / > sendmsg call does fail, caused by a wrong structure alignment, bad > structure at all or bad musl code at all. I think what nsz was asking for, and what I'd like to see, is a way to reproduce the bug. I'm going to try building iproute2 for mips64 and running it on a prebuilt kernel from Aboriginal Linux under qemu-system-mips64, but I don't know what specific commands are needed to hit the affected code path. > its all resulting in the same failing recvmsg / sendmsg call.. so > yes libnetlink.c does not work with musl on mips64 (it does work on > x64 and everything else, just not mips64) unless the hack i offered > was applied which again fixed all. > before you ask again for a problem description, just read again. it > wont change the description if you ask again and just makes people > tired on this list. Both versions of the struct (musl's and your modified one that matches the kernel) have the exact same layout, but due to having a member with 64-bit type, yours has 8-byte alignment and musl's only has 4-byte alignment. This means, at least: 1. When musl's sendmsg.c makes its copy to zero out the padding, the copy may not be correctly aligned for 64-bit writes, and the kernel faults or manually produces an error for this case, causing the whole operation to fail. However, I don't see where iproute2 is actually passing control messages to sendmsg, so while this is a problem, I don't think it's the cause. Maybe I'm missing the affected call point; this is why I'd like steps to reproduce the issue so I can see it. 2. iproute2's libnetlink.c's rtnl_listen function does not properly declare its cmsgbuf with the alignment of cmsghdr; it has type char[] so the compiler is free not to align it at all. This is presumably a bug in iproute2, but I can't find any good documentation (in the standards or Linux-specific) for how you're supposed to allocate this space, so maybe the kernel is able to handle aligning the buffer itself. I don't see any way the alignment of musl's cmsghdr type affects recvmsg though. Maybe there are other effects I'm missing? I'll follow up again once I get a test build/run of iproute2 and let you know whether I can see the problem. Rich