From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 32107 invoked from network); 16 Mar 2021 02:13:42 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 16 Mar 2021 02:13:42 -0000 Received: (qmail 15377 invoked by uid 550); 16 Mar 2021 02:13:39 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 14329 invoked from network); 16 Mar 2021 02:13:38 -0000 Date: Mon, 15 Mar 2021 22:13:25 -0400 From: Rich Felker To: Alexander Monakov Cc: musl@lists.openwall.com, Dominic Chen Message-ID: <20210316021325.GH32655@brightrain.aerifal.cx> References: <50e56e3e-d448-c568-b3d8-fbab98939ff8@gmail.com> <20210315215120.GF32655@brightrain.aerifal.cx> <20210315222916.GG32655@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] Issue with fread() and unaligned readv() On Tue, Mar 16, 2021 at 01:42:48AM +0300, Alexander Monakov wrote: > On Mon, 15 Mar 2021, Rich Felker wrote: > > > > If musl always does such off-by-one, it is an efficiency issue (forces > > > a copy with mismatching source/dest alignment). > > > > It's necessary to work around a kernel bug, whereby the kernel fails > > to honor the requirement that a readv of total length n behave > > identically, except for where the data is stored, as a single read of > > length n. For vfs backends that don't implement a proper readv > > operation, the kernel executes readv as a sequence of reads. When this > > happens, if the amount of data to read is exactly the length of the > > first iov (the length requested by the application), continuing to the > > second iov with no more data available will cause the operation to > > block indefinitely until more data is available. By reducing the > > length of the first iov (the caller's buffer) by 1, we ensure that at > > least 1 byte of the second iov (the FILE's buffer) is actually needed > > to satisfy the caller, and thus that the call will return without > > blocking as soon as everything the caller requested is available. > > Thanks. Can musl reduce the first iov tuple by, say, 8 bytes rather than > 1 byte, to avoid forcing the kernel to perform a misaligned copy? Well then you have to do more copy in userspace afterwards, and reduce the effective buffer size by a bit, going back to kernel slightly more often or spending extra memory to compensate. There's also no strong reason to believe one will be aligned and the other won't, except at beginning of file. The alignment mod 8 depends on file position and access history, and neither the caller's buffer nor the FILE buffer have any inherent alignment. Rich