From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 10514 invoked from network); 3 Aug 2020 01:58:21 -0000 Received: from mother.openwall.net (195.42.179.200) by inbox.vuxu.org with ESMTPUTF8; 3 Aug 2020 01:58:21 -0000 Received: (qmail 5562 invoked by uid 550); 3 Aug 2020 01:58:18 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 5541 invoked from network); 3 Aug 2020 01:58:16 -0000 Date: Sun, 2 Aug 2020 21:58:02 -0400 From: Rich Felker To: musl@lists.openwall.com Message-ID: <20200803015802.GA6949@brightrain.aerifal.cx> References: <20200802225426.32002-1-ariadne@dereferenced.org> <20200803004519.GZ6949@brightrain.aerifal.cx> <782493ee-0e97-8749-ed38-f9e0e8f27298@dereferenced.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <782493ee-0e97-8749-ed38-f9e0e8f27298@dereferenced.org> User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Re: [musl] [PATCH v4] implement recallocarray(3) On Sun, Aug 02, 2020 at 07:35:23PM -0600, Ariadne Conill wrote: > Hello, > > On 2020-08-02 18:45, Rich Felker wrote: > >On Sun, Aug 02, 2020 at 04:54:26PM -0600, Ariadne Conill wrote: > >>This OpenBSD extension is similar to reallocarray(3), but > >>zero-initializes the new memory area. > >> > >>This extension is placed in _BSD_SOURCE, like > >>reallocarray(3). > >> > >>Changes from v3: > >>- use calloc() instead of realloc() and always copy > >>- explicitly zero old memory block > >>- restore overflow checking for old size > >> > >>Changes from v2: > >>- drop overflow checking for old size > >> > >>Changes from v1: > >>- use realloc() instead of reallocarray() > > > >Can we finish discussion of questions raised before making new > >versions with changes that haven't even been agreed upon? I asked > >about EINVAL so we could determine if it's expected and if it's > >reasonable to copy; that's not a demand to remove it. And the new > >memset before free is a complete no-op. If there's a contract to > >behave like explicit_bzero on the old memory, we may want to honor > >that if we implement recallocarray, but that likely makes it entirely > >unsuitable to the purpose you want it for and will bring back the > >atrociously bad apk performance you previously saw with mallocng if > >you use it there, since each increment of array size will incur a > >whole memcpy of the existing array (quadratic time) rather than copies > >only occuring a logarithmic number of times (yielding O(n log n) > >overall) as would happen with realloc. > > At this time, there are no plans to use recallocarray(3) in apk, but > that would only be a problem if used in a hot path. I don't > envision any hot path in apk where we would ever use > recallocarray(3). For managing buffers, we would continue using > realloc() as we do now, since it's all bytes anyway. > > *My* use case for recallocarray(3) is simply that I want a > reallocarray() that also zeros the new members, *and* I do not want > to reimplement that function every time I write a program, because > this is the sort of thing a libc should ideally provide in 2020. It > provides calloc() after all, why not a realloc version of it? > > In order to do that, you have to have a similar interface to what > recallocarray(3) uses, because you need to know where to start > zeroing. There's not any way around it that is also interposable. > > I do agree that the hardness guarantees of the OpenBSD version > implement significant overhead, and if we don't provide them, there > will be some heartbleed type bug down the road. > > So, I stick with v4 patch behavior, because I do not wish for anyone > to shoot themselves in the foot. If you want a function that differs from the OpenBSD function, I don't think this is something that belongs in libc but rather something you can just drop into projects you want to use it in. Dropping whatever random utility function seemed useful to have shared between system programs into libc is a legacy BSD and GNU practice that musl intentionally does not immitate. A function that has broad usefulness and consensus across implementors about what the signature and semantics should be is reasonable for consideration, but it's looking like that's not what we have here, at least not yet. I'm open to discussing this on libc-coord if it's something you want to try to make into a cross-implementation consensus. Note that reallocarray still looks fine and is already something glibc has adopted too. Unlike recallocarray it doesn't have subtle behaviors that are a tradeoff between usability and hardening. If you have reallocarray, I think the version of recallocarray you want is a very thin wrapper around it (on success, if m>om, memset tail). Rich