Re: [musl] [PATCH v4] implement recallocarray(3)

[Rich Felker <dalias@libc.org>]

On Sun, Aug 02, 2020 at 07:35:23PM -0600, Ariadne Conill wrote:
> Hello,
> 
> On 2020-08-02 18:45, Rich Felker wrote:
> >On Sun, Aug 02, 2020 at 04:54:26PM -0600, Ariadne Conill wrote:
> >>This OpenBSD extension is similar to reallocarray(3), but
> >>zero-initializes the new memory area.
> >>
> >>This extension is placed in _BSD_SOURCE, like
> >>reallocarray(3).
> >>
> >>Changes from v3:
> >>- use calloc() instead of realloc() and always copy
> >>- explicitly zero old memory block
> >>- restore overflow checking for old size
> >>
> >>Changes from v2:
> >>- drop overflow checking for old size
> >>
> >>Changes from v1:
> >>- use realloc() instead of reallocarray()
> >
> >Can we finish discussion of questions raised before making new
> >versions with changes that haven't even been agreed upon? I asked
> >about EINVAL so we could determine if it's expected and if it's
> >reasonable to copy; that's not a demand to remove it. And the new
> >memset before free is a complete no-op. If there's a contract to
> >behave like explicit_bzero on the old memory, we may want to honor
> >that if we implement recallocarray, but that likely makes it entirely
> >unsuitable to the purpose you want it for and will bring back the
> >atrociously bad apk performance you previously saw with mallocng if
> >you use it there, since each increment of array size will incur a
> >whole memcpy of the existing array (quadratic time) rather than copies
> >only occuring a logarithmic number of times (yielding O(n log n)
> >overall) as would happen with realloc.
> 
> At this time, there are no plans to use recallocarray(3) in apk, but
> that would only be a problem if used in a hot path.  I don't
> envision any hot path in apk where we would ever use
> recallocarray(3).  For managing buffers, we would continue using
> realloc() as we do now, since it's all bytes anyway.
> 
> *My* use case for recallocarray(3) is simply that I want a
> reallocarray() that also zeros the new members, *and* I do not want
> to reimplement that function every time I write a program, because
> this is the sort of thing a libc should ideally provide in 2020.  It
> provides calloc() after all, why not a realloc version of it?
> 
> In order to do that, you have to have a similar interface to what
> recallocarray(3) uses, because you need to know where to start
> zeroing. There's not any way around it that is also interposable.
> 
> I do agree that the hardness guarantees of the OpenBSD version
> implement significant overhead, and if we don't provide them, there
> will be some heartbleed type bug down the road.
> 
> So, I stick with v4 patch behavior, because I do not wish for anyone
> to shoot themselves in the foot.

If you want a function that differs from the OpenBSD function, I don't
think this is something that belongs in libc but rather something you
can just drop into projects you want to use it in. Dropping whatever
random utility function seemed useful to have shared between system
programs into libc is a legacy BSD and GNU practice that musl
intentionally does not immitate.

A function that has broad usefulness and consensus across implementors
about what the signature and semantics should be is reasonable for
consideration, but it's looking like that's not what we have here, at
least not yet. I'm open to discussing this on libc-coord if it's
something you want to try to make into a cross-implementation
consensus.

Note that reallocarray still looks fine and is already something glibc
has adopted too. Unlike recallocarray it doesn't have subtle behaviors
that are a tradeoff between usability and hardening. If you have
reallocarray, I think the version of recallocarray you want is a very
thin wrapper around it (on success, if m>om, memset tail).

Rich