RE: LUA + musl, garbage collection issue?

[Scott Valentine <scottvalen@hotmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3342 bytes --]



> Date: Sun, 21 Sep 2014 00:38:31 -0400
> From: dalias@libc.org
> To: musl@lists.openwall.com
> Subject: Re: [musl] LUA + musl, garbage collection issue?
> 
> On Sat, Sep 20, 2014 at 04:41:14PM -1000, Scott Valentine wrote:
> > I noticed that in order to free memory, it basically calls realloc
> > with 0 as the new size. Is this something musl doesn't handle well?
> > 
> > I'm trying a rebuild with a check for n == 0 in musl's realloc
> > function to just free the pointer, and I'll report back.
> > 
> > What is "the right thing to do" to fix this? Should lua not be using
> > realloc to free memory, or should musl handle the case better, if,
> > in fact this is the problem?
> 
> This is a bug in lua; it's depending on a bug in glibc. POSIX attempts
> to allow the glibc behavior (see the text here:
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/realloc.html)
> but this allowance is invalid since it conflicts with the requirements
> of ISO C (and, as you can see in the above link, "The functionality
> described on this reference page is aligned with the ISO C standard.
> Any conflict between the requirements described here and the ISO C
> standard is unintentional. This volume of POSIX.1-2008 defers to the
> ISO C standard.") The relevant ISO C requirement is:
> 
> "The realloc function returns a pointer to the new object (which may
> have the same value as a pointer to the old object), or a null pointer
> if the new object could not be allocated."
> 
> In particular, the only way realloc is permitted to return 0 is if the
> operation failed, in which case the old pointer is still valid (not
> freed).
> 
> But even if the glibc behavior weren't conflicting with the C
> standard, it's still not valid for an application to assume it, and
> it's still undesirable behavior (because it's inconsistent with
> glibc's malloc, which returns a non-null, unique pointer for each
> malloc(0), and because it makes it impossible for applications to
> reliably determine whether the operation succeeded or failed).
> 
> The whole situation is a big enough mess that I feel I can safely say
> that any application that ever passes 0 as the size argument to
> realloc has potentially-serious bugs. So if lua wants to free memory,
> it just needs to call free.
> 
> Rich
Interestingly, I added the explicit free and return 0 at the top of musl's realloc function, but I am still seeing the same behavior, so now I am scratching my head. I've eliminated most pieces of the stack as sources of the problem. I can use scp to copy the file without issues, uhttpd pretty much never mallocs anything, I turned off ssl, and luci is almost entire written in lua. It does use the nixio library to actually write the file, but that doesn't really malloc anything and is really just a lightweight lua to native wrapper.
I'm going to try valgrind on the mips target to see who is causing trouble.
Could this possibly have something to do with the read/write stdio functions in musl? I can't really see how, but at present that is the only other implementation specific feature I can find that is relevant. Well, that and musl uses BUFSIZ = 1024 and glibc uses BUFSIZ = 8192. LUA references this variable directly, but it is really only a default.
Thanks,-Scott V. 		 	   		  

[-- Attachment #2: Type: text/html, Size: 3941 bytes --]