Re: Bug#245678: zsh: built-in rm -rf fills up the memory

[Bart Schaefer <schaefer@brasslantern.com>]

On Sat, 8 May 2004, Clint Adams wrote:

> So, for example, if you have a directory with one hundred files with
> 9-byte filenames, zsh might allocate one hundred 4096-byte areas.

If that's really what's happening, it's a bug somewhere else, because zsh
should only allocate another heap page if the current heap page doesn't
have room for the requested growth.

That is, 409 9-byte file names should need 4090 bytes (add 1 to each for
the NUL byte), and it should put all 4090 of those bytes in the same
4096-byte heap block.  It shouldn't allocate a new heap block until it
gets to the 410th file, at which point it should allocate another
8192-byte block and copy the old block into it, and return the original
4096-byte block to the heap pool for re-use.

(Where I'm assuming HEAP_ARENA_SIZE is 4096 here.)

The following code should *never* be executed unless hrealloc() has
previously been used to *shrink* a block:

    if (p + old < arena(h) + h->used) {
	if (new > old) {
	    char *ptr = (char *) zhalloc(new);
	    memcpy(ptr, p, old);
#ifdef ZSH_MEM_DEBUG
	    memset(p, 0xff, old);
#endif
	    unqueue_signals();
	    return ptr;
	} else {
	    unqueue_signals();
	    return new ? p : NULL;
	}
    }

Because unless we shrank, (p + old == arena(h) + h->used), as asserted by
the DPUTS() that immediately follows.

So, taking #ifdefs for MMAP into account, this code is what should run:

    if (h->used + (new - old) <= HEAP_ARENA_SIZE) {
	h->used += new - old;
	unqueue_signals();
	return p;
    } else {
	char *t = zhalloc(new);
	memcpy(t, p, old > new ? new : old);
	h->used -= old;
#ifdef ZSH_MEM_DEBUG
	memset(p, 0xff, old);
#endif
	unqueue_signals();
	return t;
    }

And you're saying that (h->used + (new - old) <= HEAP_ARENA_SIZE) is 
always false, so the zhalloc() is always called?

If that's not what you find to be the case, then the bug is elsewhere and
we need to keep looking.  If it is what you find, then there's a problem
with h->used record-keeping, or something.