From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 447 invoked from network); 9 May 2004 23:52:32 -0000 Received: from ns2.primenet.com.au (HELO primenet.com.au) (6@203.24.36.3) by ns1.primenet.com.au with SMTP; 9 May 2004 23:52:32 -0000 Received: (qmail 3907 invoked from network); 9 May 2004 23:52:29 -0000 Received: from thor.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.86) by proxy.melb.primenet.com.au with SMTP; 9 May 2004 23:52:29 -0000 Received: (qmail 1394 invoked from network); 9 May 2004 23:52:16 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 9 May 2004 23:52:16 -0000 Received: (qmail 23603 invoked by alias); 9 May 2004 23:52:05 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 19903 Received: (qmail 23593 invoked from network); 9 May 2004 23:52:05 -0000 Received: from thor.dotsrc.org (HELO a.mx.sunsite.dk) (qmailr@130.225.247.86) by sunsite.dk with SMTP; 9 May 2004 23:52:02 -0000 Received: (qmail 963 invoked from network); 9 May 2004 23:52:01 -0000 Received: from unknown (HELO moonbase.zanshin.com) (root@167.160.213.139) by a.mx.sunsite.dk with SMTP; 9 May 2004 23:51:59 -0000 Received: from toltec.zanshin.com (toltec.zanshin.com [167.160.213.166]) by moonbase.zanshin.com (8.12.11/8.12.11) with ESMTP id i49Nph3e007586; Sun, 9 May 2004 16:51:43 -0700 Date: Sun, 9 May 2004 16:51:03 -0700 (PDT) From: Bart Schaefer Sender: schaefer@toltec.zanshin.com Reply-To: zsh-workers@sunsite.dk To: zsh-workers@sunsite.dk cc: 245678-submitter@bugs.debian.org Subject: Re: Bug#245678: zsh: built-in rm -rf fills up the memory In-Reply-To: <20040509233619.GB11066@scowler.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Checker-Version: SpamAssassin 2.63 on a.mx.sunsite.dk X-Spam-Level: X-Spam-Status: No, hits=0.0 required=6.0 tests=none autolearn=no version=2.63 X-Spam-Hits: 0.0 On May 9, 7:30pm, Clint Adams wrote: } } > And you're saying that (h->used + (new - old) <= HEAP_ARENA_SIZE) is } > always false, so the zhalloc() is always called? } } I don't know that it's always false, but here's a small snapshot of } those comparisons during a zsh/files rm -rf attempt. } } [ h->used + (new - old) > HEAP_ARENA_SIZE ] } 63432 + (63504 - 63432) > 16360 Aha! Note that h->used > HEAP_ARENA_SIZE all by itself! Comparing to HEAP_ARENA_SIZE is likely wrong, it should compare to the maximum of either HEAP_ARENA_SIZE or the previously-mmapped page size. I'm not entirely sure of that ... ... but that's why it begins re-zhalloc()-ing. In the non-MMAP case it falls back on realloc() when a single allocation exceeds HEAP_ARENA_SIZE. So what probably needs to happen to prevent the out-of-memory condition is that we need to explicitly munmap the old mmap'd block, somewhere. And then we need to emulate realloc() behavior somehow, in terms of knowing whether to actually re-mmap-and-copy or whether the existing mapped pages are large enough to contain the additional requested space.