From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 13127 invoked from network); 10 May 2004 17:24:51 -0000 Received: from thor.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.86) by ns1.primenet.com.au with SMTP; 10 May 2004 17:24:51 -0000 Received: (qmail 19258 invoked from network); 10 May 2004 17:24:36 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 10 May 2004 17:24:36 -0000 Received: (qmail 28911 invoked by alias); 10 May 2004 17:24:24 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 19909 Received: (qmail 28902 invoked from network); 10 May 2004 17:24:24 -0000 Received: from thor.dotsrc.org (HELO a.mx.sunsite.dk) (qmailr@130.225.247.86) by sunsite.dk with SMTP; 10 May 2004 17:24:21 -0000 Received: (qmail 18743 invoked from network); 10 May 2004 17:24:20 -0000 Received: from lhuumrelay3.lnd.ops.eu.uu.net (62.189.58.19) by a.mx.sunsite.dk with SMTP; 10 May 2004 17:24:19 -0000 Received: from MAILSWEEPER01.csr.com (mailhost1.csr.com [62.189.183.235]) by lhuumrelay3.lnd.ops.eu.uu.net (8.11.0/8.11.0) with ESMTP id i4AHNqv23796 for ; Mon, 10 May 2004 17:23:52 GMT Received: from EXCHANGE02.csr.com (unverified [192.168.137.45]) by MAILSWEEPER01.csr.com (Content Technologies SMTPRS 4.3.12) with ESMTP id ; Mon, 10 May 2004 18:23:22 +0100 Received: from csr.com ([192.168.144.127]) by EXCHANGE02.csr.com with Microsoft SMTPSVC(5.0.2195.6713); Mon, 10 May 2004 18:25:25 +0100 To: zsh-workers@sunsite.dk cc: 245678-submitter@bugs.debian.org Subject: Re: Bug#245678: zsh: built-in rm -rf fills up the memory In-reply-to: "Bart Schaefer"'s message of "Mon, 10 May 2004 09:51:53 PDT." Date: Mon, 10 May 2004 18:23:52 +0100 Message-ID: <11453.1084209832@csr.com> From: Peter Stephenson X-OriginalArrivalTime: 10 May 2004 17:25:25.0351 (UTC) FILETIME=[C7DE7F70:01C436B3] X-Spam-Checker-Version: SpamAssassin 2.63 on a.mx.sunsite.dk X-Spam-Level: ** X-Spam-Status: No, hits=2.1 required=6.0 tests=BAYES_90 autolearn=no version=2.63 X-Spam-Hits: 2.1 Bart Schaefer wrote: > > However, I don't think even *that* is the basic problem. I think this: > > > > old = (old + H_ISIZE - 1) & ~(H_ISIZE - 1); > > new = (new + H_ISIZE - 1) & ~(H_ISIZE - 1); > > > > is wrong --- it should be the heap size here, not H_ISIZE which is > > simply sizeof(union mem_align), which is probably only 4 or 8 words. > > It took me a while staring at this to decide that it was doing the right > thing, but I came to the conclusion that it's only actually _wrong_ if > HEAP_ARENA_SIZE is _not_ a multiple of H_ISIZE. And even then I'm pretty > sure "wrong" just means "inefficient". I'd have to follow through with a debugger, really... But I can't see any mechanism to prevent it reallocating for every few bytes of increase once the size goes beyond HEAP_ARENA_SIZE, which can't be the intention. The chunk #ifdef'd out for USE_MMAP is the first point where we see if we have enough space for `new' already (except for special cases old == new, we allocated the space then shrank, new is NULL). So for new and old larger than HEAP_ARENA_SIZE (which is bound to happen after the first attempt to grow), we call realloc() every time the difference crosses an H_ISIZE boundary. What have I missed? > > We only need to reallocate to heap size boundaries > > That's correct, but we _do_ only reallocate to heap size boundaries in the > non-MMAP case. I think. It's only the MMAP'd code, which as I said was > pasted in long after the original algorithm was written, that botches it. Yes, but we then call realloc unnecessarily in between actually growing the heap, unless my logic above is wrong. It will reallocate even though the new size of the heap is the same as the old one. > > I might be tempted by proposed solutions involving the complete removal > > of hrealloc. > > Which would mean what, exactly? Never grow heap blocks? hrealloc is only used as an optimisation. If necessary we could always allocate new space. In the case of the files module we could use zalloc and zrealloc and let the system worry about the problem. Then we lose MMAP altogether, of course. -- Peter Stephenson Software Engineer CSR Ltd., Science Park, Milton Road, Cambridge, CB4 0WH, UK Tel: +44 (0)1223 692070 ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com **********************************************************************