From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 14927 invoked from network); 7 Sep 2008 09:27:19 -0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 Received: from news.dotsrc.org (HELO a.mx.sunsite.dk) (130.225.247.88) by ns1.primenet.com.au with SMTP; 7 Sep 2008 09:27:19 -0000 Received-SPF: none (ns1.primenet.com.au: domain at sunsite.dk does not designate permitted sender hosts) Received: (qmail 82700 invoked from network); 7 Sep 2008 09:26:55 -0000 Received: from sunsite.dk (130.225.247.90) by a.mx.sunsite.dk with SMTP; 7 Sep 2008 09:26:55 -0000 Received: (qmail 11676 invoked by alias); 7 Sep 2008 09:26:47 -0000 Mailing-List: contact zsh-workers-help@sunsite.dk; run by ezmlm Precedence: bulk X-No-Archive: yes X-Seq: 25631 Received: (qmail 11650 invoked from network); 7 Sep 2008 09:26:45 -0000 Received: from bifrost.dotsrc.org (130.225.254.106) by sunsite.dk with SMTP; 7 Sep 2008 09:26:45 -0000 Received: from flock1.newmail.ru (flock1.newmail.ru [82.204.219.207]) by bifrost.dotsrc.org (Postfix) with SMTP id 5D2B6809A182 for ; Sun, 7 Sep 2008 11:26:30 +0200 (CEST) Received: (qmail 20130 invoked from network); 7 Sep 2008 09:25:58 -0000 Received: from unknown (HELO cooker.net) (arvidjaar@newmail.ru@91.77.237.20) by smtpd.newmail.ru with SMTP; 7 Sep 2008 09:25:58 -0000 From: Andrey Borzenkov To: zsh-workers@sunsite.dk Subject: Re: [PATCH] compsys maps anonymous memory and never frees it Date: Sun, 7 Sep 2008 13:25:49 +0400 User-Agent: KMail/1.9.10 References: <48BDF1EC.4050204@gmail.com> <080904072526.ZM12341@torch.brasslantern.com> <48C38F88.9040202@gmail.com> In-Reply-To: <48C38F88.9040202@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1760577.fpd0VPuR9e"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200809071325.56390.arvidjaar@newmail.ru> X-Virus-Scanned: ClamAV 0.92.1/8177/Sun Sep 7 06:29:39 2008 on bifrost X-Virus-Status: Clean --nextPart1760577.fpd0VPuR9e Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sunday 07 September 2008, xRaich[o]=B2x wrote: >=20 > Bart Schaefer wrote: > > On Sep 4, 1:04am, =3D?ISO-8859-1?Q?Bj=3DF6rn_Herzig?=3D wrote: > > } > > } I looked at the problem a little closer. Zsh does not call mmap to > > } allocate them and they dont get allocated when completion happens but > > } when the next command gets issued. > > > > If this is true, then this is something happening down in the library > > or kernel implementation of fork() and is out of zsh's control. > > > > Did you build zsh yourself? Can you check config.h for USE_MMAP ? > > If USE_MMAP is defined then anytime zsh parses a command it will have > > called mmap() to allocate zsh-heap space. You can try reconfiguring > > with --enable-zsh-mem and then check the pmap behavior again. > > > > } So in my example the new maps got added to the process' address space > > } when i executed pmap, but the same happens with any other programm. > > } Builtins however are an exception. So things start to go wrong when it > > } comes to forking. > > > > If you run pmap from another shell window rather than executing it > > from within the shell whose map you're examining, does the behavior > > change at all? > > > > My only guess goes something like this: > > > > Zsh has mapped memory for the heap during parsing etc. Those pages > > have had data written and therefore are marked "dirty". When fork() > > is called, those pages become shared address space with the child > > process. Zsh munmap()s them later but they aren't returned to the > > system because the child process is still using them. > > > > I'm not really happy with any of these explanations yet. > > > > =20 >=20 > Ok, i found it.... after wandering around in a few deadends and getting=20 > some stuff wrong... but well. >=20 > i did the following change to mem.c >=20 > mod_export void > old_heaps(Heap old) > { > Heap h, n; >=20 > queue_signals(); > for (h =3D heaps; h; h =3D n) { > n =3D h->next; > DPUTS(h->sp, "BUG: old_heaps() with pushed heaps"); > #ifdef USE_MMAP > //munmap((void *) h, sizeof(*h)); > munmap((void *) h, h->size); Good catch (it was obvious bug) > #else > //zfree(h, sizeof(*h)); > zfree(h, h->size); In most other places we are using zfree(h, HEAPSIZE); there is no guarantee heap is actually of size HEAPSIZE, but that does not matter (second argument is used only with enable-zsh-mem and serves as plausibility check only). Here is patch in proper format. It is using HEAPSIZE for consistency: Index: Src/mem.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /cvsroot/zsh/zsh/Src/mem.c,v retrieving revision 1.16 diff -u -p -r1.16 mem.c =2D-- Src/mem.c 17 May 2008 17:55:38 -0000 1.16 +++ Src/mem.c 7 Sep 2008 09:23:32 -0000 @@ -153,9 +153,9 @@ old_heaps(Heap old) n =3D h->next; DPUTS(h->sp, "BUG: old_heaps() with pushed heaps"); #ifdef USE_MMAP =2D munmap((void *) h, sizeof(*h)); + munmap((void *) h, h->size); #else =2D zfree(h, sizeof(*h)); + zfree(h, HEAPSIZE); #endif } heaps =3D old; --nextPart1760577.fpd0VPuR9e Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEABECAAYFAkjDnh8ACgkQR6LMutpd94zPSwCgwFd4lzqHI3VZLFwpgN6rzeX5 D28AnR1ttlfgOPNAZfiWLrthhErCqxyH =1w+j -----END PGP SIGNATURE----- --nextPart1760577.fpd0VPuR9e--