From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/42983 Path: main.gmane.org!not-for-mail From: Wes Hardaker Newsgroups: gmane.emacs.gnus.general Subject: Agent .overview files not getting truncated (was Re: memory usage results (ack!)) Date: Thu, 07 Feb 2002 14:20:27 -0800 Organization: Network Associates - NAI Labs Sender: owner-ding@hpc.uh.edu Message-ID: References: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: main.gmane.org 1035178148 15102 80.91.224.250 (21 Oct 2002 05:29:08 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Mon, 21 Oct 2002 05:29:08 +0000 (UTC) Return-Path: Original-Received: (qmail 25682 invoked from network); 7 Feb 2002 22:28:26 -0000 Original-Received: from malifon.math.uh.edu (mail@129.7.128.13) by mastaler.com with SMTP; 7 Feb 2002 22:28:26 -0000 Original-Received: from sina.hpc.uh.edu ([129.7.128.10] ident=lists) by malifon.math.uh.edu with esmtp (Exim 3.20 #1) id 16Yx1I-0002Kh-00; Thu, 07 Feb 2002 16:27:56 -0600 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Thu, 07 Feb 2002 16:27:32 -0600 (CST) Original-Received: from sclp3.sclp.com (qmailr@sclp3.sclp.com [209.196.61.66]) by sina.hpc.uh.edu (8.9.3/8.9.3) with SMTP id QAA08334 for ; Thu, 7 Feb 2002 16:26:04 -0600 (CST) Original-Received: (qmail 25651 invoked by alias); 7 Feb 2002 22:26:01 -0000 Original-Received: (qmail 25646 invoked from network); 7 Feb 2002 22:26:00 -0000 Original-Received: from dns2.hardaker.davis.ca.us (HELO wanderer.hardakers.net) (?HUgmDdqw6QVd2qjpsMbjAmVLs52nMOBI?@168.150.190.2) by gnus.org with SMTP; 7 Feb 2002 22:26:00 -0000 Original-Received: (from hardaker@localhost) by wanderer.hardakers.net (8.11.6/8.11.6) id g17MKVN03003; Thu, 7 Feb 2002 14:20:31 -0800 Original-To: ding@gnus.org X-Face: #qW^}a%m*T^{A:Cp}$R\"38+d}41-Z}uU8,r%F#c#s:~Nzp0G9](s?,K49KJ]s"*7gvRgA SrAvQc4@/}L7Qc=w{)]ACO\R{LF@S{pXfojjjGg6c;q6{~C}CxC^^&~(F]`1W)%9j/iS/ IM",B1M.?{w8ckLTYD'`|kTr\i\cgY)P4 In-Reply-To: (Wes Hardaker's message of "Wed, 06 Feb 2002 17:36:33 -0800") Original-Lines: 57 User-Agent: Gnus/5.090006 (Oort Gnus v0.06) XEmacs/21.5 (bamboo, i686-pc-linux) Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:42983 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:42983 >>>>> On Wed, 06 Feb 2002 17:36:33 -0800, Wes Hardaker said: ... Wes> *nntpd*: Wes> text 34088184 ... Wes> Yikes. Ok, so the nttpd buffer (note all these have a leading space) Wes> is 34megs. The agent overview is also 34 megs. Double ouch. Ok, I think I've figured out where the memory has gone: % ll .overview -rw------- 1 hardaker wheel 13765613 Feb 7 14:08 .overview % wc -l .overview 47372 .overview So, there are 47k lines in my agent's .overview file. But there are only a handfull of messages in the directory itself. So this quickly made me check what else is in the .overview file: % egrep '^10015' .overview | wc -l 36 Ahh... So 36 lines are functionally duplicates for a single message. But why? So, I move this file aside and reenter the group and bang! What a speed difference (and I'm sure a memory difference too, but I probably need to restart emacs to claim everything). Now the .overview file: % egrep '^10015' .overview | wc -l 1 Yay. A single line. keystrokes: q ENTER Nada. No change. A good thing. keystrokes: q 2 g ENTER % egrep '^10015' .overview | wc -l 2 Ack. And so we begin again collecting too much data. Apparently the new auto-header-fetch code is duplicating stuff? % ll `find . -name .overview` | awk '{ size=$5 + size } END { print size }' 15750346 15 megs of overview files anyone? (and this is after removing the two largest ones which were 13 megs and 18 megs each. -- "Ninjas aren't dangerous. They're more afraid of you than you are of them."