From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/65293 Path: news.gmane.org!not-for-mail From: Gaute Strokkenes Newsgroups: gmane.emacs.gnus.general Subject: Re: Huge memory consumption on accessing large newsgroup Date: Sat, 29 Sep 2007 22:04:18 +0100 Message-ID: <87hcldp4j1.fsf@srcf.ucam.org> References: <87wsw4u21m.fsf@gmx.de> <87sl6rh886.fsf@gmx.de> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1191099968 19485 80.91.229.12 (29 Sep 2007 21:06:08 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sat, 29 Sep 2007 21:06:08 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M13805@lists.math.uh.edu Sat Sep 29 23:06:05 2007 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by lo.gmane.org with esmtp (Exim 4.50) id 1IbjVf-0006CO-9f for ding-account@gmane.org; Sat, 29 Sep 2007 23:05:59 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1IbjUk-0000bB-HM; Sat, 29 Sep 2007 16:05:02 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1IbjUh-0000am-Fk for ding@lists.math.uh.edu; Sat, 29 Sep 2007 16:04:59 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtp (Exim 4.67) (envelope-from ) id 1IbjUa-0008Q1-Uo for ding@lists.math.uh.edu; Sat, 29 Sep 2007 16:04:59 -0500 Original-Received: from mtaout01-winn.ispmail.ntl.com ([81.103.221.47]) by quimby.gnus.org with esmtp (Exim 3.35 #1 (Debian)) id 1IbjUS-0000ZB-00 for ; Sat, 29 Sep 2007 23:04:44 +0200 Original-Received: from aamtaout04-winn.ispmail.ntl.com ([81.103.221.35]) by mtaout01-winn.ispmail.ntl.com with ESMTP id <20070929210421.BSWF1783.mtaout01-winn.ispmail.ntl.com@aamtaout04-winn.ispmail.ntl.com> for ; Sat, 29 Sep 2007 22:04:21 +0100 Original-Received: from hagu ([86.7.23.241]) by aamtaout04-winn.ispmail.ntl.com with ESMTP id <20070929210421.SXTK29112.aamtaout04-winn.ispmail.ntl.com@hagu> for ; Sat, 29 Sep 2007 22:04:21 +0100 Original-Received: from gs234 by hagu with local (Exim 4.67) (envelope-from ) id 1IbjU2-0000xr-MR for ding@gnus.org; Sat, 29 Sep 2007 22:04:18 +0100 Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAG1BMVEWWWBTly7aGMwb+/vz9 +ffIilb///+yaCxNCwHGVC3gAAACU0lEQVR4nF2TQWvjMBBGVUGwj3WxaY/aAWNfi0zuwZPuHp0Q ob079OwYjHrsChb0s3dGctJ0ddTzm9F8lsR7WiNiWWL/flviul9Uh0NR4vY/gJUxJjOEtt/AubBy UsotRfH2DbxWE0hom3w5XJUEKis9hACt2hR4B86VDOtqTNHfgeEGvLiUN7Ddze26Dx6y1CUa+ygo NwFQm0N/BeMzCZAtNIoQrd9EhcH5mWpnztB8ZpJBlH23jaAbuE4EhZ2Bam0TeJ0DRMCKVP6CegWk q4z2TVFUx0946FOPcSe9ELE3rSHAj15rBuddS0C4BGyAutfpVAQkKdZOQiwMMIHxlUDrgeej5Ns7 I/ccSWNBLda1XwYBEG1Qswy1yXJf879nA4cjkNI0H+FzoUyuRodPA5eqZwFONG34i30ccOxeKFUy flNe1szqsgLdvXwKCj6zjTXODHV9M46gZPCLlYKCGTYO8S0aOAvJzRfD0w+1XcGIsxLUw/sYl5sM 9n0yjkoSUDOByrijwRRiAq13NhqLJYB3ACazrgN2bIwanyJQKXhHAJPRPQruEaCJP2Shy4DJ2As+ FQjJZH7AZIy6w0pQiiRYAh8nTEZHYDfRqSDeB5OXq4EEcAC4NocvoBl4+n1cyl54nyLRNA3iIwRP wNGFOK1g1Gz8jNfaCjdtqJLm2LXuqMc+50cDIJQpr6X0OHa4X58ULAw03QbRd7GUiG/K04mj0bNB xQhwJkEsJiZ1Bbgb6CnF79cBE+jwF91qniOCP3fghUFKxOShjIATw/wOfIQTg39LpFHtxhlKywAA AABJRU5ErkJggg== Mail-Copies-To: never X-Hashcash: 1:20:070929:ding@gnus.org::ZeedLNSkXvbapgl+:00002juf In-Reply-To: <87sl6rh886.fsf@gmx.de> (Sven Joachim's message of "Fri, 10 Aug 2007 14:42:01 +0200") User-Agent: Gnus/5.110007 (No Gnus v0.7) Emacs/22.1 (gnu/linux) X-Spam-Score: -2.5 (--) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:65293 Archived-At: On 10 aug 2007, svenjoac@gmx.de wrote: > Katsumi Yamaoka writes: > >> I visited http://news.motzarella.org/ and got an account on >> Motzarella out of curiosity. ;-) >> >>> When accessing comp.os.linux.misc on news.motzarella.org, a _very_ >>> large newsgroup with more than 30,000,000 articles, Emacs' memory >>> footprint grew heavily. >> >> Yes, now the ACTIVE of that group is: >> >> (gnus-active "nntp+motzarella:comp.os.linux.misc") >> => (3437 . 30538699) >> >> However, I verified there are actually no more than less than >> 3,000 articles. It should not be a cause of this problem. > > I had already suspected the number is bogus and that they do not > really keep 30 million articles for several newsgroups. ;-) > >> The real cause is that Gnus first expands this ACTIVE data into: >> >> (3437 3438 3439 3440 ...... 30538696 30538697 30538698 30538699) > > Aha, that's not exactly a small list, I see. Sorry for following up on an ancient thread, but: I wonder if it would be possible to make Gnus work solely with compressed ranges (i.e. lists where dotted pairs are used to represent runs of consecutive integers)? Unless there is some deeper reason why this cannot work, I might have a stab at it (eventually). -- Gaute Strokkenes