From mboxrd@z Thu Jan  1 00:00:00 1970
X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/65038
Path: news.gmane.org!not-for-mail
From: Sven Joachim <svenjoac@gmx.de>
Newsgroups: gmane.emacs.gnus.general
Subject: Re: Huge memory consumption on accessing large newsgroup
Date: Fri, 10 Aug 2007 14:43:32 +0200
Message-ID: <87ps1vh85n.fsf@gmx.de>
References: <87wsw4u21m.fsf@gmx.de> <b4m3ayriwp1.fsf@jpl.org>
	<b4mvebnty89.fsf@jpl.org>
NNTP-Posting-Host: lo.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Trace: sea.gmane.org 1186749673 13610 80.91.229.12 (10 Aug 2007 12:41:13 GMT)
X-Complaints-To: usenet@sea.gmane.org
NNTP-Posting-Date: Fri, 10 Aug 2007 12:41:13 +0000 (UTC)
Cc: bugs@gnus.org,  ding@gnus.org
To: Katsumi Yamaoka <yamaoka@jpl.org>
Original-X-From: ding-owner+M13548@lists.math.uh.edu Fri Aug 10 14:41:09 2007
Return-path: <ding-owner+M13548@lists.math.uh.edu>
Envelope-to: ding-account@gmane.org
Original-Received: from util0.math.uh.edu ([129.7.128.18])
	by lo.gmane.org with esmtp (Exim 4.50)
	id 1IJTnf-0005Y1-ON
	for ding-account@gmane.org; Fri, 10 Aug 2007 14:41:08 +0200
Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu)
	by util0.math.uh.edu with smtp (Exim 4.63)
	(envelope-from <ding-owner+M13548@lists.math.uh.edu>)
	id 1IJTnd-0002Xo-PU; Fri, 10 Aug 2007 07:41:05 -0500
Original-Received: from mx2.math.uh.edu ([129.7.128.33])
	by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.63)
	(envelope-from <svenjoac@gmx.de>)
	id 1IJTnc-0002XN-9m
	for ding@lists.math.uh.edu; Fri, 10 Aug 2007 07:41:04 -0500
Original-Received: from quimby.gnus.org ([80.91.231.51])
	by mx2.math.uh.edu with esmtp (Exim 4.67)
	(envelope-from <svenjoac@gmx.de>)
	id 1IJTnY-00050O-I0
	for ding@lists.math.uh.edu; Fri, 10 Aug 2007 07:41:04 -0500
Original-Received: from mail.gmx.net ([213.165.64.20])
	by quimby.gnus.org with smtp (Exim 3.35 #1 (Debian))
	id 1IJTnX-0000rH-00
	for <ding@gnus.org>; Fri, 10 Aug 2007 14:40:59 +0200
Original-Received: (qmail invoked by alias); 10 Aug 2007 12:40:28 -0000
Original-Received: from p54863BE1.dip.t-dialin.net (EHLO debian) [84.134.59.225]
  by mail.gmx.net (mp017) with SMTP; 10 Aug 2007 14:40:28 +0200
X-Authenticated: #28250155
X-Provags-ID: V01U2FsdGVkX18Xfsd9y8gXRhBG+252hYt1Mo8PsKPq7f/Lx592qm
	aTwS+ksQLtB/DY
In-Reply-To: <b4mvebnty89.fsf@jpl.org> (Katsumi Yamaoka's message of "Fri\, 10 Aug 2007 20\:39\:34 +0900")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1.50 (gnu/linux)
X-Y-GMX-Trusted: 0
X-Spam-Score: -2.6 (--)
List-ID: <ding.gnus.org>
Precedence: bulk
Xref: news.gmane.org gmane.emacs.gnus.general:65038
Archived-At: <http://permalink.gmane.org/gmane.emacs.gnus.general/65038>

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> I overlooked that the argument of this function can be complicated.
> Try the following one instead:
>
> (defadvice gnus-uncompress-range (before narrow-range (ranges) activate)
>   "Narrow the range if it is unreasonably wide."
>   (let ((ttl 10000)
> 	(rest (if (and (cdr ranges) (not (consp (cdr ranges))))
> 		  (list ranges)
> 		(nreverse ranges)))
> 	range diff)
>     (setq ranges nil)
>     (while rest
>       (setq range (car rest)
> 	    rest (cdr rest))
>       (if (numberp range)
> 	  (progn
> 	    (push range ranges)
> 	    (setq ttl (1- ttl)))
> 	(if (= ttl 1)
> 	    (progn
> 	      (push (cdr range) ranges)
> 	      (setq ttl 0))
> 	  (setq diff (min (- (cdr range) (car range) -1) ttl)
> 		ttl (- ttl diff))
> 	  (push (cons (max (car range) (- (cdr range) diff -1))
> 		      (cdr range))
> 		ranges)))
>       (when (zerop ttl)
> 	(setq rest nil)))))

Does not seem to help much, it seems. :-)  With this advice, Emacs'
memory footprint grew to 303 MB on accessing be.politics at
news.motzarella.org, another random example of a group with supposedly
30m+ articles.  And fetching 50 headers increased it to 574 MB.

Kind regards,
             Sven