Re: Huge memory consumption on accessing large newsgroup

[Katsumi Yamaoka <yamaoka@jpl.org>]

[-- Attachment #1: Type: text/plain, Size: 582 bytes --]

(I added the ding list to Cc.)

>>>>> Sven Joachim wrote:

> Gnus v5.11
> GNU Emacs 22.1.50.1 (i486-pc-linux-gnu, GTK+ Version 2.10.13)
>  of 2007-08-06 on debian, modified by Debian
> 200 news.motzarella.org InterNetNews NNRP server INN 2.4.4 (20060818 snapshot) ready (posting ok).

I visited http://news.motzarella.org/ and got an account on
Motzarella out of curiosity. ;-)

> When accessing comp.os.linux.misc on news.motzarella.org, a _very_
> large newsgroup with more than 30,000,000 articles, Emacs' memory
> footprint grew heavily.

Yes, now the ACTIVE of that group is:


[-- Attachment #2: Type: application/emacs-lisp, Size: 73 bytes --]

[-- Attachment #3: Type: text/plain, Size: 759 bytes --]


However, I verified there are actually no more than less than
3,000 articles.  It should not be a cause of this problem.

> It took ~2 minutes to display its question
> "How many articles...?" and the memory usage was at 555 MB (RSS).
> When I answered "500" and the summary buffer finally appeared, it grew
> up to 893 MB.  Which is a bit scary, since my computer has "only" 1 GB
> of RAM and is now already paging quite a bit.

> Will this become better if I subscribe to the newsgroup and catch up?
> In any case I probably will have to kill my current Emacs session soon.

The real cause is that Gnus first expands this ACTIVE data into:

(3437 3438 3439 3440 ...... 30538696 30538697 30538698 30538699)

If you run Emacs on a super computer, try this:


[-- Attachment #4: Type: application/emacs-lisp, Size: 43 bytes --]

[-- Attachment #5: Type: text/plain, Size: 116 bytes --]


I think the possible solution is to narrow the range into the
one with which Emacs can work lightly.  Here it is:


[-- Attachment #6: Type: application/emacs-lisp, Size: 183 bytes --]

[-- Attachment #7: Type: text/plain, Size: 576 bytes --]


The 10000 will probably need to be a customizable variable.  I'm
going to do it next week.

BTW, I needed to set the server variable `nntp-authinfo-force'
to t in order to let No Gnus v0.7 send the AUTHINFO data to the
Motzarella server as follows:

(nntp "motzarella"
      (nntp-address "news.motzarella.org")
      (nntp-authinfo-user "yamaoka")
      (nntp-authinfo-password "********")
      (nntp-authinfo-force t)
      ...)

I guess you have the FORCE element in the ~/.authinfo file since
such a server variable has not been implemented yet in Gnus v5.11.

Regards,