Slow header download on one group

Slow header download on one group

[Patrick May]

     I see especially slow header downloads when accessing one
particular newsgroup.  Even if that group shows only a dozen unread
posts, it loads much more slowly than a groups with over a hundred.

     elp shows the following for the fast groups:

Function Name                                                                 Call Count  Elapsed Time  Average Time
============================================================================  ==========  ============  ============
gnus-retrieve-headers                                                         3           5.690393      1.8967976666
gnus-group-read-group                                                         1           3.42949       3.42949
gnus-summary-read-group                                                       1           3.429393      3.429393
gnus-summary-read-group-1                                                     1           3.429363      3.429363
gnus-select-newsgroup                                                         1           2.889938      2.889938
gnus-fetch-headers                                                            1           2.650536      2.650536
gnus-cache-retrieve-headers                                                   1           2.638718      2.638718
gnus-agent-retrieve-headers                                                   1           2.637814      2.637814

The slow group shows:

Function Name                                           Call Count  Elapsed Time  Average Time
======================================================  ==========  ============  ============
gnus-retrieve-headers                                   3           78.807254     26.269084666
gnus-group-read-group                                   1           47.025916     47.025916
gnus-summary-read-group                                 1           47.02571      47.02571
gnus-summary-read-group-1                               1           47.025689     47.025689
gnus-select-newsgroup                                   1           42.567341     42.567341
gnus-fetch-headers                                      1           40.455486     40.455486
gnus-cache-retrieve-headers                             1           39.304998     39.304998
gnus-agent-retrieve-headers                             1           39.304134     39.304134

Could fragmentation in .newsrc.eld be the source of the problem?

Thanks,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc.  | Large scale, mission-critical, distributed OO
                       | systems design and implementation.
          pjm@spe.com  | (C++, Java, Common Lisp, Jini, middleware, SOA)

Re: Slow header download on one group

[Katsumi Yamaoka]

>>>>> Patrick May wrote:

>      I see especially slow header downloads when accessing one
> particular newsgroup.  Even if that group shows only a dozen unread
> posts, it loads much more slowly than a groups with over a hundred.

>      elp shows the following for the fast groups:

> Function Name           Call Count  Elapsed Time  Average Time
> =====================  ==========  ============  ============
> gnus-retrieve-headers  3           5.690393      1.8967976666

[...]

> The slow group shows:

> Function Name           Call Count  Elapsed Time  Average Time
> =====================  ==========  ============  ============
> gnus-retrieve-headers  3           78.807254     26.269084666

> Could fragmentation in .newsrc.eld be the source of the problem?

I don't think so.  Though I haven't tried supernews.com, I guess
there might be a big gap between old article numbers and latest
ones in the server.  What does evaluating the following Lisp form
return?

(gnus-active "news.group.name")

It says for my local news server as:

(gnus-active "gnu.emacs.gnus")
 => (57391 . 59194)

but for news.motzarella.org it says:

(gnus-active "nntp+motzarella:gnu.emacs.gnus")
 => (156 . 30068314)

In the later case, Gnus consumes memory wastefully and takes time
while fetching headers even if there are no more than hundreds
articles.  For such cases Gnus offers a workaround.  If it is just
your case, try setting the `gnus-newsgroup-maximum-articles'
variable to a certain number.  For example:

(setq gnus-newsgroup-maximum-articles 10000)

It lets Gnus disregard old articles than the latest 10000
articles in all groups and work fast.

(This variable is not in the released version of Gnus and Emacs.)

Re: Slow header download on one group

[Patrick May]

Katsumi Yamaoka <yamaoka@jpl.org> writes:
>>>>>> Patrick May wrote:
>>      I see especially slow header downloads when accessing one
>> particular newsgroup.  Even if that group shows only a dozen unread
>> posts, it loads much more slowly than a groups with over a hundred.
[ . . . ]
>> Could fragmentation in .newsrc.eld be the source of the problem?
>
> I don't think so.  Though I haven't tried supernews.com, I guess
> there might be a big gap between old article numbers and latest ones
> in the server.  What does evaluating the following Lisp form return?
>
> (gnus-active "news.group.name")

(1122781 . 1963402)

> It says for my local news server as:
>
> (gnus-active "gnu.emacs.gnus")
>  => (57391 . 59194)
>
> but for news.motzarella.org it says:
>
> (gnus-active "nntp+motzarella:gnu.emacs.gnus")
>  => (156 . 30068314)
>
> In the later case, Gnus consumes memory wastefully and takes time
> while fetching headers even if there are no more than hundreds
> articles.

     My difference is 840 thousand or so.  Is that enough to be a
problem?

> For such cases Gnus offers a workaround.  If it is just your case,
> try setting the `gnus-newsgroup-maximum-articles' variable to a
> certain number.  For example:
>
> (setq gnus-newsgroup-maximum-articles 10000)
>
> It lets Gnus disregard old articles than the latest 10000 articles
> in all groups and work fast.

     I'll give that a try.  Thanks for the help!

Regards,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc.  | Large scale, mission-critical, distributed OO
                       | systems design and implementation.
          pjm@spe.com  | (C++, Java, Common Lisp, Jini, middleware, SOA)