How to prevent gnus from always downloading full index

How to prevent gnus from always downloading full index

[Andreas Hilboll]

Hello,

I'd like to start using GNUs to read mailing lists via nntp/Gmane. Works
fine so far, but each time I press RET on a group name in the *Group*
buffer, GNUs seems to download a lot of data (in case of the org-mode
list, some 30+M, which takes 10+ seconds).

Maybe I don't understand the workings of NNTP and GNUs properly, but
shouldn't it be possible to somehow cache these data locally? Waiting
many seconds for entering a group seems to be a waste of time,
especially since at this point, GNUs already knows the number of unread
messages (as it's being displayed in the *Group* buffer) ...

Any help is greatly appreciated!
  Andreas.

Re: How to prevent gnus from always downloading full index

[Adam Sjøgren]

Andreas writes:

> I'd like to start using GNUs to read mailing lists via nntp/Gmane. Works
> fine so far, but each time I press RET on a group name in the *Group*
> buffer, GNUs seems to download a lot of data (in case of the org-mode
> list, some 30+M, which takes 10+ seconds).

Sounds strange, Gnus should only download the "overview" for articles.

How many articles do you ask Gnus to display when entering a group?

Have you "caught up" the groups in question (i.e. marked all the old
articles as read)?

What does your configuration look like?


  Best regards,

    Adam

-- 
 "But after all, who is? At least in our case - no one        Adam Sjøgren
  is such a great musician. But we are stubborn."        asjo@koldfront.dk


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Andreas Hilboll]

asjo@koldfront.dk (Adam Sjøgren) writes:

> Andreas writes:
>
>> I'd like to start using GNUs to read mailing lists via nntp/Gmane. Works
>> fine so far, but each time I press RET on a group name in the *Group*
>> buffer, GNUs seems to download a lot of data (in case of the org-mode
>> list, some 30+M, which takes 10+ seconds).
>
> Sounds strange, Gnus should only download the "overview" for articles.
>
> How many articles do you ask Gnus to display when entering a group?

200

> Have you "caught up" the groups in question (i.e. marked all the old
> articles as read)?

Yes, the *group* buffer shows e.g. 2 unread messages for that group.

> What does your configuration look like?

I'm using spacemacs, so the base config is at

   https://github.com/syl20bnr/spacemacs/blob/master/layers/%2Bemail/gnus/packages.el

and I have the following user config:

    (setq gnus-nntp-server nil)
    (setq gnus-select-method
          '(nntp "news.gmane.org"
                 (nntp-connection-timeout 240)
                 (nntp-open-connection-function nntp-open-network-stream)
                 (nntp-address "news.gmane.org")
                 )
          )

    ;; Archive outgoing email in Sent folder on local Maildir
    (setq gnus-message-archive-method '(nnmaildir "Private" (directory "~/mail/Private"))
          gnus-message-archive-group "Sent")

    ;; set return email address based on incoming email address
    (setq gnus-posting-styles
          '((".*"
             (address "andreas-h@posteo.de"))))

    ;; mark Gcc (group Cc) messages as read, from http://stackoverflow.com/q/9799871/152439
    (setq gnus-gcc-mark-as-read t)

When from the *Group* buffer I have point on a group and call one of
`gnus-topic-select-group`, `gnus-group-read-group` and
`gnus-group-select-group`, then I can see "nntp read XXXX" in the
minibuffer, with a counter going to ~5k in case of this newsgroup.  When
from the summary buffer i press q to go back to the *group* buffer, and
press RET on the same group again, it again does the "nntp read XXXX"
thing.

Thanks for your help!
  Andreas.

_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Eric Abrahamsen]

Andreas Hilboll <andreas-h@posteo.de> writes:

> asjo@koldfront.dk (Adam Sjøgren) writes:
>
>> Andreas writes:
>>
>>> I'd like to start using GNUs to read mailing lists via nntp/Gmane. Works
>>> fine so far, but each time I press RET on a group name in the *Group*
>>> buffer, GNUs seems to download a lot of data (in case of the org-mode
>>> list, some 30+M, which takes 10+ seconds).

Have you agentized the server ("J a" in the *Server* buffer)? I can't
say for sure, but I don't think Gnus will keep a cache by default.

Eric


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Adam Sjøgren]

Andreas writes:

>> How many articles do you ask Gnus to display when entering a group?

> 200

I think it should only fetch the "overview" for the newest 200 then. Odd.

>> Have you "caught up" the groups in question (i.e. marked all the old
>> articles as read)?

> Yes, the *group* buffer shows e.g. 2 unread messages for that group.

Then it should only fetch 2 when you enter the group.

>> What does your configuration look like?
>
> I'm using spacemacs, so the base config is at
>
>    https://github.com/syl20bnr/spacemacs/blob/master/layers/%2Bemail/gnus/packages.el

Ah! This looks suspicious:

  gnus-fetch-old-headers t

From the documentation:

 "This feature can seriously impact performance it ignores all locally
  cached header entries. Setting it to t for groups for a server that
  doesn't expire articles (such as news.gmane.org), leads to very slow
  summary generation."

I have nil here. I'm pretty sure this is the culprit.

Setting this twice seems odd:

  gnus-keep-backlog 'nil
  gnus-keep-backlog '0

I have it set to 20. I think 'nil and '0 are weird values to use - nil
or 0 would make more sense, but keeping the default 20 maybe even more :-)


  Best regards,

    Adam

-- 
 "It's like somebody with obsessive-compulsive                Adam Sjøgren
  disorder constantly rearranging the names to fit       asjo@koldfront.dk
  into some sort of perfect ontology they have in
  their heads."


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Andreas Hilboll]

Adam Sjøgren <asjo <at> koldfront.dk> writes:

> Ah! This looks suspicious:
> 
>   gnus-fetch-old-headers t
> 
> From the documentation:
> 
>  "This feature can seriously impact performance it ignores all locally
>   cached header entries. Setting it to t for groups for a server that
>   doesn't expire articles (such as news.gmane.org), leads to very slow
>   summary generation."
> 
> I have nil here. I'm pretty sure this is the culprit.

Indeed, you're right.  Thanks for your help, Adam!

Best,
  Andreas.
_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Adam Sjøgren]

Andreas writes:

> Adam Sjøgren <asjo <at> koldfront.dk> writes:
>
>> Ah! This looks suspicious:
>> 
>>   gnus-fetch-old-headers t
>> 
>> From the documentation:
>> 
>>  "This feature can seriously impact performance it ignores all locally
>>   cached header entries. Setting it to t for groups for a server that
>>   doesn't expire articles (such as news.gmane.org), leads to very slow
>>   summary generation."
>> 
>> I have nil here. I'm pretty sure this is the culprit.
>
> Indeed, you're right.  Thanks for your help, Adam!

Great! You're welcome.

It might be worth to feed this information back to the spaceemacs(?)
folks?


  Best regards,

    Adam

-- 
 "Since it is completely undocumented, your faith that        Adam Sjøgren
  it will stick around until they invent v4 is           asjo@koldfront.dk
  charmingly quaint."


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[B.V. Raghav]

Hi,

>>> Ah! This looks suspicious:
>>> 
>>>   gnus-fetch-old-headers t

I run:
GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2015-10-25 on trouble, modified by Debian

and the C-h v gnus-fetch-old-headers value is nil

I have a little different issue. When I subscribe to a group like
debian.user -- The history is pretty large, 150MB or so. I had marked it
read by using `c' key to catch-up the group.

However, now when entering the summary buffer, to check unread mails, I
find that Gnus downloads 150M, almost every time to retrieve the
history.

I am paranoid, and sort of never check unread news there!

Thanks,
r

-- 
(B.V. Raghav)

Re: How to prevent gnus from always downloading full index

[Adam Sjøgren]

B.V. writes:

> and the C-h v gnus-fetch-old-headers value is nil
>
> I have a little different issue. When I subscribe to a group like
> debian.user -- The history is pretty large, 150MB or so. I had marked it
> read by using `c' key to catch-up the group.
>
> However, now when entering the summary buffer, to check unread mails, I
> find that Gnus downloads 150M, almost every time to retrieve the
> history.

So, to give us any chance of guessing what the problem might be in your
setup, you'll have to share your configuration as well.


  Best regards,

    Adam

-- 
 "the office is quiet now, i am at the desk, preparing        Adam Sjøgren
  to landscape for another day."                         asjo@koldfront.dk


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[B.V. Raghav]

Hi,

I am sorry.

>> and the C-h v gnus-fetch-old-headers value is nil
           ^^^^^                                 ^^^ This was from my
           box at home! and the one that misbehaved was the one that was
           at workplace. And indeed it had the same configuration error

The erroneous behaviour was indeed because of
(setq gnus-fetch-old-headers t)
in the .gnus.el file

Sorry for the noise.

r
-- 
(B.V. Raghav)

Re: How to prevent gnus from always downloading full index

[Adam Sjøgren]

B.V. writes:

>>> and the C-h v gnus-fetch-old-headers value is nil
>            ^^^^^                                 ^^^ This was from my
>            box at home! and the one that misbehaved was the one that was
>            at workplace. And indeed it had the same configuration error

Ah, ok, great that you found the reason!


  Best regards,

    Adam

-- 
 "HALLELUJAH!                                                 Adam Sjøgren
  Så att säga."                                          asjo@koldfront.dk


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: How to prevent gnus from always downloading full index

[Andreas Hilboll]

asjo@koldfront.dk (Adam Sjøgren) writes:

> Andreas writes:
>
>> Adam Sjøgren <asjo <at> koldfront.dk> writes:
>>
>>> Ah! This looks suspicious:
>>> 
>>>   gnus-fetch-old-headers t
>>> 
>>> From the documentation:
>>> 
>>>  "This feature can seriously impact performance it ignores all locally
>>>   cached header entries. Setting it to t for groups for a server that
>>>   doesn't expire articles (such as news.gmane.org), leads to very slow
>>>   summary generation."
>>> 
>>> I have nil here. I'm pretty sure this is the culprit.
>>
>> Indeed, you're right.  Thanks for your help, Adam!
>
> Great! You're welcome.
>
> It might be worth to feed this information back to the spaceemacs(?)
> folks?

I'll start the discussion there.  The thing is, in Spacemacs, the Gnus
configuration seems to be geared towards e-mail / IMAP usage.  But at
least I'll document this behaviour on the Spacemacs side.


_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
https://lists.gnu.org/mailman/listinfo/info-gnus-english