Re: Excessive nntp reads since today

[Tassilo Horn <tassilo@member.fsf.org>]

Wolfgang Jenkner <wjenkner@inode.at> writes:

>> It matches the behavior in that it had to fetch 2 additional messages
>> to fill in gaps in lose thread branches.
>
> The docstring states that it fetches all old headers (though only some
> subset of them is displayed).

You are right.  But speaking from experience it didn't do so, yet the
effect of connecting loose threads with old read articles somehow seemed
to work.

Ok, now I've edebugged the function as you suggested and checked what's
going on.  The difference between now and then is that now all headers
of a group are fetched once and put into the .overview (NOV) files.
Then they are cached and entering a group (even with 'some) is fast
again.

So basically, before uncached-articles was always the list of unread
articles, now it is (set-difference all-articles articles-in-overview).

      (if (setq uncached-articles (gnus-agent-uncached-articles articles group
                                                                t))
          (progn
            ;; Populate nntp-server-buffer with uncached headers
            (set-buffer nntp-server-buffer)
            (erase-buffer)

The reason that I get those excessive nntp reads (only once for each
group) is that my overview files didn't start with article 1, because
when subscribing to a new group, I restricted fetching by entering it
with, say, C-u 1000 RET and then catching up.

So to conclude:

  - Your patch is correct with respect to the docs

  - Your patch is bad, because now you can't use 'some for
    `gnus-fetch-old-headers' unless you have a super-fast internet
    connection.  (Several gmane groups have millions of messages which
    on first entering will all be downloaded.  No way to restrict that
    with a prefix arg as I always did.  Also, you end up with huge
    .overview files, which slow down Gnus' operations quite a bit.)

  - The old and buggy 'some semantics of `gnus-fetch-old-headers' were
    exactly what I want.  I.e., I want Gnus to *fetch* only new
    messages, but it should also *display* summary lines of old messages
    to fill loose threads in case those can be determined from the
    overview.

Now what?  Possibly `gnus-fetch-old-headers' should be split into two
variables to entangle fetching from displaying?  That'd be pretty bad
with respect to backwards compatibility...

Or how about allowing values like '(some . 100) meaning fetch at most
100 old headers and display old articles where needed to fill loose
threads?

Bye,
Tassilo