Gnus development mailing list
 help / color / mirror / Atom feed
From: Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann)
Subject: Agent downloads too many headers
Date: Tue, 22 Oct 2002 08:20:46 +0200	[thread overview]
Message-ID: <84u1jfvvsx.fsf@crybaby.cs.uni-dortmund.de> (raw)

The situation is as follows: in gnus-agent-fetch-headers, if
gnus-agent-consider-all-articles is non-nil, the list of articles is
set to the active range of the group.  The active range could be
something like (1 . 4711), so the list of articles would be (1 2 ...
4710 4711).  Then we remove from that list the list of
already-downloaded articles.  But probably the user has started
reading the group when the article numbers were greater than 1
already.  So most probably there are a lot of articles in the
low-number range which are not in the group at all.

Then the agent fetches the headers from that group for this list of
articles.

So the agent ends up fetching (almost) all headers for all groups.

What can we do?

We want to avoid fetching headers for the low-numbered articles where
we already learned yesterday that these articles don't exist.

One possibility would be to tell the agent to never fetch articles
with numbers less than what we've already fetched.  This would be (fairly)
easy to implement, but it would lead to a problem: people who start
using the Agent and download the (unread) message 4711, then decide
they want to download old articles, too.  For them, the agent would
never download articles with numbers lower than 4711 because that's
the lowest number fetched already.  One workaround would be to enter
the group and type `C-u J u' which would fetch even those articles.
But I think it is not nice to require them to do that, they might
have lots of groups.

Another possibility is to keep an "unactive list (of ranges)".  This
would be a range of articles known not to exist.  This would require
storing more data in the agent.  But it would also be precise.  I see
two problems with this, a minor one and a major one.  The minor
problem is that I don't know how to store additional data in the
agent.  The major problem is that I don't know if it will be
efficient: the unactive ranges might grow quite large.  For example,
if you start reading a group starting with article 1 and the agent
fetches every tenth article, then the unactive ranges will be ((2 .
10) (12 . 20) ...) and after a few tens of thousands of articles
there will be information about those long-gone articles 1, 11, 21
that have presumably long been expired from the agent.

What do you think?

Please help!

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)



             reply	other threads:[~2002-10-22  6:20 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2002-10-22  6:20 Kai Großjohann [this message]
2002-10-23  6:42 ` Kai Großjohann
2002-10-23 13:55   ` Wes Hardaker
2002-10-23 14:30     ` Kai Großjohann
2002-10-23 14:31     ` Kai Großjohann
2002-10-23 19:14     ` Josh Huber
2002-10-23 15:54 ` Kai Großjohann
2002-10-23 20:33   ` Kai Großjohann
2002-10-23 21:42     ` Henrik Enberg
2002-10-24  7:14       ` Kai Großjohann
2002-10-24 20:51         ` Henrik Enberg
2002-10-25  8:36           ` Kai Großjohann
2002-10-25  7:32     ` Danny Siu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=84u1jfvvsx.fsf@crybaby.cs.uni-dortmund.de \
    --to=kai.grossjohann@cs.uni-dortmund.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).