Why so many k data when just fetching headers.

Why so many k data when just fetching headers.

[Mats Lidell]

Hi,

Just courious about how much data is fetched when reading the
headers. The status info in the mini-bugger can show as much as 30-40k
data for a newsgroup with 30 new messages. That's much more than I was
expecting.

Comments?

%% Mats

Re: Why so many k data when just fetching headers.

[Lars Magne Ingebrigtsen]

Mats Lidell <Mats.Lidell@contactor.se> writes:

> Just courious about how much data is fetched when reading the
> headers. The status info in the mini-bugger can show as much as 30-40k
> data for a newsgroup with 30 new messages. That's much more than I was
> expecting.

It depends on your variable settings (gnus-fetch-old-headers in
particular) and whether the 30 unread messages were spread out
numerically. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Why so many k data when just fetching headers.

[visigoth]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> It depends on your variable settings (gnus-fetch-old-headers in
> particular) and whether the 30 unread messages were spread out
> numerically. 

Hmm.  Does Gnus do anything fancy to try to detect if the news server
provides overview lines?  We use a little proxy thing locally which
will provide overviews, but produces errors on "list overview.fmt".
Will Gnus switch to using headers because of this?

On a related note--the tool that we use is called dnntpd.  It merges
the output of two or more news servers into one.  Using this, we have
separate machines to spool "frivolous" groups and "important" groups,
and could split further.  This allows the machines to be less heavily
loaded and allows us to better manage disk space.  dnntpd, though, is
not the best program in the world.  How hard would a dnntp-like
backend be to write?  Specifically, a backend that would take a list
of virtual servers and provide each newsgroup from the first server on
the list that carries it, and merge ACTIVE files from those servers?

Gnus can sort of already do this by having multiple servers, but it
doesn't really support transparently moving groups from one server to
another and back, which we can do with dnntpd.  The main problem I see
with this is if two new servers don't use the same numbering
information--but this shouldn't be a problem if, as in our case, one
server carries some groups, and the other carries some other groups.
Or, in another case, the preferred server never "loses" the group.

Can I get this on the list of things for Quassia Gnus, or is it too
special-purpose?

John.

Re: Why so many k data when just fetching headers.

[Lars Magne Ingebrigtsen]

visigoth@naiad.fac.cs.cmu.edu writes:

> Hmm.  Does Gnus do anything fancy to try to detect if the news server
> provides overview lines?  We use a little proxy thing locally which
> will provide overviews, but produces errors on "list overview.fmt".
> Will Gnus switch to using headers because of this?

No -- Gnus will try to do an XOVER and see what it gets back.

> Specifically, a backend that would take a list of virtual servers
> and provide each newsgroup from the first server on the list that
> carries it, and merge ACTIVE files from those servers?

Not hard at all -- it's just a SMOP.

> Can I get this on the list of things for Quassia Gnus, or is it too
> special-purpose?

It seems to special-purpose to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen