Re: nnchoke-retrieve-groups and nnchoke-request-group

Re: nnchoke-retrieve-groups and nnchoke-request-group

[David Hedbor]

anonymous@sunsite.auc.dk writes:

> itenet.net> <m2zptl34gd.fsf@proletcult.slip.ifi.uio.no> <vafvi486b12.fsf@ls6.cs.uni-dortmund.de> <m2u3jriuyo.fsf@proletcult.slip.ifi.uio.no>
> Sender: neotron@mouser.hedbor.org
> X-Url: http://david.hedbor.org/
> X-Face: 3su_O5R!|})4gc|&nXdVILe~2hIncsaJF_<Oc-Jh^/CMkcS<hf^xDqKMy*k"_g5l>TC>rnR
>  NA0Aj<M}5Wt'(7ed8SwQYk-hVi2lG\mkc2$h/Kg706R%]c(14w:>-u\*nln*:ktY!d+[%K()Q|1*-U
>  [+xczKd$Xev:F#3'rFgot\l.I+w>DQ%:%HOyZ]EgFw@0[gMFsR-E`Hs~tjkc>y.<T7~KVE0-$=ca5]
>  LEV[[CXa(!|P:Y--3j|)%&B,FVU(-DDBBHU%aRKLrtO>i#y<55R_CScR
> From: David Hedbor <david@hedbor.org>
> Mime-Version: 1.0 (generated by tm-edit 7.106)
> Content-Type: text/plain; charset=US-ASCII
> Date: 25 May 1997 15:17:37 -0700
> Message-ID: <m3sozbnqm6.fsf@mouser.hedbor.org>
> Lines: 21
> X-Newsreader: Gnus v5.4.50/Emacs 19.34
> Path: c91-rizzo.blarg.net
> NNTP-Posting-Host: c91-rizzo.blarg.net

Why does this happen? It seems like the References-header is mangled
or something. Is it because of it being to long or something? (in this
letter, I removed the references-header before posting).

-- 
	"It's today!" said Piglet.
	"My favorite day," said Pooh.

nnchoke-retrieve-groups and nnchoke-request-group

[Greg Stark]

In rewriting and using my backend i've realized that gnus sometimes uses
*-retrieve-groups to activate a bunch of groups and sometimes repeatedly calls
*-request-group. I think it should always use *-retrieve-groups. That way the
backend can choose to handle it however is most efficient. In my particular
case it's about five times faster and the user feedback is vastly superior
(since i can indicate how many groups remain to be handled) if Gnus uses
*-retrieve-groups. 


On a similar note, if i set gnus-read-active-file to 'some Gnus is smart
enough to only ask for some groups, but it seems to still ask for more than it
really needs to. It seems to ask for anything that isn't a zombie or killed
group, even though i have gnus-activate-level and
gnus-activate-foreign-newsgroups set much lower so it's asking for active file
information about group information it isn't even planning on using.


I think these problems are actually related. It doesn't really make sense to
have a gnus-read-active-file variable, there should maybe be a
nntp-read-active-file which helps decide which NNTP query to use to complete
the retrieve-groups request, but there's no reason for a user to change how
the interface between Gnus and the backends go about they're business. Well i
suppose the distinction between 'some and t is useful, but i don't think the
distinction between nil and 'some should be made by Gnus; i think that should
be decided by an nntp variable.


greg

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

Greg Stark <gsstark@mit.edu> writes:

> In rewriting and using my backend i've realized that gnus sometimes
> uses *-retrieve-groups to activate a bunch of groups and sometimes
> repeatedly calls *-request-group.

It will use the former if you use the `some' setting of
`gnus-read-active-file' and the latter if you use the nil setting.
The `some' setting will, as you've seen, never be slower than the nil
setting. 

> On a similar note, if i set gnus-read-active-file to 'some Gnus is
> smart enough to only ask for some groups, but it seems to still ask
> for more than it really needs to. It seems to ask for anything that
> isn't a zombie or killed group, even though i have
> gnus-activate-level and gnus-activate-foreign-newsgroups set much
> lower so it's asking for active file information about group
> information it isn't even planning on using.

That's true.  Those variables aren't consulted at all when deciding
what groups to ask for active info on.  We need read active info on
all living groups, though, to have functions like the "bogus group
checking" work reliably.

> I think these problems are actually related. It doesn't really make
> sense to have a gnus-read-active-file variable, there should maybe
> be a nntp-read-active-file which helps decide which NNTP query to
> use to complete the retrieve-groups request, but there's no reason
> for a user to change how the interface between Gnus and the backends
> go about they're business. Well i suppose the distinction between
> 'some and t is useful, but i don't think the distinction between nil
> and 'some should be made by Gnus; i think that should be decided by
> an nntp variable.

This is decided by nntp -- if `some' is used.  nil isn't a useful
value, really.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Greg Stark]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Greg Stark <gsstark@mit.edu> writes:
> 
> > In rewriting and using my backend i've realized that gnus sometimes
> > uses *-retrieve-groups to activate a bunch of groups and sometimes
> > repeatedly calls *-request-group.
> 
> It will use the former if you use the `some' setting of
> `gnus-read-active-file' and the latter if you use the nil setting.
> The `some' setting will, as you've seen, never be slower than the nil
> setting. 

Well, i think nil should still use retrieve-groups but should only list the
groups it really really needs. Then nntp should have a way of avoiding the
active file query if that option is needed. Currently some will be slower than
the nil setting if i have lots of groups i don't want to activate, but the nil
setting is very inefficient because my backend is forced to do every query one
by one and wait a round trip for each one instead of submitting lots of
queries at once. 

I don't want to save the killed list because my nntp server has a bazillion
groups, but i do want to have a fair number of groups around that even if i
don't read them often. 

> nil isn't a useful value, really.

Well, i would be happy to whatever setting convinces Gnus to use the algorithm
'some uses but only on the groups nil currently activates.

greg

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

Greg Stark <gsstark@mit.edu> writes:

> Well, i think nil should still use retrieve-groups but should only
> list the groups it really really needs.

That's not really possible using the current methodology.  If you use
a nil `gnus-read-active-file', Gnus will bypass all active file
requests and jump straight to the function for computing unread
articles in the groups.  For each of these groups, it'll do a
`*-request-group' to find out what the active range of articles is.
Using `*-retrieve-groups' requires an extra pass over the newsrc
alist. 

> Then nntp should have a way of avoiding the active file query if
> that option is needed. Currently some will be slower than the nil
> setting if i have lots of groups i don't want to activate, but the
> nil setting is very inefficient because my backend is forced to do
> every query one by one and wait a round trip for each one instead of
> submitting lots of queries at once.

Yes.  So the solution is to kill off all the groups you aren't
interested in and using `some'.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Greg Stark]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Yes.  So the solution is to kill off all the groups you aren't
> interested in and using `some'.

Well that's hardly a solution. Let me explain what i did, what i was trying to
accomplish and what Gnus actually did:

I read groups from various backends including nnfolder, nntp, and nndsc (the
one i wrote for dealing with a network news-like system called discuss). I'm
behind a slow PPP link so i want to minimize the amount of network traffic
Gnus needs to do just to get started. To this end i set the levels of most of
my nnfolder groups to level 2, the nntp and nndsc groups i often read to 4
and the groups i rarely if ever read to 5. Then i set gnus-activate-level and
gnus-activate-foreign-newsgroups to 3. I expected that to make M-x gnus only
fetch information about the few level 2 groups all of which are nnfolder
groups. And when i want to read the other groups i read i can do C-u 4 g to
activate those groups. And I expected Gnus to never bother getting information
on the hundreds of groups at level 5 that i only skim through on rare
occasions.

But that's not how it seems to work out. M-x gnus causes gnus to open the nntp
server even though not a single nntp group is level 2 or 3. And C-u 4 g
doesn't cause gnus to actually activate the level 3 groups, or even open the
nndsc server on which some of the level 4 groups are. Furthermore, if i have
gnus-read-active-file set to 'some then gnus reads active file information on
all the groups level 7 and below it just throws that information away. 

I want to have those hundreds of groups at level 5 available, and i want gnus
to keep track of read ticked articles etc. so killing them isn't appropriate,
but i don't want every time i start up gnus to have to wait while gnus reads
the active file information for them. And I dont want to set
gnus-read-active-file to nil because it's so much less inefficient (in
particular for the nndsc server where it defeats a speed optimization i can
only do for retrieve-groups style batch queries). Ideally gnus wouldn't even
open the nntp server at all unless i was starting it at a level high enough to
include at least one nntp group. That might be hard, but at the very least i
would expect C-u 4 g to activate the groups level 4 and lower and only to
fetch active file information on those groups, not any higher level groups.

greg


PS

It's somewhat unfortunate that M-g doesn't sound like it will ever be able to
do what i want (batch all the groups into a retrieve-groups style request).
I'm not really concerned with the semantic difference between the two queries
that sounds nntp-specific, that the active file can differ and be out of date
compared to the request-group query information. If that is the only problem
then you should just add a flag argument to the retrieve-groups query to
indicate it needs the most up-to-date info and if nntp.el needs to use a
different query to satisfy that that's fine. Other backends are unlikely to
make such a disinction and it's silly to restrict the interface between Gnus
and the backends to an inefficient interface because of it. But it sounds like
it would also be difficult with the current organization of the code. I'll
look at it myself, but if C-u 4 g and C-u 3 g only queried about the groups
they really needed M-g being inefficient wouldn't really bother me as much.

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Justin Sheehy]

Greg Stark <gsstark@MIT.EDU> writes:

> But that's not how it seems to work out. M-x gnus causes gnus to
> open the nntp server even though not a single nntp group is level 2
> or 3.

And what does gnus-no-server do?

-Justin

 

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

Greg Stark <gsstark@mit.edu> writes:

> But that's not how it seems to work out. M-x gnus causes gnus to
> open the nntp server even though not a single nntp group is level 2
> or 3.

To avoid opening the servers, use `M-x gnus-no-server'.

> And C-u 4 g doesn't cause gnus to actually activate the level 3
> groups, or even open the nndsc server on which some of the level 4
> groups are.

Hm, yes...  The prefix to `g' doesn't affect foreign groups.  This
might be a bug.  Should I change this?

> Furthermore, if i have gnus-read-active-file set to 'some then gnus
> reads active file information on all the groups level 7 and below it
> just throws that information away.

It's not thrown away, I think; it's just not used for much.

> It's somewhat unfortunate that M-g doesn't sound like it will ever be able to
> do what i want (batch all the groups into a retrieve-groups style request).
> I'm not really concerned with the semantic difference between the two queries
> that sounds nntp-specific, that the active file can differ and be out of date
> compared to the request-group query information.

It's up to the backend, of course, but the idea is that `M-g', should
report a "thorough" answer while `-request-list' should give a "fast"
answer. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Aaron M. Ucko]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Greg Stark <gsstark@mit.edu> writes:
>
> > And C-u 4 g doesn't cause gnus to actually activate the level 3
> > groups, or even open the nndsc server on which some of the level 4
> > groups are.
> 
> Hm, yes...  The prefix to `g' doesn't affect foreign groups.  This
> might be a bug.  Should I change this?

If you held a poll, I'd vote "yes".

-- 
Aaron M. Ucko (amu@mit.edu) | For Geek Code, PGP public key, and other info,
finger amu@monk.mit.edu. | "Kids! Bringing about Armageddon can be dangerous.
Do not attempt it in your home." -- T. Pratchett & N. Gaiman, _Good Omens_

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Hrvoje Niksic]

amu@MIT.EDU (Aaron M. Ucko) writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> 
> > Greg Stark <gsstark@mit.edu> writes:
> >
> > > And C-u 4 g doesn't cause gnus to actually activate the level 3
> > > groups, or even open the nndsc server on which some of the level 4
> > > groups are.
> > 
> > Hm, yes...  The prefix to `g' doesn't affect foreign groups.  This
> > might be a bug.  Should I change this?
> 
> If you held a poll, I'd vote "yes".

Agreed (or should I say: me too).

-- 
Hrvoje Niksic <hniksic@srce.hr> | Student at FER Zagreb, Croatia
--------------------------------+--------------------------------
Oh lord won't you buy me a color TV...

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Kai Grossjohann]

>>>>> Lars Magne Ingebrigtsen writes:

  Lars> Hm, yes...  The prefix to `g' doesn't affect foreign groups.
  Lars> This might be a bug.  Should I change this?

IMVHO the distinction between primary and secondary servers (foreign
groups are those from secondary servers, aren't they) should be
lessened if possible.

kai
-- 
Life is hard and then you die.

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

Kai Grossjohann <grossjohann@ls6.cs.uni-dortmund.de> writes:

>   Lars> Hm, yes...  The prefix to `g' doesn't affect foreign groups.
>   Lars> This might be a bug.  Should I change this?
> 
> IMVHO the distinction between primary and secondary servers (foreign
> groups are those from secondary servers, aren't they) should be
> lessened if possible.

Primary and secondary groups are always (I think) handled the same.
"Foreign groups" are groups that are neither primary nor secondary.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[François Pinard]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

| Primary and secondary groups are always (I think) handled the same.
| "Foreign groups" are groups that are neither primary nor secondary.  

Food for the manual, most probably! :-)

-- 
François Pinard                                 pinard@iro.umontreal.ca
Support Programming Freedom, join our League!  Ask lpf@lpf.org for info

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

pinard@iro.umontreal.ca (François Pinard) writes:

> | Primary and secondary groups are always (I think) handled the same.
> | "Foreign groups" are groups that are neither primary nor secondary.  
> 
> Food for the manual, most probably! :-)

I've now added this to the "Terminology" section of the manual.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[anonymous]

itenet.net> <m2zptl34gd.fsf@proletcult.slip.ifi.uio.no> <vafvi486b12.fsf@ls6.cs.uni-dortmund.de>
Sender: neotron@mouser.hedbor.org
X-Url: http://david.hedbor.org/
X-Face: 3su_O5R!|})4gc|&nXdVILe~2hIncsaJF_<Oc-Jh^/CMkcS<hf^xDqKMy*k"_g5l>TC>rnR
 NA0Aj<M}5Wt'(7ed8SwQYk-hVi2lG\mkc2$h/Kg706R%]c(14w:>-u\*nln*:ktY!d+[%K()Q|1*-U
 [+xczKd$Xev:F#3'rFgot\l.I+w>DQ%:%HOyZ]EgFw@0[gMFsR-E`Hs~tjkc>y.<T7~KVE0-$=ca5]
 LEV[[CXa(!|P:Y--3j|)%&B,FVU(-DDBBHU%aRKLrtO>i#y<55R_CScR
From: David Hedbor <david@hedbor.org>
Mime-Version: 1.0 (generated by tm-edit 7.106)
Content-Type: text/plain; charset=US-ASCII
Date: 25 May 1997 01:28:12 -0700
Message-ID: <m3g1vcot0j.fsf@mouser.hedbor.org>
Lines: 21
X-Newsreader: Gnus v5.4.50/Emacs 19.34
Path: c91-rizzo.blarg.net
NNTP-Posting-Host: c91-rizzo.blarg.net

Kai Grossjohann <grossjohann@charly.cs.uni-dortmund.de> writes:

> >>>>> Lars Magne Ingebrigtsen writes:
> 
>   Lars> Hm, yes...  The prefix to `g' doesn't affect foreign groups.
>   Lars> This might be a bug.  Should I change this?
> 
> IMVHO the distinction between primary and secondary servers (foreign
> groups are those from secondary servers, aren't they) should be
> lessened if possible.

I agree with you. It might be my lack of Gnus knowledge, but I had
problems subscribing to groups on a secondary server because every
time I wanted to subscribe to something, I had to download the 450k
active file (less than fun on 28.8). It would be nice to have a list
of killed groups (or inactive whatever) on each foreign server.

-- 
It is impossible to travel faster than light, and certainly not desirable,
as one's hat keeps blowing off.
		-- Woody Allen

Re: nnchoke-retrieve-groups and nnchoke-request-group

[anonymous]

itenet.net> <m2zptl34gd.fsf@proletcult.slip.ifi.uio.no> <vafvi486b12.fsf@ls6.cs.uni-dortmund.de> <m2u3jriuyo.fsf@proletcult.slip.ifi.uio.no>
Sender: neotron@mouser.hedbor.org
X-Url: http://david.hedbor.org/
X-Face: 3su_O5R!|})4gc|&nXdVILe~2hIncsaJF_<Oc-Jh^/CMkcS<hf^xDqKMy*k"_g5l>TC>rnR
 NA0Aj<M}5Wt'(7ed8SwQYk-hVi2lG\mkc2$h/Kg706R%]c(14w:>-u\*nln*:ktY!d+[%K()Q|1*-U
 [+xczKd$Xev:F#3'rFgot\l.I+w>DQ%:%HOyZ]EgFw@0[gMFsR-E`Hs~tjkc>y.<T7~KVE0-$=ca5]
 LEV[[CXa(!|P:Y--3j|)%&B,FVU(-DDBBHU%aRKLrtO>i#y<55R_CScR
From: David Hedbor <david@hedbor.org>
Mime-Version: 1.0 (generated by tm-edit 7.106)
Content-Type: text/plain; charset=US-ASCII
Date: 25 May 1997 15:17:37 -0700
Message-ID: <m3sozbnqm6.fsf@mouser.hedbor.org>
Lines: 21
X-Newsreader: Gnus v5.4.50/Emacs 19.34
Path: c91-rizzo.blarg.net
NNTP-Posting-Host: c91-rizzo.blarg.net

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Kai Grossjohann <grossjohann@ls6.cs.uni-dortmund.de> writes:
> 
> >   Lars> Hm, yes...  The prefix to `g' doesn't affect foreign groups.
> >   Lars> This might be a bug.  Should I change this?
> > 
> > IMVHO the distinction between primary and secondary servers (foreign
> > groups are those from secondary servers, aren't they) should be
> > lessened if possible.
> 
> Primary and secondary groups are always (I think) handled the same.
> "Foreign groups" are groups that are neither primary nor secondary.  

Aha, maybe that was my problem. I didn't realize this, as the
secondary and the foreign ones _look_ the same in the group buffer. I
will have to experiment with this some more.

-- 
The road to hell is paved with NAND gates.
		-- J. Gooding

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Greg Stark]

Furthermore there are other instances in which Gnus will repeatedly use the
request-group call on a single group. If i use M-g on a topic or use the
process mark or prefix args to activate a bunch of groups, for example.

Looking purely at the interface between Gnus and the backend it never makes
sense for Gnus to repeatedly query individual groups when it can instead do a
single vastly more efficient query.

greg

Re: nnchoke-retrieve-groups and nnchoke-request-group

[Lars Magne Ingebrigtsen]

Greg Stark <gsstark@mit.edu> writes:

> Furthermore there are other instances in which Gnus will repeatedly use the
> request-group call on a single group. If i use M-g on a topic or use the
> process mark or prefix args to activate a bunch of groups, for example.

`M-g' on a topic is the same as `M-g' on each indiviual group in the
topic.  `*-request-group' and the active file often disagree on what
the active range is, so doing `*-retrieve-groups' here would be the
wrong thing to do.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen