The A A command takes forever to complete

The A A command takes forever to complete

[Magnus Hammerin]

The subject says it all, I have actually never waited for it to complete
(I have waited for several hours though).

Is it just my setup or is this a problem for other people as well?!??

I have checked the newserver and I the list active commands completes in
a matter of seconds.

I am running rgnus-0.68 in XEmacs 19.14.

-Magnus.

Re: The A A command takes forever to complete

[David Moore]

Magnus Hammerin <magnush@epact.se> writes:

> The subject says it all, I have actually never waited for it to complete
> (I have waited for several hours though).
> 
> Is it just my setup or is this a problem for other people as well?!??

	It's a known "bug" (quotes because it technically does work,
just not in your lifetime).  Fixing it is currently on the top of my
performance list.  You can expect it to take several hours if you have
have 10000+ newsgroups.

	The same bug occurs with 'Y c' in the summary buffer.  So I
recommend never hitting 'Y c' unless you only have less than about 10
cached articles.  It takes about 8 minutes to insert 145 cached articles
into the summary buffer on my machine. ;)  So, it's usually faster just
to hit 'C-u Z R'.

	The 'A A' bug could probably be fixed with some minor (?)
changes to it.  However, it's reflective of a larger performance problem
which shows up in multiple places, so I'm concentrating on the general
solution, which is taking more time to do. :/


-- 
David Moore <dmoore@ucsd.edu>       | Computer Systems Lab      __o
UCSD Dept. Computer Science - 0114  | Work: (619) 534-8604    _ \<,_
La Jolla, CA 92093-0114             | Fax:  (619) 534-1445   (_)/ (_)
<URL:http://oj.egbt.org/dmoore/>    | Solo Furnace Creek 508 -- 1996!

Re: The A A command takes forever to complete

[Lars Magne Ingebrigtsen]

David Moore <dmoore@UCSD.EDU> writes:

> 	The same bug occurs with 'Y c' in the summary buffer.  So I
> recommend never hitting 'Y c' unless you only have less than about 10
> cached articles.  It takes about 8 minutes to insert 145 cached articles
> into the summary buffer on my machine. ;)  So, it's usually faster just
> to hit 'C-u Z R'.

Yup.  Commands like `Y c' (and `^') rebuild threads, which means that
they have to do all kinds of things to work properly.  I'm sure they
do too much -- that part of the code is a wilderness I've yet to
tame. 

Perhaps `Y c' should just do a complete regeneration of the summary
buffer once if there's more than a couple of articles to be
inserted...  Either that, or we'll have to rewrite the general summary
buffer thread regeneration code.  Which might be a good idea.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: The A A command takes forever to complete

[David Moore]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> David Moore <dmoore@UCSD.EDU> writes:
> 
> > 	The same bug occurs with 'Y c' in the summary buffer.  So I
> > recommend never hitting 'Y c' unless you only have less than about 10
> > cached articles.  It takes about 8 minutes to insert 145 cached articles
> > into the summary buffer on my machine. ;)  So, it's usually faster just
> > to hit 'C-u Z R'.
> 
> Yup.  Commands like `Y c' (and `^') rebuild threads, which means that
> they have to do all kinds of things to work properly.  I'm sure they
> do too much -- that part of the code is a wilderness I've yet to
> tame. 

	I seem to recall my profiling showing the thread stuff being
reasonable, and that all of the time was spent doing text-property-any's
and inserting the line.  Similar to the costs for
gnus-group-update-group which still hits nnvirtual harder than it
should.  And also hurts when you do an 'l' in the group buffer on a line
which will not be shown after that display.  

> Perhaps `Y c' should just do a complete regeneration of the summary
> buffer once if there's more than a couple of articles to be
> inserted...  Either that, or we'll have to rewrite the general summary
> buffer thread regeneration code.  Which might be a good idea.

	'Y c' could do the complete regeneration which would help with
the summary buffer, but I don't think rewriting the thread code will
help much (although it might speed it up a slight bit, I didn't save
traces of it, since it didn't seem to be the big cost).

	At the moment I'm looking into a data-structure to replace the
need for the text properties in the group buffer, which allows very
quick inserts and the appropriate movement searchs.  Hopefully the
approach will be general enough to work with the summary buffer as well.

-- 
David Moore <dmoore@ucsd.edu>       | Computer Systems Lab      __o
UCSD Dept. Computer Science - 0114  | Work: (619) 534-8604    _ \<,_
La Jolla, CA 92093-0114             | Fax:  (619) 534-1445   (_)/ (_)
<URL:http://oj.egbt.org/dmoore/>    | Solo Furnace Creek 508 -- 1996!

Re: The A A command takes forever to complete

[Lars Magne Ingebrigtsen]

David Moore <dmoore@UCSD.EDU> writes:

> 	I seem to recall my profiling showing the thread stuff being
> reasonable, and that all of the time was spent doing text-property-any's
> and inserting the line. 

Oh, I had forgotten about those...  They are really totally
unnecessary -- Gnus knows the position of all articles in the summary
buffer.  The `text-property-any' stuff was an emergency solution
because I got tired of fiddling with the thread stuff.

I'll take a new look at that code.

> 	At the moment I'm looking into a data-structure to replace the
> need for the text properties in the group buffer, which allows very
> quick inserts and the appropriate movement searchs.  Hopefully the
> approach will be general enough to work with the summary buffer as well.

The summary buffer keeps the positions of all articles in the
`gnus-newsgroup-data' alist.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: The A A command takes forever to complete

[Lars Magne Ingebrigtsen]

David Moore <dmoore@UCSD.EDU> writes:

> 	I seem to recall my profiling showing the thread stuff being
> reasonable, and that all of the time was spent doing text-property-any's
> and inserting the line.

I've now removed the `text-property-any' stuff, and things seem to
work fine.  There were some bugs to be fixed, but fixing them was
rather straightforward.  Odd that I gave up on this last time...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen