* extreme cpu usage with nndir (on specific files)
@ 2000-06-12 13:46 Harry Putnam
2000-06-12 20:49 ` Kai Großjohann
0 siblings, 1 reply; 3+ messages in thread
From: Harry Putnam @ 2000-06-12 13:46 UTC (permalink / raw)
I've run into a problem this morning that has me somewhat stymied.
Using gnus to create an nndir group of an archive I want to copy to a
second nnml server.
The directorys look like ~/00 ~/99 ~/98 and contain messages from
those years.
With file names like so:
awk00:
00853
00854
00855
00856
[...]
The biggest directory contains some 3000+ messages, the other
two have under 2000 each.
Making an nndir group of anyone of them causes gnus to go into a spasm
of cpu usage that seems clear out of line with the job at hand.
Watching top display I see cpu pegged near 99 % and memory around 42%.
This lasts for a very long time, like 5 minutes. Just to get to the
point where gnus asks how many messages I want to see.
Answering with <RET> and the cpu usage is pegged again for an even
longer time. Eventually the group appears. After sorting date and
marking with `M P a' (both seem to take unduly long as well). Then
copy to the desired group. The copying is also very very slow.
These files have the following format:
These two line are the first two headers in every post:
#! rnews 1596
Xref: news.demon.co.uk comp.lang.awk:8026
REST OF HEADERS
BLANK LINE
body
body
I wondered if the "#!" may be causing all the commotion?
I've also tried it with a vanilla emacs and gnus on a dummy account, but
still see the hefty cpu usage etc. Seems it must be something about these
particular files.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: extreme cpu usage with nndir (on specific files)
2000-06-12 13:46 extreme cpu usage with nndir (on specific files) Harry Putnam
@ 2000-06-12 20:49 ` Kai Großjohann
2000-06-12 23:35 ` Harry Putnam
0 siblings, 1 reply; 3+ messages in thread
From: Kai Großjohann @ 2000-06-12 20:49 UTC (permalink / raw)
Cc: ding
One way to find out what the heck Emacs is doing is to M-x
toggle-debug-on-quit RET, then hit C-g when it takes the long time.
Do this several times, and you might get an idea what takes so long.
Hm.
Or maybe M-x elp-instrument-package RET nndir RET? Anybody out there
who groks elp?
kai
--
I like BOTH kinds of music.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: extreme cpu usage with nndir (on specific files)
2000-06-12 20:49 ` Kai Großjohann
@ 2000-06-12 23:35 ` Harry Putnam
0 siblings, 0 replies; 3+ messages in thread
From: Harry Putnam @ 2000-06-12 23:35 UTC (permalink / raw)
Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> One way to find out what the heck Emacs is doing is to M-x
> toggle-debug-on-quit RET, then hit C-g when it takes the long time.
> Do this several times, and you might get an idea what takes so long.
>
> Hm.
>
> Or maybe M-x elp-instrument-package RET nndir RET? Anybody out there
> who groks elp?
I found out a little by a different method. First I stripped the
`^#! rnews ' line replaced it with `From gnus@rnews'. That didn't
help so it wasn't the header with the #! causing the trouble. Next I
stripped the Xref header. That helped a lot so that header was
causing trouble. Then I began to notice that gnus had created a
number of messages from my archive with `nobody' as sender and `no
subject'.
Checking those messages in the actual archive I found those particular
messages had the `From: ' line above the `Path: ' line.
Finally I made a small awk program that puts `X-' in front of:
X-#! rnews
X-Xref:
X-Path:
Adds my own Path: line with a blank line before it:
fix.awk:
---------
/^#! / {a=1}
/^$/ {a=0}
/^#! rnews/ && a=1 \
{print "\nPath: harrys.news.service!verygood!at.home.now\nX-"$0;next}
/^Xref: / && a=1 {print "X-"$0;next}
/^Path: / && a=1 {print "X-"$0;next}{print}
Gnus liked it... pulled the unsorted complete archive in as a nndoc
and sorted it by date and copied to the desired nnml+n2m groups.
All handling of these files now take normal amounts of time and I was
able to preserver the original headers (sort of).
I think the main problem was the Xref: header. Is that likely to give
gnus a problem?
This all took so long that I think I'll wait till next time to try the
approach you suggested.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2000-06-12 23:35 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2000-06-12 13:46 extreme cpu usage with nndir (on specific files) Harry Putnam
2000-06-12 20:49 ` Kai Großjohann
2000-06-12 23:35 ` Harry Putnam
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).