* Benchmarking Gnus
@ 2002-01-27 0:30 Lars Magne Ingebrigtsen
2002-01-27 0:35 ` IPmonger
2002-01-30 22:02 ` Wes Hardaker
0 siblings, 2 replies; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-27 0:30 UTC (permalink / raw)
When entering large groups, Gnus generates quite a lot of garbage,
which results in time being spent collecting that garbage. It would
be interesting to see where the garbage is being generated.
Wasn't there some version of, er, some package that would keep track
of how much memory each function uses? Perhaps ELP, even?
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-27 0:30 Benchmarking Gnus Lars Magne Ingebrigtsen
@ 2002-01-27 0:35 ` IPmonger
2002-01-27 0:45 ` Lars Magne Ingebrigtsen
2002-01-30 22:02 ` Wes Hardaker
1 sibling, 1 reply; 54+ messages in thread
From: IPmonger @ 2002-01-27 0:35 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> When entering large groups, Gnus generates quite a lot of garbage,
> which results in time being spent collecting that garbage. It would
> be interesting to see where the garbage is being generated.
>
> Wasn't there some version of, er, some package that would keep
> track of how much memory each function uses? Perhaps ELP, even?
How do you engage it?
My largest group is the mailbox I use for the linux kernel mailing
list. Of course, it's now very sparse since I don't keep everything,
but the article #s exceed 42000... Is that big enough?
-IPmonger
--
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-27 0:35 ` IPmonger
@ 2002-01-27 0:45 ` Lars Magne Ingebrigtsen
2002-01-29 16:03 ` IPmonger
0 siblings, 1 reply; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-27 0:45 UTC (permalink / raw)
IPmonger <ipmonger@delamancha.org> writes:
>> Wasn't there some version of, er, some package that would keep
>> track of how much memory each function uses? Perhaps ELP, even?
I found it -- it was a version of ELP I had hacked to keep track of
all that stuff. I even had to hack the Emacs internals to get at the
info, and those internals were still there. :-)
> My largest group is the mailbox I use for the linux kernel mailing
> list. Of course, it's now very sparse since I don't keep everything,
> but the article #s exceed 42000... Is that big enough?
That's plenty big.
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-27 0:45 ` Lars Magne Ingebrigtsen
@ 2002-01-29 16:03 ` IPmonger
2002-01-30 8:53 ` Kai Großjohann
2002-02-04 23:35 ` IPmonger
0 siblings, 2 replies; 54+ messages in thread
From: IPmonger @ 2002-01-29 16:03 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> IPmonger <ipmonger@delamancha.org> writes:
>
>>> Wasn't there some version of, er, some package that would keep
>>> track of how much memory each function uses? Perhaps ELP, even?
>
> I found it -- it was a version of ELP I had hacked to keep track of
> all that stuff. I even had to hack the Emacs internals to get at the
> info, and those internals were still there. :-)
>
>> My largest group is the mailbox I use for the linux kernel mailing
>> list. Of course, it's now very sparse since I don't keep everything,
>> but the article #s exceed 42000... Is that big enough?
>
> That's plenty big.
I'm ready to profile, but when I interactively try
M-x elp-instrument-package RET gnus RET I get errors indicating that
it can't instrument autoloaded functions.
-IPmonger
--
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-29 16:03 ` IPmonger
@ 2002-01-30 8:53 ` Kai Großjohann
2002-01-30 16:42 ` IPmonger
2002-02-04 23:35 ` IPmonger
1 sibling, 1 reply; 54+ messages in thread
From: Kai Großjohann @ 2002-01-30 8:53 UTC (permalink / raw)
Cc: ding
IPmonger <ipmonger@delamancha.org> writes:
> I'm ready to profile, but when I interactively try
> M-x elp-instrument-package RET gnus RET I get errors indicating that
> it can't instrument autoloaded functions.
Load Gnus first. For example, start Gnus, then enter a group, then
exit the group, then M-x elp-instrument-package RET, then enter the
group a second time for profiling entering groups.
kai
--
~/.signature is: umop 3p!sdn (Frank Nobis)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 8:53 ` Kai Großjohann
@ 2002-01-30 16:42 ` IPmonger
2002-01-30 17:38 ` Kai Großjohann
0 siblings, 1 reply; 54+ messages in thread
From: IPmonger @ 2002-01-30 16:42 UTC (permalink / raw)
Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> IPmonger <ipmonger@delamancha.org> writes:
>
>> I'm ready to profile, but when I interactively try M-x
>> elp-instrument-package RET gnus RET I get errors indicating that it
>> can't instrument autoloaded functions.
>
> Load Gnus first. For example, start Gnus, then enter a group, then
> exit the group, then M-x elp-instrument-package RET, then enter the
> group a second time for profiling entering groups.
Same result, although the complaint is about a different function.
:-)
Here's the results of M-x elp-results RET:
Function Name Call Count Elapsed Time Average Time
==================================== ========== ============ ============
gnus-short-group-name 1 0.0003010000 0.0003010000
Does that seem close to being correct? Admittedly, it didn't take
very long to enter the group, but I suppose that I expected more than
1 function call to be profiled.
Someone else suggested that I might need to upgrade my XEmacs from
21.1 to a version closer to what they run (21.5), so that's what I'll
try next.
-IPmonger
--
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 16:42 ` IPmonger
@ 2002-01-30 17:38 ` Kai Großjohann
2002-01-30 18:15 ` Lars Magne Ingebrigtsen
0 siblings, 1 reply; 54+ messages in thread
From: Kai Großjohann @ 2002-01-30 17:38 UTC (permalink / raw)
Cc: Gnus Mailing List
IPmonger <ipmonger@delamancha.org> writes:
> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>
>> IPmonger <ipmonger@delamancha.org> writes:
>>
>>> I'm ready to profile, but when I interactively try M-x
>>> elp-instrument-package RET gnus RET I get errors indicating that it
>>> can't instrument autoloaded functions.
>>
>> Load Gnus first. For example, start Gnus, then enter a group, then
>> exit the group, then M-x elp-instrument-package RET, then enter the
>> group a second time for profiling entering groups.
>
> Same result, although the complaint is about a different function.
> :-)
Gah?
Well, go through the list of *.el files that are part of Gnus and do
M-x load-library RET on all of them...
(Or use dired and the `L' key.)
kai
--
~/.signature is: umop 3p!sdn (Frank Nobis)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 17:38 ` Kai Großjohann
@ 2002-01-30 18:15 ` Lars Magne Ingebrigtsen
2002-01-30 18:42 ` Josh Huber
0 siblings, 1 reply; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-30 18:15 UTC (permalink / raw)
Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> Well, go through the list of *.el files that are part of Gnus and do
> M-x load-library RET on all of them...
I think more recent XEmacsen has this elp bug fixed...
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 18:15 ` Lars Magne Ingebrigtsen
@ 2002-01-30 18:42 ` Josh Huber
0 siblings, 0 replies; 54+ messages in thread
From: Josh Huber @ 2002-01-30 18:42 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> I think more recent XEmacsen has this elp bug fixed...
Yes, it's at least fixed in 21.5...
--
Josh Huber
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-27 0:30 Benchmarking Gnus Lars Magne Ingebrigtsen
2002-01-27 0:35 ` IPmonger
@ 2002-01-30 22:02 ` Wes Hardaker
2002-01-30 22:09 ` IPmonger
1 sibling, 1 reply; 54+ messages in thread
From: Wes Hardaker @ 2002-01-30 22:02 UTC (permalink / raw)
>>>>> On Sun, 27 Jan 2002 01:30:29 +0100, Lars Magne Ingebrigtsen <larsi@gnus.org> said:
Lars> When entering large groups, Gnus generates quite a lot of
Lars> garbage, which results in time being spent collecting that
Lars> garbage. It would be interesting to see where the garbage is
Lars> being generated.
Actually, I've recently discovered that I have a different profiling
problem I need to solve:
hardaker 3772 6.1 30.9 87020 78868 pts/2 S 13:15 2:43 xemacs
^^^^^ ^^^^^
I really don't like a 80 meg resident application running on my
machine. I think this was a recent change of some kind and I'm not
sure what's done it. I'm pretty sure it was a recent update of Gnus.
I keep meaning to run xemacs for a while without running gnus to see
if Gnus really is the responsible party for the memory leak, but,
well, I just can't seem to run xemacs that long without starting up
Gnus...
Anyone else see problems like this? cvs update from ~ Jan 25th.
--
"Ninjas aren't dangerous. They're more afraid of you than you are of them."
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 22:02 ` Wes Hardaker
@ 2002-01-30 22:09 ` IPmonger
2002-01-31 1:25 ` Wes Hardaker
0 siblings, 1 reply; 54+ messages in thread
From: IPmonger @ 2002-01-30 22:09 UTC (permalink / raw)
Wes Hardaker <wes@hardakers.net> writes:
> Actually, I've recently discovered that I have a different
> profiling problem I need to solve:
>
> hardaker 3772 6.1 30.9 87020 78868 pts/2 S 13:15 2:43 xemacs
> ^^^^^ ^^^^^
>
> Anyone else see problems like this? cvs update from ~ Jan 25th.
Um, I'm not seeing that problem. Admittedly, I'm running gnus from
~ Jan 28th w/XEmacs 21.1 on RedHat 7.2 i686 Linux:
2651 ipmonger 14 0 20180 19M 12248 S 11.2 10.7 49:49 xemacs
If gnus is responsible, then it's either not nnml (which is the only
back-end I use) or it's been corrected...
-IPmonger
--
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-30 22:09 ` IPmonger
@ 2002-01-31 1:25 ` Wes Hardaker
2002-01-31 3:22 ` ShengHuo ZHU
2002-01-31 16:34 ` Steinar Bang
0 siblings, 2 replies; 54+ messages in thread
From: Wes Hardaker @ 2002-01-31 1:25 UTC (permalink / raw)
Cc: Gnus Mailing List
>>>>> On Wed, 30 Jan 2002 17:09:56 -0500, IPmonger <ipmonger@delamancha.org> said:
ipmonger> Wes Hardaker <wes@hardakers.net> writes:
>> Actually, I've recently discovered that I have a different
>> profiling problem I need to solve:
>>
>> hardaker 3772 6.1 30.9 87020 78868 pts/2 S 13:15 2:43 xemacs
>> ^^^^^ ^^^^^
>>
>> Anyone else see problems like this? cvs update from ~ Jan 25th.
ipmonger> Um, I'm not seeing that problem. Admittedly, I'm running gnus from
ipmonger> ~ Jan 28th w/XEmacs 21.1 on RedHat 7.2 i686 Linux:
ipmonger> 2651 ipmonger 14 0 20180 19M 12248 S 11.2 10.7 49:49 xemacs
ipmonger> If gnus is responsible, then it's either not nnml (which is the only
ipmonger> back-end I use) or it's been corrected...
Gnus is definitely responsible. I finally did run it for a while
without gnus and didn't get any major memory leaks. Immediately after
starting gnus and entering a few groups, I'm back at huge memory
usages. sigh...
However, these are imap folders.
--
"Ninjas aren't dangerous. They're more afraid of you than you are of them."
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 1:25 ` Wes Hardaker
@ 2002-01-31 3:22 ` ShengHuo ZHU
2002-01-31 15:29 ` Wes Hardaker
2002-01-31 16:34 ` Steinar Bang
1 sibling, 1 reply; 54+ messages in thread
From: ShengHuo ZHU @ 2002-01-31 3:22 UTC (permalink / raw)
Wes Hardaker <wes@hardakers.net> writes:
>>>>>> On Wed, 30 Jan 2002 17:09:56 -0500, IPmonger <ipmonger@delamancha.org> said:
>
> ipmonger> Wes Hardaker <wes@hardakers.net> writes:
>>> Actually, I've recently discovered that I have a different
>>> profiling problem I need to solve:
>>>
>>> hardaker 3772 6.1 30.9 87020 78868 pts/2 S 13:15 2:43 xemacs
>>> ^^^^^ ^^^^^
>>>
>>> Anyone else see problems like this? cvs update from ~ Jan 25th.
>
> ipmonger> Um, I'm not seeing that problem. Admittedly, I'm running gnus from
> ipmonger> ~ Jan 28th w/XEmacs 21.1 on RedHat 7.2 i686 Linux:
>
> ipmonger> 2651 ipmonger 14 0 20180 19M 12248 S 11.2 10.7 49:49 xemacs
>
> ipmonger> If gnus is responsible, then it's either not nnml (which
> ipmonger> is the only back-end I use) or it's been corrected...
>
> Gnus is definitely responsible. I finally did run it for a while
> without gnus and didn't get any major memory leaks. Immediately after
> starting gnus and entering a few groups, I'm back at huge memory
> usages. sigh...
Mine is 45.5 meg right now. It was reported that the image cache used
a lot of memory. In Emacs, setting a small image-cache-eviction-delay
or running clear-image-cache may help a little. But I am not sure
about XEmacs.
Anyway, after adding 256 meg memory, I don't care much about that now.
ShengHuo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 3:22 ` ShengHuo ZHU
@ 2002-01-31 15:29 ` Wes Hardaker
0 siblings, 0 replies; 54+ messages in thread
From: Wes Hardaker @ 2002-01-31 15:29 UTC (permalink / raw)
>>>>> On Wed, 30 Jan 2002 22:22:14 -0500, ShengHuo ZHU <zsh@cs.rochester.edu> said:
ZSH> Mine is 45.5 meg right now. It was reported that the image cache used
ZSH> a lot of memory. In Emacs, setting a small image-cache-eviction-delay
ZSH> or running clear-image-cache may help a little. But I am not sure
ZSH> about XEmacs.
ZSH> Anyway, after adding 256 meg memory, I don't care much about that now.
I have 256 and I start caring when I start trying to run a lot of
things (like vmware NT with 128 for it), etc...
Is there any way to figure out where memory is going in emacs/xemacs?
The obvious things (a buffer) shows nothing. The largest buffer is a
nnimap history buffer, but it's only 4 megs.
--
"Ninjas aren't dangerous. They're more afraid of you than you are of them."
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 1:25 ` Wes Hardaker
2002-01-31 3:22 ` ShengHuo ZHU
@ 2002-01-31 16:34 ` Steinar Bang
2002-01-31 16:54 ` Bjørn Mork
2002-02-01 16:51 ` Wes Hardaker
1 sibling, 2 replies; 54+ messages in thread
From: Steinar Bang @ 2002-01-31 16:34 UTC (permalink / raw)
>>>>> Wes Hardaker <wes@hardakers.net>:
[snip! 80MB xemacs with Gnus]
> Gnus is definitely responsible. I finally did run it for a while
> without gnus and didn't get any major memory leaks. Immediately
> after starting gnus and entering a few groups, I'm back at huge
> memory usages. sigh...
> However, these are imap folders.
Do you have some old articles marked? Are you using agent? If so, I
have seen the "exploding process size on folder visit" problem as well
(search for the thread with the subject "any tips for how to tune
Gnus/emacs for low memory usage?") on a laptop with very little
physical memory by today's standards.
The problems were more severe with XEmacs 21.4, than they were with
GNU Emacs 20.7, though the problems were recognizably present there as
well.
Didn't larsi do some tuning on opening Summary buffers recently, that
should avoid creating so much stuff to be garbage collected? Or was
that just nntp?
Has anyone looked into the bug with agent fetching all articles
between the oldest ticked articles and the current (ie. re-fetching
articles that were previously expired from the agent)? It was
reported by Bjørn Mork a couple of weeks back.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 16:34 ` Steinar Bang
@ 2002-01-31 16:54 ` Bjørn Mork
2002-01-31 20:50 ` Steinar Bang
2002-02-01 16:51 ` Wes Hardaker
1 sibling, 1 reply; 54+ messages in thread
From: Bjørn Mork @ 2002-01-31 16:54 UTC (permalink / raw)
Steinar Bang <sb@dod.no> writes:
> Has anyone looked into the bug with agent fetching all articles
> between the oldest ticked articles and the current (ie. re-fetching
> articles that were previously expired from the agent)? It was
> reported by Bjørn Mork a couple of weeks back.
I believe this was fixed by larsi a week ago or so. Agent works more
or less as expected for me at the moment.
Bjørn
--
Save the sexist jokes!
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 16:54 ` Bjørn Mork
@ 2002-01-31 20:50 ` Steinar Bang
2002-01-31 22:35 ` Bjørn Mork
0 siblings, 1 reply; 54+ messages in thread
From: Steinar Bang @ 2002-01-31 20:50 UTC (permalink / raw)
>>>>> Bjørn Mork <bmork@dod.no>:
> Steinar Bang <sb@dod.no> writes:
>> Has anyone looked into the bug with agent fetching all articles
>> between the oldest ticked articles and the current (ie. re-fetching
>> articles that were previously expired from the agent)? It was
>> reported by Bjørn Mork a couple of weeks back.
> I believe this was fixed by larsi a week ago or so. Agent works more
> or less as expected for me at the moment.
Maybe it's time to upgrade then? I saw some nnimap expiry problems
reported earlier. Have you seen any indications of this?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 20:50 ` Steinar Bang
@ 2002-01-31 22:35 ` Bjørn Mork
2002-02-01 7:14 ` Steinar Bang
0 siblings, 1 reply; 54+ messages in thread
From: Bjørn Mork @ 2002-01-31 22:35 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 569 bytes --]
Steinar Bang <sb@dod.no> writes:
>>>>>> Bjørn Mork <bmork@dod.no>:
>
>> I believe this was fixed by larsi a week ago or so. Agent works more
>> or less as expected for me at the moment.
>
> Maybe it's time to upgrade then? I saw some nnimap expiry problems
> reported earlier. Have you seen any indications of this?
Haven't noticed anything. What sort of problems? I am still running
with the attached patch (provided by Simon, I think) to avoid problems
with the combination nnimap, agent and expiry-target.
Bjørn
--
Man, I can't believe how sadistic you are.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: agent-expiry-fix.diff --]
[-- Type: text/x-patch, Size: 880 bytes --]
Index: lisp/gnus-int.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/gnus-int.el,v
retrieving revision 6.9
diff -u -r6.9 gnus-int.el
--- lisp/gnus-int.el 2002/01/11 02:36:58 6.9
+++ lisp/gnus-int.el 2002/01/31 22:38:29
@@ -514,9 +514,9 @@
(let ((mail-parse-charset message-default-charset))
(mail-encode-encoded-word-buffer)))
(message-encode-message-body)))
- (let ((func (car (or gnus-command-method
- (gnus-find-method-for-group group)))))
- (funcall (intern (format "%s-request-accept-article" func))
+ (let ((gnus-command-method (or gnus-command-method
+ (gnus-find-method-for-group group))))
+ (funcall (gnus-get-function gnus-command-method 'request-accept-article)
(if (stringp group) (gnus-group-real-name group) group)
(cadr gnus-command-method)
last)))
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 22:35 ` Bjørn Mork
@ 2002-02-01 7:14 ` Steinar Bang
2002-02-01 9:17 ` Bjørn Mork
0 siblings, 1 reply; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 7:14 UTC (permalink / raw)
>>>>> Bjørn Mork <bmork@dod.no>:
> Haven't noticed anything. What sort of problems? I am still running
> with the attached patch (provided by Simon, I think) to avoid
> problems with the combination nnimap, agent and expiry-target.
Is it stuff in the patch, and not the CVS trunk, that avoids
re-fetching articles that has already been expired in agent?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 7:14 ` Steinar Bang
@ 2002-02-01 9:17 ` Bjørn Mork
2002-02-01 10:00 ` Steinar Bang
0 siblings, 1 reply; 54+ messages in thread
From: Bjørn Mork @ 2002-02-01 9:17 UTC (permalink / raw)
Steinar Bang <sb@dod.no> writes:
>>>>>> Bjørn Mork <bmork@dod.no>:
>
>> Haven't noticed anything. What sort of problems? I am still running
>> with the attached patch (provided by Simon, I think) to avoid
>> problems with the combination nnimap, agent and expiry-target.
>
> Is it stuff in the patch, and not the CVS trunk, that avoids
> re-fetching articles that has already been expired in agent?
Nope.
Bjørn
--
You're always totally wrong.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 9:17 ` Bjørn Mork
@ 2002-02-01 10:00 ` Steinar Bang
2002-02-01 11:24 ` Bjørn Mork
2002-02-01 11:29 ` Simon Josefsson
0 siblings, 2 replies; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 10:00 UTC (permalink / raw)
>>>>> Bjørn Mork <bmork@dod.no>:
> Steinar Bang <sb@dod.no> writes:
>>>>>>> Bjørn Mork <bmork@dod.no>:
>>> Haven't noticed anything. What sort of problems? I am still
>>> running with the attached patch (provided by Simon, I think) to
>>> avoid problems with the combination nnimap, agent and
>>> expiry-target.
>> Is it stuff in the patch, and not the CVS trunk, that avoids
>> re-fetching articles that has already been expired in agent?
> Nope.
What are the problems with the above combination?
One thing that has never worked for me, is that expiration of
agentized nnimap groups, run in unplugged mode, are made to be imap
delete operations in the folders on the server, when I go plugged
again.
I'm not even sure if this is _supposed_ to be working...? :-)
Does the patch have anything to to with this problem?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 10:00 ` Steinar Bang
@ 2002-02-01 11:24 ` Bjørn Mork
2002-02-01 11:29 ` Simon Josefsson
1 sibling, 0 replies; 54+ messages in thread
From: Bjørn Mork @ 2002-02-01 11:24 UTC (permalink / raw)
Steinar Bang <sb@dod.no> writes:
> What are the problems with the above combination?
I'm using total-expire and another imap mailbox as expiry-target in
one of my imap inboxes. This caused an error when I tried to exit from
the inbox while unplugged. The patch eliminates the error message.
Don't ask me why and how - I don't know :-)
Expiry won't work with the patch either (since articles cannot be
moved while unplugged), but that doesn't matter to me since expiry
process will run again as soon as I'm plugged anyway. The only problem
with this is that it kind of assumes that offline deletion isn't
implemented...
> One thing that has never worked for me, is that expiration of
> agentized nnimap groups, run in unplugged mode, are made to be imap
> delete operations in the folders on the server, when I go plugged
> again.
Really? I didn't think deleting worked at all when unplugged. Let me
try. No, it doesn't. That's good, or I would have lost a lot of mail
without noticing.
> I'm not even sure if this is _supposed_ to be working...? :-)
No, I don't think it is.
Bjørn
--
You sound like a real coward.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 10:00 ` Steinar Bang
2002-02-01 11:24 ` Bjørn Mork
@ 2002-02-01 11:29 ` Simon Josefsson
2002-02-01 12:52 ` Steinar Bang
1 sibling, 1 reply; 54+ messages in thread
From: Simon Josefsson @ 2002-02-01 11:29 UTC (permalink / raw)
Cc: ding
On Fri, 1 Feb 2002, Steinar Bang wrote:
> One thing that has never worked for me, is that expiration of
> agentized nnimap groups, run in unplugged mode, are made to be imap
> delete operations in the folders on the server, when I go plugged
> again.
>
> I'm not even sure if this is _supposed_ to be working...? :-)
It sounds like a bug, expiring the agent nnml group should not delete
anything in the original nnimap group.
> Does the patch have anything to to with this problem?
No, I don't think so. I'll check in the patch as well, I think it is
correct.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 11:29 ` Simon Josefsson
@ 2002-02-01 12:52 ` Steinar Bang
2002-02-01 13:40 ` Simon Josefsson
0 siblings, 1 reply; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 12:52 UTC (permalink / raw)
>>>>> Simon Josefsson <jas@extundo.com>:
> It sounds like a bug, expiring the agent nnml group should not
> delete anything in the original nnimap group.
No it's not a bug. :-)
What happens is what you describe.
I expected unplugged expiry to be transferred to the original nnimap
group when I went plugged. Ie. I expected what you term buggy
behaviour. :-)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 12:52 ` Steinar Bang
@ 2002-02-01 13:40 ` Simon Josefsson
2002-02-01 14:12 ` Steinar Bang
0 siblings, 1 reply; 54+ messages in thread
From: Simon Josefsson @ 2002-02-01 13:40 UTC (permalink / raw)
Cc: ding
On Fri, 1 Feb 2002, Steinar Bang wrote:
> >>>>> Simon Josefsson <jas@extundo.com>:
>
> > It sounds like a bug, expiring the agent nnml group should not
> > delete anything in the original nnimap group.
>
> No it's not a bug. :-)
> What happens is what you describe.
>
> I expected unplugged expiry to be transferred to the original nnimap
> group when I went plugged. Ie. I expected what you term buggy
> behaviour. :-)
Hm. There is two kinds of expiry in a agent group, one is the normal
expiring in the real backend via E marks (nnimap), which should work (does
it?), but you should also be able to locally expire the agent cache with
gnus-agent-expire-articles (does it work?).
The first should delete articles on the server, the second should not.
Hm. I don't use the agent much, but does this make sense?
(Btw, wouldn't it be nice if it was possible to enable the agent via
customize instead of forcing people to manually edit ~/.gnus?)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 13:40 ` Simon Josefsson
@ 2002-02-01 14:12 ` Steinar Bang
2002-02-01 14:39 ` ShengHuo ZHU
2002-02-01 14:59 ` Simon Josefsson
0 siblings, 2 replies; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 14:12 UTC (permalink / raw)
>>>>> Simon Josefsson <jas@extundo.com>:
> Hm. There is two kinds of expiry in a agent group, one is the normal
> expiring in the real backend via E marks (nnimap), which should work
> (does it?),
If you mean: "does it work if you run expiry when plugged?", the
answer is yes.
If you mean: "if you run expiry when unplugged, are the results
transferred to the nnimap group, when you go plugged?", the answer was
no, the last time I tried.
> but you should also be able to locally expire the agent cache with
> gnus-agent-expire-articles (does it work?).
I haven't tried recently, and I'm not in physical reach of my laptop
right now, but it worked the last time I tried. Ie. it _did_ expire
articles older than 7 days, but if I had some old ticked articles,
the articles in between were re-fetched the next time I fetched
articles into the agent.
According to Bjørn Mork earlier in this thread, this should be fixed
now...?
> The first should delete articles on the server, the second should
> not.
> Hm. I don't use the agent much,
This means that you don't use a laptop with suspend/resume? :-)
> but does this make sense?
Sure.
> (Btw, wouldn't it be nice if it was possible to enable the agent via
> customize instead of forcing people to manually edit ~/.gnus?)
You can toggle server agent status interactively in the server buffer,
if that's what you're thinking about...?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 14:12 ` Steinar Bang
@ 2002-02-01 14:39 ` ShengHuo ZHU
2002-02-01 15:01 ` Simon Josefsson
2002-02-01 14:59 ` Simon Josefsson
1 sibling, 1 reply; 54+ messages in thread
From: ShengHuo ZHU @ 2002-02-01 14:39 UTC (permalink / raw)
Steinar Bang <sb@dod.no> writes:
>>>>>> Simon Josefsson <jas@extundo.com>:
>
>> Hm. There is two kinds of expiry in a agent group, one is the normal
>> expiring in the real backend via E marks (nnimap), which should work
>> (does it?),
>
> If you mean: "does it work if you run expiry when plugged?", the
> answer is yes.
>
> If you mean: "if you run expiry when unplugged, are the results
> transferred to the nnimap group, when you go plugged?", the answer was
> no, the last time I tried.
Gnus should skip expiring offline servers, shouldn't it?
[...]
>> (Btw, wouldn't it be nice if it was possible to enable the agent via
>> customize instead of forcing people to manually edit ~/.gnus?)
>
> You can toggle server agent status interactively in the server buffer,
> if that's what you're thinking about...?
One possible solution is customizing the gnus-agent variable and
running gnus-agentize if gnus-agent is t after loading ~/.gnus.
ShengHuo
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 14:12 ` Steinar Bang
2002-02-01 14:39 ` ShengHuo ZHU
@ 2002-02-01 14:59 ` Simon Josefsson
2002-02-01 15:18 ` Steinar Bang
1 sibling, 1 reply; 54+ messages in thread
From: Simon Josefsson @ 2002-02-01 14:59 UTC (permalink / raw)
Cc: ding
On Fri, 1 Feb 2002, Steinar Bang wrote:
> > Hm. There is two kinds of expiry in a agent group, one is the normal
> > expiring in the real backend via E marks (nnimap), which should work
> > (does it?),
>
> If you mean: "does it work if you run expiry when plugged?", the
> answer is yes.
Good.
> If you mean: "if you run expiry when unplugged, are the results
> transferred to the nnimap group, when you go plugged?", the answer was
> no, the last time I tried.
Bad. I wonder why? Hm. Ah, nnagent-request-expire-articles needs to
save the invokation into a file that is evaluated when you plug in. Much
like the "flags" file. Maybe it can even use that file. This looks like
a five-liner.
> > but you should also be able to locally expire the agent cache with
> > gnus-agent-expire-articles (does it work?).
>
> I haven't tried recently, and I'm not in physical reach of my laptop
> right now, but it worked the last time I tried. Ie. it _did_ expire
> articles older than 7 days, but if I had some old ticked articles,
> the articles in between were re-fetched the next time I fetched
> articles into the agent.
>
> According to Bjørn Mork earlier in this thread, this should be fixed
> now...?
Fixed in what sense? I think the agent MUST have ticked (and generally
marked articles) locally stored, otherwise the marks will be removed by
code in gnus-sum.el that removes flags on articles that has disappeared.
Hm. Maybe that code could be changed, instead of requiring the agent to
download all marked articles though.
> > The first should delete articles on the server, the second should
> > not.
>
> > Hm. I don't use the agent much,
>
> This means that you don't use a laptop with suspend/resume? :-)
My laptop always has a network connection. :-)
And I never expire anything anyway. Maybe that can be useful for the
benchmark/design discussion, I have some IMAP groups with >500000 messages
or so. (Which are completely unusable in Gnus).
> > (Btw, wouldn't it be nice if it was possible to enable the agent via
> > customize instead of forcing people to manually edit ~/.gnus?)
>
> You can toggle server agent status interactively in the server buffer,
> if that's what you're thinking about...?
No, I was thinking about that you need to call (gnus-agentize) in ~/.gnus
to enable that code (and other code as well). I'd like a
`gnus-use-agent' variable, much like the `gnus-use-cache' variable.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 14:39 ` ShengHuo ZHU
@ 2002-02-01 15:01 ` Simon Josefsson
2002-02-01 15:31 ` Steinar Bang
0 siblings, 1 reply; 54+ messages in thread
From: Simon Josefsson @ 2002-02-01 15:01 UTC (permalink / raw)
Cc: ding
On Fri, 1 Feb 2002, ShengHuo ZHU wrote:
> > If you mean: "if you run expiry when unplugged, are the results
> > transferred to the nnimap group, when you go plugged?", the answer was
> > no, the last time I tried.
>
> Gnus should skip expiring offline servers, shouldn't it?
IMHO it should remember the expiry command and run it when you get back
online. Hm. But the conditions that decides which articles should be
expired could have changed until then. Nah, I think you are right, if you
want to expire a group, you need to be online. If you are offline, maybe
it should expire the local agent directory instead. Opinions?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 14:59 ` Simon Josefsson
@ 2002-02-01 15:18 ` Steinar Bang
0 siblings, 0 replies; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 15:18 UTC (permalink / raw)
>>>>> Simon Josefsson <jas@extundo.com>:
[snip!]
>> If you mean: "if you run expiry when unplugged, are the results
>> transferred to the nnimap group, when you go plugged?", the answer
>> was no, the last time I tried.
> Bad. I wonder why? Hm. Ah, nnagent-request-expire-articles needs
> to save the invokation into a file that is evaluated when you plug
> in. Much like the "flags" file. Maybe it can even use that file.
> This looks like a five-liner.
Kewl!
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-01 15:01 ` Simon Josefsson
@ 2002-02-01 15:31 ` Steinar Bang
0 siblings, 0 replies; 54+ messages in thread
From: Steinar Bang @ 2002-02-01 15:31 UTC (permalink / raw)
>>>>> Simon Josefsson <jas@extundo.com>:
> On Fri, 1 Feb 2002, ShengHuo ZHU wrote:
>>> If you mean: "if you run expiry when unplugged, are the results
>>> transferred to the nnimap group, when you go plugged?", the answer
>>> was no, the last time I tried.
>> Gnus should skip expiring offline servers, shouldn't it?
> IMHO it should remember the expiry command and run it when you get
> back online.
That's what I would think.
> Hm. But the conditions that decides which articles should be
> expired could have changed until then.
Some articles may have been ticked from a different client, you mean?
If they have, Gnus should detect that, not delete the article, import
the flags for the changed articles into agent, fetching or re-fetching
copies of the articles if neccessary.
This may be more than a 5-liner...:-) but see below.
> Nah, I think you are right, if you want to expire a group, you need
> to be online. If you are offline, maybe it should expire the local
> agent directory instead. Opinions?
IMO an agentized group should reflect the underlying group as closely
as possible. This means that all operations, including move and
expiry, should be available, and operate the same way as they do when
plugged.
The basic worldview is that the underlying group is the master, and
that the agent group will reflect the master as closely as possible.
This goes for nnimap groups. Ie. if an article has been deleted on
the server, I would like the article to be deleted in the agent as
well, even if I had it ticked there.
However, for nntp, I have a different requirement. On nntp, I would
like to keep a ticked article, even if the article have been expired
on the server.
Though I can live with an identical behaviour to what I see with
nnimap, if it is still possible to save local copies of articles, in
the way we did before agent. Is it?
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-31 16:34 ` Steinar Bang
2002-01-31 16:54 ` Bjørn Mork
@ 2002-02-01 16:51 ` Wes Hardaker
1 sibling, 0 replies; 54+ messages in thread
From: Wes Hardaker @ 2002-02-01 16:51 UTC (permalink / raw)
Cc: ding
>>>>> On Thu, 31 Jan 2002 17:34:29 +0100, Steinar Bang <sb@dod.no> said:
>>>>> Wes Hardaker <wes@hardakers.net>:
Steinar> [snip! 80MB xemacs with Gnus]
>> Gnus is definitely responsible. I finally did run it for a while
>> without gnus and didn't get any major memory leaks. Immediately
>> after starting gnus and entering a few groups, I'm back at huge
>> memory usages. sigh...
>> However, these are imap folders.
Steinar> Do you have some old articles marked? Are you using agent?
Yep. Though I'm typically online most of the time.
I have done a recent update and the problem still persists so the
recent agent changes didn't help.
(I should try turning off agent support entirely and see if the
problem still exists... Maybe I can narrow it down further to agent
vs not-agent)
Steinar> The problems were more severe with XEmacs 21.4, than they
Steinar> were with GNU Emacs 20.7, though the problems were
Steinar> recognizably present there as well.
I'm at 21.4.5, so... (it's probably time to update it again).
--
"Ninjas aren't dangerous. They're more afraid of you than you are of them."
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-29 16:03 ` IPmonger
2002-01-30 8:53 ` Kai Großjohann
@ 2002-02-04 23:35 ` IPmonger
2002-02-04 23:48 ` IPmonger
2002-02-04 23:57 ` Daniel Pittman
1 sibling, 2 replies; 54+ messages in thread
From: IPmonger @ 2002-02-04 23:35 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 573 bytes --]
Ok, I updated to a newer XEmacs (21.5.4) and ran ELP on gnus with my
44000+ nnml folder for the Linux kernel, but I need some help
understanding what these results indicate.
To generate them, I:
started XEmacs
instrumented gnus (elp-instrument-package)
loaded gnus (M-x gnus)
hid empty topics
listed all topics
entered linux-kernel nnml folder
exited back to *Group* buffer
Are profiling results cumulative for an Emacs/XEmacs session?
-IPmonger
--
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
[-- Attachment #2: ELP Profiling Results: Gnus + nnml --]
[-- Type: text/plain, Size: 2616 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-run-hooks 1147 1.542707 0.0013449930
gnus-remassoc 18861 1.2365829999 6.556...e-05
gnus-update-alist-soft 7757 1.1905260000 0.0001534776
gnus-put-text-property-excluding-characters-with-faces 1119 0.6554359999 0.0005857336
gnus-byte-compile 1 0.380348 0.380348
gnus-define-keys-1 47 0.285708 0.0060788936
gnus-extract-address-components 1632 0.2381220000 0.0001459080
gnus-split-references 1346 0.112672 8.370...e-05
gnus-message 25 0.0536890000 0.00214756
gnus-point-at-eol 4381 0.0289090000 6.598...e-06
gnus-prin1 2 0.017061 0.0085305000
gnus-set-work-buffer 48 0.0059399999 0.00012375
gnus-point-at-bol 661 0.0057210000 8.655...e-06
gnus-write-buffer 1 0.005479 0.005479
gnus-make-hashtable 11 0.003488 0.0003170909
gnus-prin1-to-string 1 0.003236 0.003236
gnus-goto-colon 9 0.00223 0.0002477777
gnus-newsgroup-savable-name 4 0.000636 0.000159
gnus-make-directory 4 0.000534 0.0001335
gnus-turn-off-edit-menu 2 0.000359 0.0001795
gnus-buffer-live-p 13 0.0002739999 2.107...e-05
gnus-create-hash-size 9 0.000148 1.644...e-05
gnus-alive-p 1 0.000122 0.000122
gnus-make-sort-function 1 5e-05 5e-05
gnus-delete-if 3 4.9e-05 1.633...e-05
gnus-delete-alist 1 4e-05 4e-05
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-04 23:35 ` IPmonger
@ 2002-02-04 23:48 ` IPmonger
2002-02-04 23:57 ` Daniel Pittman
1 sibling, 0 replies; 54+ messages in thread
From: IPmonger @ 2002-02-04 23:48 UTC (permalink / raw)
IPmonger <ipmonger@delamancha.org> writes:
> Are profiling results cumulative for an Emacs/XEmacs session?
Please ignore this stupid question. I see elp-reset-all. :-)
-IPmonger
------------------
IPmonger
ipmonger@delamancha.org
CCIE #8338
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-02-04 23:35 ` IPmonger
2002-02-04 23:48 ` IPmonger
@ 2002-02-04 23:57 ` Daniel Pittman
1 sibling, 0 replies; 54+ messages in thread
From: Daniel Pittman @ 2002-02-04 23:57 UTC (permalink / raw)
On Mon, 04 Feb 2002, ipmonger@delamancha.org wrote:
> Ok, I updated to a newer XEmacs (21.5.4) and ran ELP on gnus with my
> 44000+ nnml folder for the Linux kernel, but I need some help
> understanding what these results indicate.
[...]
> Are profiling results cumulative for an Emacs/XEmacs session?
Maybe. If you set `elp-reset-after-results' to non-nil, getting the
results discards them at the same time.
Daniel
--
Among persons under 30, the inability to program is regarded as a form
of illiteracy. -- James Martin
(Comment predicting trends of the late 1990s, Made in the early 70's)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-20 20:03 ` Lars Magne Ingebrigtsen
@ 2002-01-20 20:33 ` Ami Fischman
0 siblings, 0 replies; 54+ messages in thread
From: Ami Fischman @ 2002-01-20 20:33 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Try `M-x debug-on-entry RET nntp-open-connection RET' and see what the
> parameters to `nntp-open-server' are in each of the nnvirtual-related
> openings.
The servers (and they are spelled identically).
Although, since at least this morning's CVS update, the subsequent "opening"
messages are very quick, so either my net cnxns are less loaded b/c it's
sunday, or it really is keeping the connections open. Wait on further
debugging on this for tuesday (monday is a holiday in the US) and I'll see
if the delay is back.
--
Ami Fischman
usenet@fischman.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-20 19:15 ` Lars Magne Ingebrigtsen
@ 2002-01-20 20:05 ` Ami Fischman
2002-01-20 20:03 ` Lars Magne Ingebrigtsen
0 siblings, 1 reply; 54+ messages in thread
From: Ami Fischman @ 2002-01-20 20:05 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> (gnus-server-equal
> (gnus-find-method-for-group "comp.lang.lisp")
> (gnus-find-method-for-group "comp.emacs"))
> (Use the group names that you have in the virtual group instead.)
> If this returns t, then it shouldn't open more than one connection.
> In theory. :-)
Theory schmeory. Indeed, it returns t...
--
Ami Fischman
usenet@fischman.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-20 20:05 ` Ami Fischman
@ 2002-01-20 20:03 ` Lars Magne Ingebrigtsen
2002-01-20 20:33 ` Ami Fischman
0 siblings, 1 reply; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-20 20:03 UTC (permalink / raw)
Ami Fischman <usenet@fischman.org> writes:
> Theory schmeory. Indeed, it returns t...
Try `M-x debug-on-entry RET nntp-open-connection RET' and see what the
parameters to `nntp-open-server' are in each of the nnvirtual-related
openings.
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-20 19:11 ` Ami Fischman
@ 2002-01-20 19:15 ` Lars Magne Ingebrigtsen
2002-01-20 20:05 ` Ami Fischman
0 siblings, 1 reply; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-20 19:15 UTC (permalink / raw)
Ami Fischman <usenet@fischman.org> writes:
> They all just say:
>
> "nntp:news.ISP.net"
>
> (same thing in each of the groups coming from this server).
Try evaluating the following:
(gnus-server-equal
(gnus-find-method-for-group "comp.lang.lisp")
(gnus-find-method-for-group "comp.emacs"))
(Use the group names that you have in the virtual group instead.)
If this returns t, then it shouldn't open more than one connection.
In theory. :-)
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-19 23:28 ` Lars Magne Ingebrigtsen
@ 2002-01-20 19:11 ` Ami Fischman
2002-01-20 19:15 ` Lars Magne Ingebrigtsen
0 siblings, 1 reply; 54+ messages in thread
From: Ami Fischman @ 2002-01-20 19:11 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
[...]
> This definitely sounds like a bug. Gnus is supposed to keep the
> connections open and re-use them as needed. The most likely reason I
> see for this not happening is that Gnus isn't able to recognize the
> groups as coming from the same server.
>
> What is the `G e' output on two of the component groups that are
> included in the two different nnvirtual groups?
They all just say:
"nntp:news.ISP.net"
(same thing in each of the groups coming from this server).
Then the virtual (say for my all-gnus-all-the-time group) says:
(nnvirtual "^nntp\\+news\\.ISP\\.net:gnu\\.emacs\\.gnus\\|nntp\\+news\\.gnus\\.org:gnus\\.ding$")
(and the other virtuals say similar things).
Ideas?
--
Ami Fischman
usenet@fischman.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-17 17:04 ` Ami Fischman
@ 2002-01-19 23:28 ` Lars Magne Ingebrigtsen
2002-01-20 19:11 ` Ami Fischman
0 siblings, 1 reply; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-19 23:28 UTC (permalink / raw)
Ami Fischman <usenet@fischman.org> writes:
> Not quite a "benchmarking" issue, more concerned w/ speed/effeciency. I
> have several nntp servers I use (main one + several "vendor" servers for
> specific programs, like gnus). I noticed that since adding virtual groups
> (which contain groups from multiple servers) the same server will be opened
> several times in one 'g' operation.
[...]
> Opening nntp server on news.ISP.net...done
[...]
> Opening nnvirtual server on ^nntp\+news\.ISP\.net:comp\.dcom\.xdsl\|nntp\+news\.ISP\.net:comp\.dcom\.telecom.tech$...done
> Opening nntp server on news.ISP.net...done
[...]
> Opening nnvirtual server on ^nntp\+news\.ISP\.net:gnu\.emacs\.gnus\|nntp\+news\.gnus\.org:gnus\.ding$...done
> Opening nntp server on news.ISP.net...done
[...]
> You'll note that news.ISP.net is opened several times, as is news.gnus.org.
> This wouldn't bother me, except the "Opening nntp server on..." message
> always takes a few seconds to scroll by, making me think that opening the
> extra connection is taking time that needn't be taken (for instance, if the
> first connection were kept open, or else, if _before_ checking new news,
> gnus figured out all the info it needs to ask of each server, and do it at
> the same time).
This definitely sounds like a bug. Gnus is supposed to keep the
connections open and re-use them as needed. The most likely reason I
see for this not happening is that Gnus isn't able to recognize the
groups as coming from the same server.
What is the `G e' output on two of the component groups that are
included in the two different nnvirtual groups?
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 6:35 ` Jesper Harder
@ 2002-01-19 23:25 ` Lars Magne Ingebrigtsen
0 siblings, 0 replies; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-19 23:25 UTC (permalink / raw)
Jesper Harder <harder@ifa.au.dk> writes:
> Oort was nearly twice as fast as 5.9 and around 10% quicker than 5.8.8
Cool!
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 4:42 ` Russ Allbery
@ 2002-01-19 23:25 ` Lars Magne Ingebrigtsen
0 siblings, 0 replies; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-19 23:25 UTC (permalink / raw)
Russ Allbery <rra@stanford.edu> writes:
> Any luck on the speed problems when pressing "g"? I'm not sure there's
> any other good profiling data that I can provide....
I haven't looked into that, but, yes, it seems to have gotten slower
lately, too...
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-18 23:42 ` Jesper Harder
@ 2002-01-19 11:38 ` Simon Josefsson
0 siblings, 0 replies; 54+ messages in thread
From: Simon Josefsson @ 2002-01-19 11:38 UTC (permalink / raw)
Cc: ding
Jesper Harder <harder@ifa.au.dk> writes:
> larsi@gnus.org (Lars Magne Ingebrigtsen) writes:
>
>> There are now three things that take about the same time when
>> entering a group, and account for 95% of the time spent:
>>
>> 1) parsing NOV headers (this has gotten slower due to MIME
>> stuff, I'm guessing) 2) scoring 3) creating and outputting
>> the threads
>>
>> If someone wants to look closer at this, then I'm all for it.
:-)
>
> Here's a small tweak, which speeds up threading a bit (around
> 5-10%): `gnus-parent-id' is usually called without the optional
> argument -- in that case we only want the last msg.id in
> References, so we don't really need to call
> `gnus-split-references' to parse *all* the references and build
> a list of them.
I committed it, thanks.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 2:12 Lars Magne Ingebrigtsen
` (3 preceding siblings ...)
2002-01-17 17:04 ` Ami Fischman
@ 2002-01-18 23:42 ` Jesper Harder
2002-01-19 11:38 ` Simon Josefsson
4 siblings, 1 reply; 54+ messages in thread
From: Jesper Harder @ 2002-01-18 23:42 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 696 bytes --]
larsi@gnus.org (Lars Magne Ingebrigtsen) writes:
> There are now three things that take about the same time when entering
> a group, and account for 95% of the time spent:
>
> 1) parsing NOV headers (this has gotten slower due to MIME stuff, I'm
> guessing)
> 2) scoring
> 3) creating and outputting the threads
>
> If someone wants to look closer at this, then I'm all for it. :-)
Here's a small tweak, which speeds up threading a bit (around 5-10%):
`gnus-parent-id' is usually called without the optional argument -- in
that case we only want the last msg.id in References, so we don't really
need to call `gnus-split-references' to parse *all* the references and
build a list of them.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnus-util.el.diff --]
[-- Type: text/x-patch, Size: 679 bytes --]
--- ../../gnus/lisp/gnus-util.el Wed Jan 16 05:14:26 2002
+++ gnus-util.el Fri Jan 18 22:58:21 2002
@@ -479,10 +479,13 @@
If N, return the Nth ancestor instead."
(when (and references
(not (zerop (length references))))
- (let ((ids (inline (gnus-split-references references))))
- (while (nthcdr (or n 1) ids)
- (setq ids (cdr ids)))
- (car ids))))
+ (if n
+ (let ((ids (inline (gnus-split-references references))))
+ (while (nthcdr n ids)
+ (setq ids (cdr ids)))
+ (car ids))
+ (when (string-match "<[^> \t]+>\\'" references)
+ (match-string 0 references)))))
(defun gnus-buffer-live-p (buffer)
"Say whether BUFFER is alive or not."
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 2:12 Lars Magne Ingebrigtsen
` (2 preceding siblings ...)
2002-01-16 12:45 ` Simon Josefsson
@ 2002-01-17 17:04 ` Ami Fischman
2002-01-19 23:28 ` Lars Magne Ingebrigtsen
2002-01-18 23:42 ` Jesper Harder
4 siblings, 1 reply; 54+ messages in thread
From: Ami Fischman @ 2002-01-17 17:04 UTC (permalink / raw)
Not quite a "benchmarking" issue, more concerned w/ speed/effeciency. I
have several nntp servers I use (main one + several "vendor" servers for
specific programs, like gnus). I noticed that since adding virtual groups
(which contain groups from multiple servers) the same server will be opened
several times in one 'g' operation.
For example, here is the *Messages* log:
Checking new news...
Opening nnvirtual server on ^nntp\+news\.ISP\.net:sci\.electronics\..*$...done
Opening nntp server on news.ISP.net...done
Opening nntp server on news.opera.com...done
Opening nntp server on news.gnus.org...done
Opening nnvirtual server on ^nntp\+news\.ISP\.net:comp\.dcom\.xdsl\|nntp\+news\.ISP\.net:comp\.dcom\.telecom.tech$...done
Opening nntp server on news.ISP.net...done
Opening nntp server on news.vmware.com...done
nnml: Reading incoming mail from file...
nnml: Reading incoming mail (no new mail)...done
Opening nnvirtual server on ^nntp\+news\.ISP\.net:gnu\.emacs\.gnus\|nntp\+news\.gnus\.org:gnus\.ding$...done
Opening nntp server on news.ISP.net...done
Opening nntp server on news.gnus.org...done
Opening nntp server on news.ISP.net...done
Checking new news...done
You'll note that news.ISP.net is opened several times, as is news.gnus.org.
This wouldn't bother me, except the "Opening nntp server on..." message
always takes a few seconds to scroll by, making me think that opening the
extra connection is taking time that needn't be taken (for instance, if the
first connection were kept open, or else, if _before_ checking new news,
gnus figured out all the info it needs to ask of each server, and do it at
the same time).
Any idea how to avoid this?
--
Ami Fischman
usenet@fischman.org
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 12:45 ` Simon Josefsson
2002-01-16 16:51 ` Josh Huber
2002-01-16 19:26 ` Steinar Bang
@ 2002-01-17 2:06 ` Daniel Pittman
2 siblings, 0 replies; 54+ messages in thread
From: Daniel Pittman @ 2002-01-17 2:06 UTC (permalink / raw)
On Wed, 16 Jan 2002, Simon Josefsson wrote:
> On Wed, 16 Jan 2002, Lars Magne Ingebrigtsen wrote:
>
>> There are now three things that take about the same time when
>> entering a group, and account for 95% of the time spent:
[...]
> A long term solution would be to redesign this stuff and make it run
> asynchronously. The summary buffer would appear immediately containing
> as many empty lines as there are messages that should be shown, and
> then things are populated in the background. This might even give us
> some more flexibility, like allowing backends to perform threading
> (IMAP supports this).
>
> But long term solutions should perhaps be targetted at Guile.. gimme a
> guile emacs! :-)
The day they write it, and it's shown in beta, is the day to start
writing code for it, I think.
Because, you know, that means I never have to deal with a Scheme based
Emacs at all. ;)
Daniel
--
In the present-day reconstruction of physics, fragments of the Newtonian
concepts are stubbornly retained. The result is to reduce modern physics to a
sort of mystic chant over an unintelligible universe.
-- Alfred North Whitehead, _Modes of Thought_
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 12:45 ` Simon Josefsson
2002-01-16 16:51 ` Josh Huber
@ 2002-01-16 19:26 ` Steinar Bang
2002-01-17 2:06 ` Daniel Pittman
2 siblings, 0 replies; 54+ messages in thread
From: Steinar Bang @ 2002-01-16 19:26 UTC (permalink / raw)
>>>>> Simon Josefsson <jas@extundo.com>:
> A long term solution would be to redesign this stuff and make it run
> asynchronously.
Yes! Yes! Yes!
> The summary buffer would appear immediately containing as many empty
> lines as there are messages that should be shown, and then things
> are populated in the background. This might even give us some more
> flexibility, like allowing backends to perform threading (IMAP
> supports this).
Sounds good to me!
> But long term solutions should perhaps be targetted at Guile..
> gimme a guile emacs! :-)
Aww... not _that_ long term, I hope!
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 16:51 ` Josh Huber
@ 2002-01-16 17:21 ` Karl Kleinpaste
0 siblings, 0 replies; 54+ messages in thread
From: Karl Kleinpaste @ 2002-01-16 17:21 UTC (permalink / raw)
Josh Huber <huber@alum.wpi.edu> writes:
> Er, doesn't this require some support from emacs for threads? Isn't
> there none?
Not exactly. See function gnus-date-start-timer and
date-treat-date-lapsed, to say nothing of gnus-asynchronous.
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 12:45 ` Simon Josefsson
@ 2002-01-16 16:51 ` Josh Huber
2002-01-16 17:21 ` Karl Kleinpaste
2002-01-16 19:26 ` Steinar Bang
2002-01-17 2:06 ` Daniel Pittman
2 siblings, 1 reply; 54+ messages in thread
From: Josh Huber @ 2002-01-16 16:51 UTC (permalink / raw)
Simon Josefsson <jas@extundo.com> writes:
> A long term solution would be to redesign this stuff and make it run
> asynchronously. The summary buffer would appear immediately
> containing as many empty lines as there are messages that should be
> shown, and then things are populated in the background. This might
> even give us some more flexibility, like allowing backends to
> perform threading (IMAP supports this).
Er, doesn't this require some support from emacs for threads? Isn't
there none?
> But long term solutions should perhaps be targetted at Guile.. gimme a
> guile emacs! :-)
Hehe, yeah, but that's a while away, I think :)
--
Josh Huber
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 2:12 Lars Magne Ingebrigtsen
2002-01-16 4:42 ` Russ Allbery
2002-01-16 6:35 ` Jesper Harder
@ 2002-01-16 12:45 ` Simon Josefsson
2002-01-16 16:51 ` Josh Huber
` (2 more replies)
2002-01-17 17:04 ` Ami Fischman
2002-01-18 23:42 ` Jesper Harder
4 siblings, 3 replies; 54+ messages in thread
From: Simon Josefsson @ 2002-01-16 12:45 UTC (permalink / raw)
Cc: ding
On Wed, 16 Jan 2002, Lars Magne Ingebrigtsen wrote:
> There are now three things that take about the same time when entering
> a group, and account for 95% of the time spent:
>
> 1) parsing NOV headers (this has gotten slower due to MIME stuff, I'm
> guessing)
> 2) scoring
> 3) creating and outputting the threads
A short term solution would be to cache things. Shenghuo's cache and
agent unification probably makes this easier and more reliable.
A long term solution would be to redesign this stuff and make it run
asynchronously. The summary buffer would appear immediately containing as
many empty lines as there are messages that should be shown, and then
things are populated in the background. This might even give us some more
flexibility, like allowing backends to perform threading (IMAP supports
this).
But long term solutions should perhaps be targetted at Guile.. gimme a
guile emacs! :-)
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 2:12 Lars Magne Ingebrigtsen
2002-01-16 4:42 ` Russ Allbery
@ 2002-01-16 6:35 ` Jesper Harder
2002-01-19 23:25 ` Lars Magne Ingebrigtsen
2002-01-16 12:45 ` Simon Josefsson
` (2 subsequent siblings)
4 siblings, 1 reply; 54+ messages in thread
From: Jesper Harder @ 2002-01-16 6:35 UTC (permalink / raw)
larsi@gnus.org (Lars Magne Ingebrigtsen) writes:
> Inspired by the recent discussion on news.software.readers, I've been
> benchmarking group entry a bit. [..] If someone wants to look closer
> at this, then I'm all for it. :-)
Oort seems be getting getting just as fast as 5.8.8 now. I tried
entering a 10K group with
Emacs 21 + Oort Gnus,
Emacs 21 + Gnus 5.9,
Emacs 20.7 + Gnus 5.8.8.
Oort was nearly twice as fast as 5.9 and around 10% quicker than 5.8.8
^ permalink raw reply [flat|nested] 54+ messages in thread
* Re: Benchmarking Gnus
2002-01-16 2:12 Lars Magne Ingebrigtsen
@ 2002-01-16 4:42 ` Russ Allbery
2002-01-19 23:25 ` Lars Magne Ingebrigtsen
2002-01-16 6:35 ` Jesper Harder
` (3 subsequent siblings)
4 siblings, 1 reply; 54+ messages in thread
From: Russ Allbery @ 2002-01-16 4:42 UTC (permalink / raw)
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> Inspired by the recent discussion on news.software.readers, I've been
> benchmarking group entry a bit. I've used ELP, and it led me to find
> some weaknesses in the manner format specs are used. (The pre-compiled
> functions were supposed to be used for default values, but they
> weren't.) After fixing that, and inlining some other functions, I got
> group entry of a 1K group down from 2 seconds to 1.5 seconds. (Yes,
> this is a new and spiffy machine. :-)
Any luck on the speed problems when pressing "g"? I'm not sure there's
any other good profiling data that I can provide....
--
Russ Allbery (rra@stanford.edu) <http://www.eyrie.org/~eagle/>
^ permalink raw reply [flat|nested] 54+ messages in thread
* Benchmarking Gnus
@ 2002-01-16 2:12 Lars Magne Ingebrigtsen
2002-01-16 4:42 ` Russ Allbery
` (4 more replies)
0 siblings, 5 replies; 54+ messages in thread
From: Lars Magne Ingebrigtsen @ 2002-01-16 2:12 UTC (permalink / raw)
Inspired by the recent discussion on news.software.readers, I've been
benchmarking group entry a bit. I've used ELP, and it led me to find
some weaknesses in the manner format specs are used. (The
pre-compiled functions were supposed to be used for default values,
but they weren't.) After fixing that, and inlining some other
functions, I got group entry of a 1K group down from 2 seconds to 1.5
seconds. (Yes, this is a new and spiffy machine. :-)
There are now three things that take about the same time when entering
a group, and account for 95% of the time spent:
1) parsing NOV headers (this has gotten slower due to MIME stuff, I'm
guessing)
2) scoring
3) creating and outputting the threads
If someone wants to look closer at this, then I'm all for it. :-)
--
(domestic pets only, the antidote for overdose, milk.)
larsi@gnus.org * Lars Magne Ingebrigtsen
^ permalink raw reply [flat|nested] 54+ messages in thread
end of thread, other threads:[~2002-02-04 23:57 UTC | newest]
Thread overview: 54+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-01-27 0:30 Benchmarking Gnus Lars Magne Ingebrigtsen
2002-01-27 0:35 ` IPmonger
2002-01-27 0:45 ` Lars Magne Ingebrigtsen
2002-01-29 16:03 ` IPmonger
2002-01-30 8:53 ` Kai Großjohann
2002-01-30 16:42 ` IPmonger
2002-01-30 17:38 ` Kai Großjohann
2002-01-30 18:15 ` Lars Magne Ingebrigtsen
2002-01-30 18:42 ` Josh Huber
2002-02-04 23:35 ` IPmonger
2002-02-04 23:48 ` IPmonger
2002-02-04 23:57 ` Daniel Pittman
2002-01-30 22:02 ` Wes Hardaker
2002-01-30 22:09 ` IPmonger
2002-01-31 1:25 ` Wes Hardaker
2002-01-31 3:22 ` ShengHuo ZHU
2002-01-31 15:29 ` Wes Hardaker
2002-01-31 16:34 ` Steinar Bang
2002-01-31 16:54 ` Bjørn Mork
2002-01-31 20:50 ` Steinar Bang
2002-01-31 22:35 ` Bjørn Mork
2002-02-01 7:14 ` Steinar Bang
2002-02-01 9:17 ` Bjørn Mork
2002-02-01 10:00 ` Steinar Bang
2002-02-01 11:24 ` Bjørn Mork
2002-02-01 11:29 ` Simon Josefsson
2002-02-01 12:52 ` Steinar Bang
2002-02-01 13:40 ` Simon Josefsson
2002-02-01 14:12 ` Steinar Bang
2002-02-01 14:39 ` ShengHuo ZHU
2002-02-01 15:01 ` Simon Josefsson
2002-02-01 15:31 ` Steinar Bang
2002-02-01 14:59 ` Simon Josefsson
2002-02-01 15:18 ` Steinar Bang
2002-02-01 16:51 ` Wes Hardaker
-- strict thread matches above, loose matches on Subject: below --
2002-01-16 2:12 Lars Magne Ingebrigtsen
2002-01-16 4:42 ` Russ Allbery
2002-01-19 23:25 ` Lars Magne Ingebrigtsen
2002-01-16 6:35 ` Jesper Harder
2002-01-19 23:25 ` Lars Magne Ingebrigtsen
2002-01-16 12:45 ` Simon Josefsson
2002-01-16 16:51 ` Josh Huber
2002-01-16 17:21 ` Karl Kleinpaste
2002-01-16 19:26 ` Steinar Bang
2002-01-17 2:06 ` Daniel Pittman
2002-01-17 17:04 ` Ami Fischman
2002-01-19 23:28 ` Lars Magne Ingebrigtsen
2002-01-20 19:11 ` Ami Fischman
2002-01-20 19:15 ` Lars Magne Ingebrigtsen
2002-01-20 20:05 ` Ami Fischman
2002-01-20 20:03 ` Lars Magne Ingebrigtsen
2002-01-20 20:33 ` Ami Fischman
2002-01-18 23:42 ` Jesper Harder
2002-01-19 11:38 ` Simon Josefsson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).