Re: Speed difference in Gnus

Re: Speed difference in Gnus

[Reiner Steib]

On Thu, Oct 13 2005, Hrvoje Niksic wrote:

> The speed difference between XEmacs and GNU Emacs is amazing, in that
> GNU Emacs is significantly faster, at least for "heavy" usage such as
> Gnus.

Interesting.  Which versions of (X)Emacs and Gnus did you test?

> Entering a large summary buffer takes time proportional to the
> *square* of the number of messages in the folder.  (This is not a
> wild guess, a friend has actually measured.)  Time to enter a folder
> of more than 3000 articles quickly degrades to minutes.
>
> For the longest time I believed this to be a result of XEmacs's
> implementation of text properties on top of extents.  I planned to
> convert Gnus to use extents natively, in the hope that it would speed
> up summary buffer generation.  

Then `M-RET' should be significantly faster, shouldn't it?

,----[ <f1> f gnus-group-quick-select-group RET ]
| gnus-group-quick-select-group is an interactive compiled Lisp
| function in `gnus-group.el'.
| (gnus-group-quick-select-group &optional ALL)
| 
| Select the current group "quickly".
| This means that no highlighting or scoring will be performed.
| If ALL (the prefix argument) is 0, don't even generate the summary
| buffer.
| 
| This might be useful if you want to toggle threading
| before entering the group.
`----

> But according to profiling, most of the time is spent in `insert',
> at least once you discount excessive GC by increasing
> gc-cons-theshold.  I'm not sure what XEmacs is doing wrong here.  I
> suspect Mule is to blame, but I can't prove it.

`C-u 0 M-RET' probably doesn't use insert so often.  Is it comparable
to `C-u 0 M-RET' in Emacs then?

> Using FSF Emacs gave me an order-of-magnitude speedup when entering
> large newsgroups and large mailing list folders.  It's uncanny.

Bye, Reiner.

PS: Cc-ing ding.  This might be interesting for Gnus developers.  See
    http://thread.gmane.org/gmane.emacs.xemacs.beta/20690 for the
    whole thread.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "APA" == Adrian Aichner <adrian@xemacs.org> writes:

   APA> Uwe Brauer <oub@mat.ucm.es> writes:

[snip]


   APA> a* profile-command                - Run COMMAND, profiling the
   APA>    execution and displaying the results. 

Hello Adrian,

Thanks this command worked fine, I did various tests, either having all
my packages loaded or not, I attach the one, with all packages used.
I looked for profile commands in gnus emacs, there is no per default,
google did not really help, 

Could somebody of the gnu folks[1] please point me out how to profile a
command in either gnu emacs 21 or 22.

Thanks

Uwe Brauer 

[1] thats why I post also to the gnus group

Re: Speed difference in Gnus

[David Kastrup]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "APA" == Adrian Aichner <adrian@xemacs.org> writes:
>
>    APA> Uwe Brauer <oub@mat.ucm.es> writes:
>
> [snip]
>
>
>    APA> a* profile-command                - Run COMMAND, profiling the
>    APA>    execution and displaying the results. 
>
> Hello Adrian,
>
> Thanks this command worked fine, I did various tests, either having all
> my packages loaded or not, I attach the one, with all packages used.
> I looked for profile commands in gnus emacs, there is no per default,
> google did not really help, 
>
> Could somebody of the gnu folks[1] please point me out how to profile a
> command in either gnu emacs 21 or 22.

M-x elp-instrument-package RET gnus RET
Do something
M-x elp-results RET

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:

   David> M-x elp-instrument-package RET gnus RET
   David> Do something
   David> M-x elp-results RET

Thanks, this function also exists in xemacs (and I did not know about
it), in any case
the information is very aehm spare:
for emacs-21 I obtain:
--8<------------------------schnipp------------------------->8---
Function Name              Call Count  Elapsed Time  Average Time
=========================  ==========  ============  ============
gnus-summary-copy-article  1           14.03726      14.03726
--8<------------------------schnapp------------------------->8---
for xemacs 21.4.17 I get
--8<------------------------schnipp------------------------->8---
Function Name              Call Count  Elapsed Time  Average Time
=========================  ==========  ============  ============
gnus-summary-copy-article  1           15.869385     15.869385
--8<------------------------schnapp------------------------->8---

However the profile-command of xemacs give me detailed information of
the sort:
--8<------------------------schnipp------------------------->8---
Function Name                         Ticks/Total %Usage Calls GC-Usage/  Total
===========================================/===== ====== ===== ========/=======
(in garbage collection)                 237/    0 39.109                       
next-single-property-change              70/    0 11.551 24664                 
get-text-property                        33/    0  5.446 29299                 
--8<------------------------schnapp------------------------->8---


Etc etc

I tried to compile xemacs, profile package under emacs-21 without
success. Does there exist any add-on package for more profiling
details?

Thanks

Uwe Brauer 

Re: Speed difference in Gnus

[David Kastrup]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>
>    David> M-x elp-instrument-package RET gnus RET
>    David> Do something
>    David> M-x elp-results RET
>
> Thanks, this function also exists in xemacs (and I did not know about
> it), in any case
> the information is very aehm spare:
> for emacs-21 I obtain:
> --8<------------------------schnipp------------------------->8---
> Function Name              Call Count  Elapsed Time  Average Time
> =========================  ==========  ============  ============
> gnus-summary-copy-article  1           14.03726      14.03726
> --8<------------------------schnapp------------------------->8---
> for xemacs 21.4.17 I get
> --8<------------------------schnipp------------------------->8---
> Function Name              Call Count  Elapsed Time  Average Time
> =========================  ==========  ============  ============
> gnus-summary-copy-article  1           15.869385     15.869385
> --8<------------------------schnapp------------------------->8---

Well, you should instrument the package _after_ loading all relevant
functions, or just the autoloads get instrumented.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Speed difference in Gnus

[Jesper Harder]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>
>    David> M-x elp-results RET
>
> Thanks, this function also exists in xemacs (and I did not know
> about it), in any case the information is very aehm spare:

You should probably also instrument the `nnml', `imap', `mm' and `mml'
packages (and maybe some more depending on which operation you're
benchmarking -- I don't recall).

Re: Speed difference in Gnus

[David Kastrup]

Jesper Harder <jesper.harder@gmail.com> writes:

> Uwe Brauer <oub@mat.ucm.es> writes:
>
>>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>>
>>    David> M-x elp-results RET
>>
>> Thanks, this function also exists in xemacs (and I did not know
>> about it), in any case the information is very aehm spare:
>
> You should probably also instrument the `nnml', `imap', `mm' and `mml'
> packages (and maybe some more depending on which operation you're
> benchmarking -- I don't recall).

And it helps to run the test first before instrumenting it: that way
not only the autoloads get instrumented.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "David" == David Kastrup <dak@gnu.org> writes:
   >> 
   >>>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
   >>> 
   David> M-x elp-results RET
   >>> 
   >>> Thanks, this function also exists in xemacs (and I did not know
   >>> about it), in any case the information is very aehm spare:
   >> 
   >> You should probably also instrument the `nnml', `imap', `mm' and `mml'
   >> packages (and maybe some more depending on which operation you're
   >> benchmarking -- I don't recall).


   David> And it helps to run the test first before instrumenting it: that way
   David> not only the autoloads get instrumented.

Ok, thanks, I think now more or less I know how to proceed. As I said
before the situation is sort of strange: for a certain amount of msg,
say n, both emacsen are almost equal, gnu emacs a little faster, once
the number of msg > n, then xemacs got stacked.

I should add, that the information of the elp packages is detailed the
profile-command of xemacs gives far more information. I don't know how
important profiling is for emacs, but may that pkg could be ported?

Re: Speed difference in Gnus

[Ben Wing]

Uwe Brauer wrote:

>>>>>>"David" == David Kastrup <dak@gnu.org> writes:
>>>>>>            
>>>>>>
>   >> 
>   >>>>>>>> "David" == David Kastrup <dak@gnu.org> writes:
>   >>> 
>   David> M-x elp-results RET
>   >>> 
>   >>> Thanks, this function also exists in xemacs (and I did not know
>   >>> about it), in any case the information is very aehm spare:
>   >> 
>   >> You should probably also instrument the `nnml', `imap', `mm' and `mml'
>   >> packages (and maybe some more depending on which operation you're
>   >> benchmarking -- I don't recall).
>
>
>   David> And it helps to run the test first before instrumenting it: that way
>   David> not only the autoloads get instrumented.
>
>Ok, thanks, I think now more or less I know how to proceed. As I said
>before the situation is sort of strange: for a certain amount of msg,
>say n, both emacsen are almost equal, gnu emacs a little faster, once
>the number of msg > n, then xemacs got stacked.
>  
>
it would be very nice to know where the problem is.  sometimes typing 
Control-Shift-G in XEmacs will break out and give you a backtrace. (it 
should always do that but maybe sometimes doesn't ...) If not, try 
setting Options->Troubleshooting->Debug on Quit, then manually do `kill 
-INT ###', where ### is the process id of XEmacs. (maybe it's `kill 
-SIGINT ###', i forget)

>I should add, that the information of the elp packages is detailed the
>profile-command of xemacs gives far more information. I don't know how
>important profiling is for emacs, but may that pkg could be ported?
>  
>
the problem is that the XEmacs profile stuff depends on C support, which 
is how it's able to do all the nifty things it does.

ben

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Ben" == Ben Wing <ben@xemacs.org> writes:

   Ben> Uwe Brauer wrote:
   >> 
   >> 
   Ben> it would be very nice to know where the problem is.  sometimes
   Ben> typing

it seems that it has to do with the size of the msg. Mails with large
attachment are the problem in xemacs, but still I have to look  that
up in detail.

   Ben> Control-Shift-G in XEmacs will break out and give you a
   Ben> backtrace. (it should always do that but maybe sometimes
   Ben> doesn't ...) If not, try setting
   Ben> Options->Troubleshooting->Debug on Quit, then manually do
   Ben> `kill -INT ###', where ### is the process id of XEmacs. (maybe
   Ben> it's `kill -SIGINT ###', i forget)
problem is that under recent X-free versions the shift key does not
work properly, that is control shift g, is the same as control g for
xemacs, but I do as you suggest set is debug on quit.

I report back, 


Uwe 

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Ben" == Ben Wing <ben@xemacs.org> writes:

   Ben> Uwe Brauer wrote:
   >> 
   >> 
   Ben> it would be very nice to know where the problem is.
   Ben> sometimes typing Control-Shift-G in XEmacs will break out
   Ben> and give you a backtrace. (it should always do that but
   Ben> maybe sometimes doesn't ...) If not, try setting
   Ben> Options->Troubleshooting->Debug on Quit, then manually do
   Ben> `kill -INT ###', where ### is the process id of
   Ben> XEmacs. (maybe it's `kill -SIGINT ###', i forget)

Ok, I continued with the testing:

    - the problem  does not occur when  copying a large amount of
      msg between nnml servers! Xemacs is as fast as gnu emacs

    - the problem seem to be connected with `large' msg: msg with
      an attachment >0.5 Mega. I  tried this with xemacs -vanilla
      either nognus-0.3 or the actual xemacs pkg release. Now
      xemacs does not really get stack, after some Control g key
      strokes I can continue typing (and even with
      debug-on-signal t no useful backtrace is produced). However the
      nnimap command got *stacked*. I can not leave neither the group
      not the server
      The command
      lisp-process
      lists me the imap command 
      
imap         open	   *nnimap* ucimap.ucm.es (none) network stream
connection 143@ucimap.ucm.es 

      could please somebody confirm this?

But  I don't know how to kill that process, ps ux does not list that
process.


Again that problem does not occur in gnu emacs and now I begin to
understand why me complains about the slowness of the nnimap command
were ignored on the gnus list: it seems to be more of a xemacs
problem.

If someone could suggest me how to enhance the testing in order to
localise the problem...

Regards

Uwe Brauer 

Re: Speed difference in Gnus

[Simon Josefsson]

Uwe Brauer <oub@mat.ucm.es> writes:

>       The command
>       lisp-process
>       lists me the imap command 
>       
> imap         open	   *nnimap* ucimap.ucm.es (none) network stream
> connection 143@ucimap.ucm.es 
>
> But  I don't know how to kill that process, ps ux does not list that
> process.

Kill the buffer " *nnimap *ucimap.ucm.es" (note leading SPC).

> If someone could suggest me how to enhance the testing in order to
> localise the problem...

Instrument imap* and nnimap* for ELP.  Note that nnimap is likely to
be slowed than nnml, but I'm not sure what magnitude of "slower" you
are talking about here.

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:

   Simon> Uwe Brauer <oub@mat.ucm.es> writes:

   Simon> Kill the buffer " *nnimap *ucimap.ucm.es" (note leading SPC).

   >> If someone could suggest me how to enhance the testing in order to
   >> localise the problem...

   Simon> Instrument imap* and nnimap* for ELP.  Note that nnimap is
I did elp-instrument-package gnus (which I thought include everyting)


Or or profile-command (gnus-summary-copy-article) for xemacs.

But right I do as you say
   Simon> likely to be slowed than nnml, but I'm not sure what
   Simon> magnitude of "slower" you are talking about here.

Get stacked, I mean not doing anything (xemacs) compared to perform a
operation (gnu emacs)

Re: Speed difference in Gnus

[Simon Josefsson]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:
>
>    Simon> Uwe Brauer <oub@mat.ucm.es> writes:
>
>    Simon> Kill the buffer " *nnimap *ucimap.ucm.es" (note leading SPC).
>
>    >> If someone could suggest me how to enhance the testing in order to
>    >> localise the problem...
>
>    Simon> Instrument imap* and nnimap* for ELP.  Note that nnimap is
> I did elp-instrument-package gnus (which I thought include everyting)

No, the (nn)imap functions in Gnus doesn't use the gnus function name
prefix, you need to instrument them manually.  Same for message, mm,
pgg, sieve etc.

>    Simon> likely to be slowed than nnml, but I'm not sure what
>    Simon> magnitude of "slower" you are talking about here.
>
> Get stacked, I mean not doing anything (xemacs) compared to perform a
> operation (gnu emacs)

Aha.  Ok.  That's bad.

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:
   >> 
   >> But  I don't know how to kill that process, ps ux does not list that
   >> process.

   Simon> Kill the buffer " *nnimap *ucimap.ucm.es" (note leading SPC).

That buffer does not exist in my case. I can only see the process but
I don't have that buffer (in xemacs-21.4.17)

Re: Speed difference in Gnus

[Simon Josefsson]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:
>    >> 
>    >> But  I don't know how to kill that process, ps ux does not list that
>    >> process.
>
>    Simon> Kill the buffer " *nnimap *ucimap.ucm.es" (note leading SPC).
>
> That buffer does not exist in my case. I can only see the process but
> I don't have that buffer (in xemacs-21.4.17)

Sorry, I meant " *nnimap* ucimap.ucm.es".  Try typing `C-x b SPC
*nnimap*' and use TAB completion.

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:


[snip]


   Simon> Sorry, I meant " *nnimap* ucimap.ucm.es".  Try typing `C-x b
   Simon> SPC *nnimap*' and use TAB completion.

Ok that works, however since the process get stacked I don't get any
reasonable backtrace. If somebody could confirm, that for large msg
nnimap copying does not work properly under xemacs?


Uwe 

Re: Speed difference in Gnus

[Simon Josefsson]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:
>
>
> [snip]
>
>
>    Simon> Sorry, I meant " *nnimap* ucimap.ucm.es".  Try typing `C-x b
>    Simon> SPC *nnimap*' and use TAB completion.
>
> Ok that works, however since the process get stacked I don't get any
> reasonable backtrace.

I don't understand.  Does XEmacs freeze completely?  If so, I don't
think there is anything we can do at the Elisp level.  It must be
debugged as an XEmacs bug.

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Simon" == Simon Josefsson <jas@extundo.com> writes:

   >> 
   >> Ok that works, however since the process get stacked I don't get any
   >> reasonable backtrace.

   Simon> I don't understand.  Does XEmacs freeze completely?  If so, I don't
   Simon> think there is anything we can do at the Elisp level.  It must be
   Simon> debugged as an XEmacs bug.

No what I say is: I want to copy that msg and xemacs starts to tell me
copying msg, but nothing happens after 2 min after several Control g I
can type again, however it seems that the nnimap command is stacked, I
cannot leave the group nor the server, but I have to do what you told
me.

So for me it not clear who is to blame. However that problem does not
occur in gnu emacs with the same gnus (ngnus 0.3)

Re: Speed difference in Gnus

[Ben Wing]

Uwe Brauer wrote:

>>>>>>"Simon" == Simon Josefsson <jas@extundo.com> writes:
>>>>>>            
>>>>>>
>
>   >> 
>   >> Ok that works, however since the process get stacked I don't get any
>   >> reasonable backtrace.
>
>   Simon> I don't understand.  Does XEmacs freeze completely?  If so, I don't
>   Simon> think there is anything we can do at the Elisp level.  It must be
>   Simon> debugged as an XEmacs bug.
>
>No what I say is: I want to copy that msg and xemacs starts to tell me
>copying msg, but nothing happens after 2 min after several Control g I
>can type again, however it seems that the nnimap command is stacked, I
>cannot leave the group nor the server, but I have to do what you told
>me.
>
>So for me it not clear who is to blame. However that problem does not
>occur in gnu emacs with the same gnus (ngnus 0.3)
>  
>
a backtrace would help.  i assume you're running under linux?  you can 
do `ps -auxww' or `ps -edalf' or something like that to list all 
processes, and then attach to the xemacs process that's hung, and do 
`where' to get a c backtrace and `call debug_backtrace()' to get a lisp 
backtrace.  one of the faq questions in section 2.4 covers this in more 
detail.

ben

Re: Speed difference in Gnus

[Uwe Brauer]

>>>>> "Ben" == Ben Wing <ben@xemacs.org> writes:

   >> 
   >> 
   Ben> it would be very nice to know where the problem is.  sometimes
   Ben> typing Control-Shift-G in XEmacs will break out and give you a
   Ben> backtrace. (it should always do that but maybe sometimes
   Ben> doesn't ...) If not, try setting
   Ben> Options->Troubleshooting->Debug on Quit, then manually do
   Ben> `kill -INT ###', where ### is the process id of XEmacs. (maybe
   Ben> it's `kill -SIGINT ###', i forget)

I forgot to add: in my nnimap folder/groups I have activated Gcc (to
that group) so that, via the nnimap command, my reply to a mail is
copied to that nnimap group. That command as well gets *stacked*, but
not in gnu emacs!

So I have the feeling that nnimap is not optimized for xemacs to put
it mildly.


Uwe 

Re: Speed difference in Gnus

[Reiner Steib]

On Mon, Oct 24 2005, Uwe Brauer wrote:

> Could somebody of the gnu folks[1] please point me out how to profile a
> command in either gnu emacs 21 or 22.

,----
| elp-instrument-function	      M-x ... RET
|   Command: Instrument FUNSYM for profiling.
| elp-instrument-list	      M-x ... RET
|   Command: Instrument for profiling, all functions in `elp-function-list'.
| elp-instrument-package	      M-x ... RET
|   Command: Instrument for profiling, all functions which start with PREFIX.
| elp-results		      M-x ... RET
|   Command: Display current profiling results.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/