* No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
@ 2006-03-20 0:23 Elias Oltmanns
2006-03-20 6:02 ` Daniel Pittman
0 siblings, 1 reply; 11+ messages in thread
From: Elias Oltmanns @ 2006-03-20 0:23 UTC (permalink / raw)
Hi there,
accessing a 25MB nnfolder takes at least twice as much time and
sometimes even up to four times as much when running No Gnus from
within emacs-snapshot (Debian packaged cvs version) than running in
emacs 21.4. The reason I'm reporting this here is that visiting the
file with C-x C-f directly from emacs does not reveal such a
difference in speed. Yet, every operation performed by gnus on this
nnfolder, as entering, reading articles, expiring, etc, challenges my
patience, to say the least. This is even true when
large-file-warning-threshold is set to nil, however, it most
definitely seems to be size related as smaller folders don't take that
much when being processed. As I'm new to emacs 22 and still rather
unfamiliar with all those nice new features, all this makes me wonder
whether I've just missed another new option which has to be adapted
for better gnus performance.
For the record, here is the string output by M-x emacs-version:
GNU Emacs 22.0.50.1 (i486-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
of 2006-01-16 on vernadsky, modified by Debian
Thanks for your attention and advice.
Regards,
Elias
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 0:23 No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders Elias Oltmanns
@ 2006-03-20 6:02 ` Daniel Pittman
2006-03-20 9:52 ` Elias Oltmanns
0 siblings, 1 reply; 11+ messages in thread
From: Daniel Pittman @ 2006-03-20 6:02 UTC (permalink / raw)
Elias Oltmanns <oltmanns@uni-bonn.de> writes:
> accessing a 25MB nnfolder takes at least twice as much time and
> sometimes even up to four times as much when running No Gnus from
> within emacs-snapshot (Debian packaged cvs version) than running in
> emacs 21.4. The reason I'm reporting this here is that visiting the
> file with C-x C-f directly from emacs does not reveal such a
> difference in speed. Yet, every operation performed by gnus on this
> nnfolder, as entering, reading articles, expiring, etc, challenges my
> patience, to say the least.
Well, the best thing to do is probably to profile the operation of
visiting a folder, allowing us to see exactly where time is spent.
Under XEmacs the function `profile-key-sequence' is very useful for
this, but I don't know if that is supported by GNU Emacs.
Searching for `profile' related functions, and reading about profiling
ELisp in the Emacs manual, should point you in the right direction,
though.
The results of that will show where time is being spent, and so allow us
(or you) to know why it is slow and, hopefully, fix it.
Daniel
--
Digital Infrastructure Solutions -- making IT simple, stable and secure
Phone: 0401 155 707 email: contact@digital-infrastructure.com.au
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 6:02 ` Daniel Pittman
@ 2006-03-20 9:52 ` Elias Oltmanns
2006-03-20 11:30 ` Daniel Pittman
0 siblings, 1 reply; 11+ messages in thread
From: Elias Oltmanns @ 2006-03-20 9:52 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]
Daniel Pittman <daniel@rimspace.net> wrote:
> Elias Oltmanns <oltmanns@uni-bonn.de> writes:
>
>> accessing a 25MB nnfolder takes at least twice as much time and
>> sometimes even up to four times as much when running No Gnus from
>> within emacs-snapshot (Debian packaged cvs version) than running in
>> emacs 21.4. The reason I'm reporting this here is that visiting the
>> file with C-x C-f directly from emacs does not reveal such a
>> difference in speed. Yet, every operation performed by gnus on this
>> nnfolder, as entering, reading articles, expiring, etc, challenges my
>> patience, to say the least.
>
> Well, the best thing to do is probably to profile the operation of
> visiting a folder, allowing us to see exactly where time is spent.
Here are the profiles. I went about as follows:
- Start up emacs2[12],
- load elp.el,
- M-x set-varialbe RET elp-sort-by-function RET elp-sort-by-average-time RET,
- start up gnus,
- M-x elp-instrument-package RET gnus RET,
- enter folder,
- M-x elp-results.
Note that there are no delays due to disk access as all data had been
in cache already.
Hope that helps.
Elias
---- Profile emacs 21.4.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: prof21 --]
[-- Type: text/x-patch, Size: 15814 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 3.616749 3.616749
gnus-group-read-group 1 3.616719 3.616719
gnus-summary-read-group 1 3.6166080000 3.6166080000
gnus-summary-read-group-1 1 3.616585 3.616585
gnus-select-newsgroup 1 2.507384 2.507384
gnus-fetch-headers 1 2.500889 2.500889
gnus-cache-retrieve-headers 2 2.32714 1.16357
gnus-retrieve-headers 4 4.653691 1.16342275
gnus-summary-prepare 1 0.979999 0.979999
gnus-summary-prepare-threads 1 0.952893 0.952893
gnus-summary-prepare-unthreaded 1 0.951861 0.951861
gnus-get-newsgroup-headers-xover 1 0.588485 0.588485
gnus-summary-setup-buffer 1 0.0971 0.0971
gnus-summary-mode 1 0.09672 0.09672
gnus-make-thread-indent-array 1 0.07127 0.07127
gnus-sort-articles 1 0.02698 0.02698
gnus-summary-make-menu-bar 1 0.011338 0.011338
gnus-group-update-group 1 0.009127 0.009127
gnus-summary-initial-limit 1 0.008538 0.008538
gnus-build-sparse-threads 1 0.004057 0.004057
gnus-group-set-mode-line 2 0.007885 0.0039425
gnus-set-mode-line 1 0.003797 0.003797
gnus-mode-line-buffer-identification 3 0.011299 0.0037663333
gnus-dribble-enter 1 0.003544 0.003544
gnus-update-summary-mark-positions 2 0.0062510000 0.0031255000
gnus-summary-insert-line 382 0.9456889999 0.0024756256
gnus-byte-compile 2 0.0048740000 0.0024370000
gnus-update-format-specifications 2 0.0034289999 0.0017144999
gnus-update-missing-marks 1 0.001617 0.001617
gnus-user-format-function-b 380 0.6013829999 0.0015825868
gnus-possibly-score-headers 1 0.001458 0.001458
gnus-all-score-files 1 0.001248 0.001248
gnus-articles-to-read 1 0.00113 0.00113
gnus-score-find-bnews 1 0.000966 0.000966
gnus-summary-auto-select-subject 1 0.000953 0.000953
gnus-summary-first-unread-subject 1 0.000935 0.000935
gnus-summary-first-subject 1 0.000866 0.000866
gnus-group-insert-group-line-info 1 0.000776 0.000776
gnus-group-insert-group-line 1 0.000737 0.000737
gnus-parse-complex-format 3 0.002126 0.0007086666
gnus-run-mode-hooks 1 0.000663 0.000663
gnus-killed-articles 1 0.000663 0.000663
gnus-agent-mode 1 0.000642 0.000642
gnus-parse-format 4 0.002465 0.00061625
gnus-compute-unseen-list 1 0.000613 0.000613
gnus-inverse-list-range-intersection 1 0.000597 0.000597
gnus-list-range-difference 1 0.000581 0.000581
gnus-score-score-files 1 0.000473 0.000473
gnus-make-sort-function 1 0.000456 0.000456
gnus-message 10 0.004097 0.0004097
gnus-request-group 1 0.000393 0.000393
gnus-configure-windows 1 0.000392 0.000392
gnus-complex-form-to-spec 6 0.002298 0.0003830000
gnus-summary-setup-default-charset 1 0.00034 0.00034
gnus-uncompress-range 1 0.000329 0.000329
gnus-update-read-articles 1 0.000307 0.000307
gnus-cache-articles-in-group 2 0.00061 0.000305
gnus-sorted-intersection 1 0.000231 0.000231
gnus-sorted-difference 2 0.0004599999 0.0002299999
gnus-cache-file-name 2 0.00045 0.000225
gnus-apply-kill-file 1 0.000222 0.000222
gnus-group-highlight-line 1 0.000203 0.000203
gnus-parameter-charset 1 0.000202 0.000202
gnus-adjust-marked-articles 1 0.000195 0.000195
gnus-prin1-to-string 1 0.000191 0.000191
gnus-agent-find-parameter 1 0.000185 0.000185
gnus-score-headers 1 0.000181 0.000181
gnus-score-load-files 1 0.00016 0.00016
gnus-all-windows-visible-p 1 0.000155 0.000155
gnus-agent-summary-make-menu-bar 1 0.000145 0.000145
gnus-parse-simple-format 10 0.0014309999 0.0001430999
gnus-score-load-file 1 0.000142 0.000142
gnus-continuum-version 4 0.000567 0.00014175
gnus-summary-from-or-to-or-newsgroups 382 0.0528690000 0.0001384005
gnus-summary-menu-split 1 0.000121 0.000121
gnus-get-newsgroup-headers 1 0.000109 0.000109
gnus-group-decoded-name 8 0.000858 0.00010725
gnus-get-buffer-window 1 0.000101 0.000101
gnus-summary-buffer-name 2 0.000201 0.0001005
gnus-parameter-ignored-charsets 1 0.0001 0.0001
gnus-run-hooks 393 0.0386120000 9.824...e-05
gnus-sort-score-files 1 9.8e-05 9.8e-05
gnus-summary-make-local-variables 2 0.000196 9.8e-05
gnus-configure-frame 2 0.00019 9.5e-05
gnus-summary-set-display-table 1 9.4e-05 9.4e-05
gnus-group-auto-expirable-p 1 8.6e-05 8.6e-05
gnus-summary-highlight-line 382 0.0313219999 8.199...e-05
gnus-summary-set-local-parameters 1 7.9e-05 7.9e-05
gnus-score-file-name 1 7.7e-05 7.7e-05
gnus-check-server 1 7.6e-05 7.6e-05
gnus-extract-address-components 382 0.0283220000 7.414...e-05
gnus-group-find-parameter 9 0.0006649999 7.388...e-05
gnus-get-unread-articles-in-group 1 7e-05 7e-05
gnus-set-global-variables 3 0.000205 6.833...e-05
gnus-group-category 1 6e-05 6e-05
gnus-group-timestamp-delta 1 6e-05 6e-05
gnus-server-opened 1 5.8e-05 5.8e-05
gnus-last-element 1 5.7e-05 5.7e-05
gnus-newsgroup-kill-file 2 0.0001089999 5.449...e-05
gnus-agent-get-undownloaded-list 1 5.2e-05 5.2e-05
gnus-score-load-score-alist 1 5.1e-05 5.1e-05
gnus-short-group-name 1 4.9e-05 4.9e-05
gnus-agent-possibly-alter-active 2 9.5e-05 4.75e-05
gnus-frames-on-display-list 1 4.7e-05 4.7e-05
gnus-remove-from-range 2 9.2e-05 4.6e-05
gnus-group-fast-parameter 7 0.000316 4.514...e-05
gnus-ephemeral-group-p 2 8.999...e-05 4.499...e-05
gnus-summary-position-point 3 0.000132 4.400...e-05
gnus-group-quit-config 1 4.3e-05 4.3e-05
gnus-tilde-max-form 2 8.5e-05 4.25e-05
gnus-make-hashtable 2 7.8e-05 3.9e-05
gnus-group-name-charset 9 0.0003380000 3.755...e-05
gnus-correct-substring 52 0.0018910000 3.636...e-05
gnus-get-function 2 7.1e-05 3.55e-05
gnus-group-name-decode 10 0.000348 3.48e-05
gnus-define-keys-1 1 3.2e-05 3.2e-05
gnus-simplify-mode-line 1 3.1e-05 3.1e-05
gnus-goto-colon 3 8.9e-05 2.966...e-05
gnus-turn-off-edit-menu 1 2.8e-05 2.8e-05
gnus-virtual-group-p 1 2.8e-05 2.8e-05
gnus-undo-register 1 2.5e-05 2.5e-05
gnus-pad-form 2 4.499...e-05 2.249...e-05
gnus-put-text-property-excluding-characters-with-faces 383 0.0073299999 1.913...e-05
gnus-group-group-name 2 3.7e-05 1.85e-05
gnus-emacs-version 3 5.4e-05 1.8e-05
gnus-newsgroup-savable-name 2 3.3e-05 1.65e-05
gnus-summary-highlight-line-0 382 0.0062829999 1.644...e-05
gnus-list-of-unread-articles 1 1.6e-05 1.6e-05
gnus-get-buffer-create 3 4.6e-05 1.533...e-05
gnus-group-get-parameter 14 0.000203 1.45e-05
gnus-set-work-buffer 15 0.0002080000 1.386...e-05
gnus-create-hash-size 2 2.600...e-05 1.300...e-05
gnus-make-local-hook 1 1.3e-05 1.3e-05
gnus-copy-sequence 5 6.5e-05 1.3e-05
gnus-summary-limit-children 413 0.0050600000 1.225...e-05
gnus-server-add-address 1 1.1e-05 1.1e-05
gnus-score-file-regexp 2 1.999...e-05 9.999...e-06
gnus-put-text-property 765 0.0071449999 9.339...e-06
gnus-parameters-get-parameter 2 1.8e-05 9e-06
gnus-find-method-for-group 20 0.00018 9e-06
gnus-undo-register-1 1 9e-06 9e-06
gnus-add-text-properties 1 9e-06 9e-06
gnus-group-parameter-value 13 0.00011 8.461...e-06
gnus-score-find-alist 1 8e-06 8e-06
gnus-mouse-face-function 3 2.4e-05 8e-06
gnus-make-sort-function-1 1 8e-06 8e-06
gnus-sorted-nintersection 1 8e-06 8e-06
gnus-server-status 1 8e-06 8e-06
gnus-agent-method-p 5 3.8e-05 7.6e-06
gnus-buffer-live-p 12 9e-05 7.5e-06
gnus-home-score-file 2 1.499...e-05 7.499...e-06
gnus-visual-p 385 0.0027019999 7.018...e-06
gnus-windows-old-to-new 1 7e-06 7e-06
gnus-undo-boundary 1 7e-06 7e-06
gnus-sorted-ndifference 2 1.4e-05 7e-06
gnus-group-group-indentation 2 1.4e-05 7e-06
gnus-online 6 4.1e-05 6.833...e-06
gnus-use-long-file-name 5 3.200...e-05 6.400...e-06
gnus-summary-make-tool-bar 1 6e-06 6e-06
gnus-remove-if 1 6e-06 6e-06
gnus-window-to-buffer-helper 2 1.1e-05 5.5e-06
gnus-article-sort-by-number 1709 0.0086089999 5.037...e-06
gnus-extent-start-open 1 5e-06 5e-06
gnus-set-default-directory 1 5e-06 5e-06
gnus-group-remove-excess-properties 1 5e-06 5e-06
gnus-summary-maybe-hide-threads 1 5e-06 5e-06
gnus-article-mark-to-type 18 8.700...e-05 4.833...e-06
gnus-range-length 3 1.400...e-05 4.666...e-06
gnus-string-width-function 4 1.8e-05 4.5e-06
gnus-substring-function 2 9e-06 4.5e-06
gnus-make-score-map 2 8e-06 4e-06
[-- Attachment #3: Type: text/plain, Size: 67 bytes --]
---- Profile emacs 21.4.1 ----
---- Profile emacs 22.0.50.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: prof22 --]
[-- Type: text/x-patch, Size: 15846 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 5.821693 5.821693
gnus-group-read-group 1 5.8216660000 5.8216660000
gnus-summary-read-group 1 5.821563 5.821563
gnus-summary-read-group-1 1 5.821543 5.821543
gnus-select-newsgroup 1 4.925118 4.925118
gnus-fetch-headers 1 4.918934 4.918934
gnus-get-newsgroup-headers-xover 1 3.211388 3.211388
gnus-cache-retrieve-headers 2 4.7366139999 2.3683069999
gnus-retrieve-headers 4 9.4725609999 2.3681402499
gnus-summary-prepare 1 0.757084 0.757084
gnus-summary-prepare-threads 1 0.73417 0.73417
gnus-summary-prepare-unthreaded 1 0.733227 0.733227
gnus-summary-setup-buffer 1 0.1152720000 0.1152720000
gnus-summary-mode 1 0.1148280000 0.1148280000
gnus-sort-articles 1 0.0227979999 0.0227979999
gnus-summary-make-menu-bar 1 0.012817 0.012817
gnus-summary-initial-limit 1 0.006404 0.006404
gnus-group-update-group 1 0.006002 0.006002
gnus-update-summary-mark-positions 2 0.007674 0.003837
gnus-build-sparse-threads 1 0.003148 0.003148
gnus-byte-compile 2 0.006036 0.003018
gnus-set-mode-line 1 0.002757 0.002757
gnus-dribble-enter 1 0.002502 0.002502
gnus-group-set-mode-line 2 0.0050019999 0.0025009999
gnus-mode-line-buffer-identification 3 0.007416 0.002472
gnus-summary-insert-line 382 0.7311690000 0.0019140549
gnus-update-missing-marks 1 0.001727 0.001727
gnus-update-format-specifications 2 0.003026 0.001513
gnus-possibly-score-headers 1 0.001275 0.001275
gnus-user-format-function-b 380 0.4646039999 0.0012226421
gnus-all-score-files 1 0.001093 0.001093
gnus-articles-to-read 1 0.001033 0.001033
gnus-score-find-bnews 1 0.000854 0.000854
gnus-make-sort-function 1 0.000811 0.000811
gnus-summary-auto-select-subject 1 0.000742 0.000742
gnus-summary-first-unread-subject 1 0.000728 0.000728
gnus-make-thread-indent-array 1 0.00068 0.00068
gnus-summary-first-subject 1 0.000676 0.000676
gnus-group-insert-group-line-info 1 0.000651 0.000651
gnus-group-insert-group-line 1 0.00062 0.00062
gnus-parse-complex-format 3 0.0018419999 0.000614
gnus-killed-articles 1 0.000596 0.000596
gnus-parse-format 4 0.002153 0.00053825
gnus-request-group 1 0.000466 0.000466
gnus-compute-unseen-list 1 0.000425 0.000425
gnus-score-score-files 1 0.000417 0.000417
gnus-message 10 0.004162 0.0004162
gnus-inverse-list-range-intersection 1 0.000411 0.000411
gnus-list-range-difference 1 0.000398 0.000398
gnus-run-mode-hooks 1 0.000357 0.000357
gnus-cache-articles-in-group 2 0.000681 0.0003405
gnus-summary-setup-default-charset 1 0.000338 0.000338
gnus-uncompress-range 1 0.000318 0.000318
gnus-complex-form-to-spec 6 0.0018869999 0.0003144999
gnus-update-read-articles 1 0.000301 0.000301
gnus-configure-windows 1 0.000278 0.000278
gnus-agent-mode 1 0.000271 0.000271
gnus-cache-file-name 2 0.0005239999 0.0002619999
gnus-run-hooks 393 0.0961059999 0.0002445445
gnus-summary-highlight-line 382 0.0899899999 0.0002355759
gnus-apply-kill-file 1 0.000215 0.000215
gnus-sorted-difference 2 0.00041 0.000205
gnus-parameter-charset 1 0.000197 0.000197
gnus-adjust-marked-articles 1 0.000192 0.000192
gnus-sorted-intersection 1 0.000188 0.000188
gnus-summary-highlight-line-0 382 0.0713300000 0.0001867277
gnus-agent-find-parameter 1 0.000184 0.000184
gnus-score-headers 1 0.000158 0.000158
gnus-group-highlight-line 1 0.000151 0.000151
gnus-summary-menu-split 1 0.000147 0.000147
gnus-score-load-files 1 0.000143 0.000143
gnus-group-decoded-name 8 0.001142 0.00014275
gnus-prin1-to-string 1 0.000138 0.000138
gnus-score-load-file 1 0.000129 0.000129
gnus-all-windows-visible-p 1 0.000122 0.000122
gnus-continuum-version 4 0.00048 0.00012
gnus-parse-simple-format 10 0.001157 0.0001157000
gnus-summary-buffer-name 2 0.00023 0.000115
gnus-agent-summary-make-menu-bar 1 0.000113 0.000113
gnus-parameter-ignored-charsets 1 0.000102 0.000102
gnus-sort-score-files 1 9.8e-05 9.8e-05
gnus-summary-make-local-variables 2 0.000182 9.1e-05
gnus-summary-from-or-to-or-newsgroups 382 0.0329829999 8.634...e-05
gnus-get-newsgroup-headers 1 8.6e-05 8.6e-05
gnus-get-buffer-window 1 8.5e-05 8.5e-05
gnus-summary-set-local-parameters 1 8.5e-05 8.5e-05
gnus-summary-set-display-table 1 8.4e-05 8.4e-05
gnus-score-file-name 1 7.5e-05 7.5e-05
gnus-group-auto-expirable-p 1 7.5e-05 7.5e-05
gnus-check-server 1 7.4e-05 7.4e-05
gnus-group-name-decode 10 0.000737 7.37e-05
gnus-configure-frame 2 0.000145 7.25e-05
gnus-group-find-parameter 9 0.0006470000 7.188...e-05
gnus-get-unread-articles-in-group 1 6.8e-05 6.8e-05
gnus-newsgroup-kill-file 2 0.000127 6.35e-05
gnus-group-category 1 6.1e-05 6.1e-05
gnus-set-global-variables 3 0.0001729999 5.766...e-05
gnus-server-opened 1 5.6e-05 5.6e-05
gnus-extract-address-components 382 0.0178539999 4.673...e-05
gnus-remove-from-range 2 9.3e-05 4.65e-05
gnus-last-element 1 4.6e-05 4.6e-05
gnus-score-load-score-alist 1 4.6e-05 4.6e-05
gnus-agent-possibly-alter-active 2 9.1e-05 4.55e-05
gnus-group-timestamp-delta 1 4.5e-05 4.5e-05
gnus-group-quit-config 1 4.3e-05 4.3e-05
gnus-group-fast-parameter 7 0.0002929999 4.185...e-05
gnus-agent-get-undownloaded-list 1 4.1e-05 4.1e-05
gnus-tilde-max-form 2 7.9e-05 3.95e-05
gnus-make-hashtable 2 7.500...e-05 3.750...e-05
gnus-frames-on-display-list 1 3.7e-05 3.7e-05
gnus-get-function 2 7.2e-05 3.6e-05
gnus-define-keys-1 1 3.6e-05 3.6e-05
gnus-group-name-charset 9 0.000321 3.566...e-05
gnus-summary-position-point 3 0.0001009999 3.366...e-05
gnus-turn-off-edit-menu 1 3.3e-05 3.3e-05
gnus-ephemeral-group-p 2 6.6e-05 3.3e-05
gnus-short-group-name 1 3.2e-05 3.2e-05
gnus-virtual-group-p 1 2.8e-05 2.8e-05
gnus-correct-substring 52 0.0012000000 2.307...e-05
gnus-pad-form 2 4.6e-05 2.3e-05
gnus-undo-register 1 2.3e-05 2.3e-05
gnus-goto-colon 3 6.9e-05 2.3e-05
gnus-simplify-mode-line 1 1.8e-05 1.8e-05
gnus-group-group-name 2 3.3e-05 1.65e-05
gnus-list-of-unread-articles 1 1.5e-05 1.5e-05
gnus-get-buffer-create 3 4.2e-05 1.4e-05
gnus-put-text-property-excluding-characters-with-faces 383 0.0051550000 1.345...e-05
gnus-copy-sequence 5 6.699...e-05 1.339...e-05
gnus-set-work-buffer 15 0.000197 1.313...e-05
gnus-newsgroup-savable-name 2 2.6e-05 1.3e-05
gnus-group-get-parameter 14 0.0001729999 1.235...e-05
gnus-server-add-address 1 1.2e-05 1.2e-05
gnus-emacs-version 3 3.5e-05 1.166...e-05
gnus-create-hash-size 2 2.100...e-05 1.050...e-05
gnus-parameters-get-parameter 2 1.9e-05 9.5e-06
gnus-summary-limit-children 413 0.0037529999 9.087...e-06
gnus-find-method-for-group 20 0.0001800000 9.000...e-06
gnus-group-parameter-value 13 0.0001059999 8.153...e-06
gnus-undo-register-1 1 8e-06 8e-06
gnus-add-text-properties 1 8e-06 8e-06
gnus-server-status 1 8e-06 8e-06
gnus-put-text-property 765 0.0057960000 7.576...e-06
gnus-score-file-regexp 2 1.499...e-05 7.499...e-06
gnus-mouse-face-function 3 2.2e-05 7.333...e-06
gnus-agent-method-p 5 3.6e-05 7.200...e-06
gnus-undo-boundary 1 7e-06 7e-06
gnus-make-sort-function-1 1 7e-06 7e-06
gnus-buffer-live-p 12 8.3e-05 6.916...e-06
gnus-online 6 4.100...e-05 6.833...e-06
gnus-windows-old-to-new 1 6e-06 6e-06
gnus-summary-make-tool-bar 1 6e-06 6e-06
gnus-score-find-alist 1 6e-06 6e-06
gnus-home-score-file 2 1.2e-05 6e-06
gnus-sorted-ndifference 2 1.2e-05 6e-06
gnus-sorted-nintersection 1 6e-06 6e-06
gnus-set-default-directory 1 6e-06 6e-06
gnus-remove-if 1 6e-06 6e-06
gnus-group-group-indentation 2 1.1e-05 5.5e-06
gnus-make-local-hook 1 5e-06 5e-06
gnus-use-long-file-name 5 2.499...e-05 4.999...e-06
gnus-range-length 3 1.499...e-05 4.999...e-06
gnus-string-width-function 4 1.9e-05 4.75e-06
gnus-article-mark-to-type 18 8.400...e-05 4.666...e-06
gnus-visual-p 385 0.0017600000 4.571...e-06
gnus-window-to-buffer-helper 2 9e-06 4.5e-06
gnus-make-score-map 2 9e-06 4.5e-06
gnus-article-sort-by-number 1709 0.0074619999 4.366...e-06
gnus-substring-function 2 8.000...e-06 4.000...e-06
gnus-extent-start-open 1 4e-06 4e-06
gnus-summary-maybe-hide-threads 1 4e-06 4e-06
gnus-group-remove-excess-properties 1 3e-06 3e-06
[-- Attachment #5: Type: text/plain, Size: 34 bytes --]
---- Profile emacs 22.0.50.1 ----
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 9:52 ` Elias Oltmanns
@ 2006-03-20 11:30 ` Daniel Pittman
2006-03-20 15:25 ` Elias Oltmanns
0 siblings, 1 reply; 11+ messages in thread
From: Daniel Pittman @ 2006-03-20 11:30 UTC (permalink / raw)
Elias Oltmanns <oltmanns@uni-bonn.de> writes:
> Daniel Pittman <daniel@rimspace.net> wrote:
>> Elias Oltmanns <oltmanns@uni-bonn.de> writes:
>>
>>> accessing a 25MB nnfolder takes at least twice as much time and
>>> sometimes even up to four times as much when running No Gnus from
>>> within emacs-snapshot (Debian packaged cvs version) than running in
>>> emacs 21.4. The reason I'm reporting this here is that visiting the
>>> file with C-x C-f directly from emacs does not reveal such a
>>> difference in speed. Yet, every operation performed by gnus on this
>>> nnfolder, as entering, reading articles, expiring, etc, challenges my
>>> patience, to say the least.
>>
>> Well, the best thing to do is probably to profile the operation of
>> visiting a folder, allowing us to see exactly where time is spent.
>
> Here are the profiles. I went about as follows:
> - Start up emacs2[12],
> - load elp.el,
> - M-x set-varialbe RET elp-sort-by-function RET elp-sort-by-average-time RET,
> - start up gnus,
> - M-x elp-instrument-package RET gnus RET,
You probably want to instrument 'nn*' here as well, to catch the
nnfolder, nnml, etc, backend implementation. Sorry, I should have
mentioned that, since it isn't obvious.
> - enter folder,
> - M-x elp-results.
>
> Note that there are no delays due to disk access as all data had been
> in cache already.
Nothing stands out to me as an obvious candidate, but perhaps someone
more familiar with the code can help out.
Daniel
--
Digital Infrastructure Solutions -- making IT simple, stable and secure
Phone: 0401 155 707 email: contact@digital-infrastructure.com.au
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 11:30 ` Daniel Pittman
@ 2006-03-20 15:25 ` Elias Oltmanns
2006-03-20 21:44 ` Romain Francoise
0 siblings, 1 reply; 11+ messages in thread
From: Elias Oltmanns @ 2006-03-20 15:25 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]
Daniel Pittman <daniel@rimspace.net> wrote:
> Elias Oltmanns <oltmanns@uni-bonn.de> writes:
>
>> Daniel Pittman <daniel@rimspace.net> wrote:
>>> Elias Oltmanns <oltmanns@uni-bonn.de> writes:
>>>
>>>> accessing a 25MB nnfolder takes at least twice as much time and
>>>> sometimes even up to four times as much when running No Gnus from
>>>> within emacs-snapshot (Debian packaged cvs version) than running in
>>>> emacs 21.4. The reason I'm reporting this here is that visiting the
>>>> file with C-x C-f directly from emacs does not reveal such a
>>>> difference in speed. Yet, every operation performed by gnus on this
>>>> nnfolder, as entering, reading articles, expiring, etc, challenges my
>>>> patience, to say the least.
>>>
>>> Well, the best thing to do is probably to profile the operation of
>>> visiting a folder, allowing us to see exactly where time is spent.
>>
>> Here are the profiles. I went about as follows:
>> - Start up emacs2[12],
>> - load elp.el,
>> - M-x set-varialbe RET elp-sort-by-function RET elp-sort-by-average-time RET,
>> - start up gnus,
>> - M-x elp-instrument-package RET gnus RET,
>
> You probably want to instrument 'nn*' here as well
Done now.
[...]
>
>> - enter folder,
>> - M-x elp-results.
>>
>> Note that there are no delays due to disk access as all data had been
>> in cache already.
New profiles attached. There is one striking thing:
nnfolder-existing-articles appears amongst the most time consuming
functions only in emacs 22 so there has to be some construct in the
function definition which behaves differently when executed in the two
emacs versions. So far I managed to rule out search-* and
re-search-*. Still, I have the feeling that it might be one of those
seeking commands, however, I didn't manage to profile goto-char which
was my next guess.
Any ideas?
Elias
---- Profile emacs 21.4.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: prof21 --]
[-- Type: text/x-patch, Size: 18418 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 3.6129189999 3.6129189999
gnus-group-read-group 1 3.612888 3.612888
gnus-summary-read-group 1 3.612775 3.612775
gnus-summary-read-group-1 1 3.612753 3.612753
gnus-select-newsgroup 1 2.515101 2.515101
gnus-fetch-headers 1 2.508358 2.508358
nnfolder-read-folder 1 1.900019 1.900019
nnheader-find-file-noselect 1 1.281949 1.281949
gnus-cache-retrieve-headers 2 2.264043 1.1320215
gnus-retrieve-headers 4 4.527462 1.1318655
nnfolder-retrieve-headers 2 2.263045 1.1315225
gnus-summary-prepare 1 1.0381070000 1.0381070000
gnus-summary-prepare-threads 1 0.94018 0.94018
gnus-summary-prepare-unthreaded 1 0.939085 0.939085
nnfolder-possibly-change-group 3 1.9010090000 0.6336696666
gnus-get-newsgroup-headers-xover 1 0.597657 0.597657
nnfolder-existing-articles 1 0.3538750000 0.3538750000
gnus-sort-articles 1 0.097786 0.097786
gnus-summary-setup-buffer 1 0.026488 0.026488
gnus-summary-mode 1 0.025155 0.025155
gnus-summary-make-menu-bar 1 0.011209 0.011209
gnus-summary-initial-limit 1 0.009112 0.009112
gnus-group-update-group 1 0.008212 0.008212
nnheader-insert-file-contents 1 0.00757 0.00757
gnus-set-mode-line 1 0.004829 0.004829
gnus-build-sparse-threads 1 0.00413 0.00413
nnfolder-retrieve-headers-with-nov 2 0.008041 0.0040205
gnus-mode-line-buffer-identification 3 0.01138 0.0037933333
gnus-dribble-enter 1 0.003509 0.003509
gnus-group-set-mode-line 2 0.0069359999 0.0034679999
gnus-update-summary-mark-positions 2 0.006251 0.0031255
gnus-summary-insert-line 382 0.9337769999 0.0024444424
gnus-byte-compile 2 0.004847 0.0024235
gnus-update-format-specifications 2 0.0034540000 0.0017270000
gnus-update-missing-marks 1 0.001635 0.001635
gnus-possibly-score-headers 1 0.001581 0.001581
gnus-user-format-function-b 380 0.5247200000 0.0013808421
gnus-all-score-files 1 0.001365 0.001365
gnus-articles-to-read 1 0.001164 0.001164
gnus-score-find-bnews 1 0.001062 0.001062
gnus-summary-auto-select-subject 1 0.000992 0.000992
nnfolder-save-active 1 0.000987 0.000987
gnus-summary-first-unread-subject 1 0.000974 0.000974
nnmail-save-active 1 0.000965 0.000965
gnus-summary-first-subject 1 0.000905 0.000905
gnus-group-insert-group-line-info 1 0.000796 0.000796
gnus-group-insert-group-line 1 0.000758 0.000758
gnus-parse-complex-format 3 0.0021520000 0.0007173333
gnus-killed-articles 1 0.000659 0.000659
gnus-parse-format 4 0.002493 0.00062325
gnus-compute-unseen-list 1 0.000609 0.000609
gnus-make-thread-indent-array 1 0.000598 0.000598
gnus-inverse-list-range-intersection 1 0.000593 0.000593
gnus-list-range-difference 1 0.000579 0.000579
gnus-request-group 1 0.00054 0.00054
gnus-score-score-files 1 0.000526 0.000526
gnus-message 10 0.005019 0.0005019
nnmail-generate-active 1 0.000496 0.000496
gnus-make-sort-function 1 0.000473 0.000473
nnfolder-request-group 1 0.000472 0.000472
gnus-complex-form-to-spec 6 0.002362 0.0003936666
gnus-configure-windows 1 0.000388 0.000388
gnus-uncompress-range 1 0.000368 0.000368
gnus-summary-setup-default-charset 1 0.000333 0.000333
gnus-cache-articles-in-group 2 0.000648 0.000324
gnus-summary-from-or-to-or-newsgroups 382 0.1211589999 0.0003171701
gnus-update-read-articles 1 0.00031 0.00031
gnus-run-mode-hooks 1 0.000254 0.000254
gnus-cache-file-name 2 0.00048 0.00024
gnus-agent-mode 1 0.000235 0.000235
gnus-sorted-intersection 1 0.00023 0.00023
gnus-sorted-difference 2 0.000453 0.0002265
gnus-apply-kill-file 1 0.000226 0.000226
gnus-group-decoded-name 8 0.0017839999 0.0002229999
gnus-group-highlight-line 1 0.000204 0.000204
gnus-parameter-charset 1 0.000198 0.000198
gnus-adjust-marked-articles 1 0.000197 0.000197
gnus-prin1-to-string 1 0.000192 0.000192
gnus-agent-find-parameter 1 0.000192 0.000192
gnus-score-headers 1 0.000187 0.000187
gnus-score-load-files 1 0.000167 0.000167
nnheader-nov-delete-outside-range 1 0.000167 0.000167
nnmail-activate 3 0.000486 0.000162
gnus-all-windows-visible-p 1 0.000153 0.000153
gnus-parse-simple-format 10 0.001456 0.0001456000
gnus-score-load-file 1 0.000144 0.000144
gnus-group-name-charset 9 0.0012869999 0.0001429999
gnus-continuum-version 4 0.0005629999 0.0001407499
nnmail-message-id 1 0.000138 0.000138
nnheader-fold-continuation-lines 1 0.000138 0.000138
nnmail-within-headers-p 378 0.0466509999 0.0001234153
nnheader-replace-regexp 1 0.000121 0.000121
gnus-check-server 1 0.00012 0.00012
gnus-summary-menu-split 1 0.00012 0.00012
gnus-server-opened 1 0.000102 0.000102
gnus-summary-buffer-name 2 0.000204 0.000102
gnus-sort-score-files 1 0.000101 0.000101
gnus-agent-summary-make-menu-bar 1 9.9e-05 9.9e-05
gnus-run-hooks 393 0.0386510000 9.834...e-05
gnus-get-buffer-window 1 9.8e-05 9.8e-05
gnus-parameter-ignored-charsets 1 9.7e-05 9.7e-05
gnus-get-newsgroup-headers 1 9.6e-05 9.6e-05
gnus-configure-frame 2 0.000192 9.6e-05
gnus-summary-make-local-variables 2 0.000187 9.35e-05
gnus-score-file-name 1 9.2e-05 9.2e-05
nnfolder-group-nov-pathname 2 0.000184 9.2e-05
gnus-group-auto-expirable-p 1 9e-05 9e-05
nnmail-search-unix-mail-delim-backward 378 0.033873 8.961...e-05
gnus-summary-set-display-table 1 8.5e-05 8.5e-05
gnus-summary-highlight-line 382 0.0313769999 8.213...e-05
gnus-get-unread-articles-in-group 1 8.2e-05 8.2e-05
gnus-summary-set-local-parameters 1 8.1e-05 8.1e-05
gnus-group-find-parameter 9 0.0006869999 7.633...e-05
gnus-extract-address-components 382 0.0273200000 7.151...e-05
nnfolder-group-pathname 6 0.000429 7.15e-05
gnus-group-category 1 6.9e-05 6.9e-05
nnheader-find-nov-line 2 0.000135 6.75e-05
gnus-set-global-variables 3 0.000199 6.633...e-05
gnus-group-timestamp-delta 1 6e-05 6e-05
gnus-last-element 1 5.7e-05 5.7e-05
gnus-newsgroup-kill-file 2 0.0001099999 5.499...e-05
gnus-score-load-score-alist 1 5.2e-05 5.2e-05
gnus-agent-get-undownloaded-list 1 5.1e-05 5.1e-05
nnheader-init-server-buffer 3 0.000146 4.866...e-05
gnus-agent-possibly-alter-active 2 9.5e-05 4.75e-05
gnus-group-fast-parameter 7 0.0003319999 4.742...e-05
gnus-frames-on-display-list 1 4.6e-05 4.6e-05
gnus-tilde-max-form 2 9.1e-05 4.55e-05
gnus-make-hashtable 2 9.1e-05 4.55e-05
gnus-short-group-name 1 4.5e-05 4.5e-05
gnus-remove-from-range 2 8.999...e-05 4.499...e-05
gnus-ephemeral-group-p 2 8.8e-05 4.4e-05
gnus-summary-position-point 3 0.00013 4.333...e-05
gnus-group-quit-config 1 4.2e-05 4.2e-05
nnfolder-server-opened 4 0.000158 3.95e-05
gnus-get-function 2 7.400...e-05 3.700...e-05
gnus-correct-substring 52 0.0018930000 3.640...e-05
gnus-group-name-decode 10 0.0003540000 3.540...e-05
gnus-goto-colon 3 8.8e-05 2.933...e-05
gnus-turn-off-edit-menu 1 2.9e-05 2.9e-05
gnus-virtual-group-p 1 2.7e-05 2.7e-05
nnheader-set-temp-buffer 1 2.7e-05 2.7e-05
gnus-define-keys-1 1 2.6e-05 2.6e-05
gnus-undo-register 1 2.4e-05 2.4e-05
gnus-pad-form 2 4.6e-05 2.3e-05
nnoo-server-opened 4 9.1e-05 2.275e-05
nnheader-report 1 2e-05 2e-05
gnus-get-buffer-create 3 5.8e-05 1.933...e-05
gnus-group-group-name 2 3.7e-05 1.85e-05
gnus-put-text-property-excluding-characters-with-faces 383 0.0069859999 1.824...e-05
gnus-emacs-version 3 5.3e-05 1.766...e-05
gnus-simplify-mode-line 1 1.7e-05 1.7e-05
gnus-newsgroup-savable-name 2 3.3e-05 1.65e-05
gnus-summary-highlight-line-0 382 0.0062679999 1.640...e-05
gnus-list-of-unread-articles 1 1.5e-05 1.5e-05
gnus-group-get-parameter 14 0.000202 1.442...e-05
gnus-set-work-buffer 15 0.000212 1.413...e-05
gnus-copy-sequence 5 6.4e-05 1.28e-05
gnus-server-add-address 1 1.2e-05 1.2e-05
gnus-make-local-hook 1 1.2e-05 1.2e-05
gnus-create-hash-size 2 2.4e-05 1.2e-05
gnus-summary-limit-children 413 0.0045820000 1.109...e-05
gnus-score-file-regexp 2 2.100...e-05 1.050...e-05
gnus-parameters-get-parameter 2 2.1e-05 1.05e-05
gnus-add-text-properties 1 1e-05 1e-05
gnus-group-parameter-value 13 0.000128 9.846...e-06
gnus-put-text-property 765 0.0071549999 9.352...e-06
gnus-make-sort-function-1 1 9e-06 9e-06
gnus-find-method-for-group 20 0.000174 8.7e-06
gnus-mouse-face-function 3 2.6e-05 8.666...e-06
gnus-score-find-alist 1 8e-06 8e-06
gnus-sorted-nintersection 1 8e-06 8e-06
gnus-undo-register-1 1 8e-06 8e-06
gnus-server-status 1 8e-06 8e-06
gnus-agent-method-p 5 3.8e-05 7.6e-06
gnus-buffer-live-p 12 9.099...e-05 7.583...e-06
gnus-group-group-indentation 2 1.5e-05 7.5e-06
gnus-online 6 4.3e-05 7.166...e-06
gnus-windows-old-to-new 1 7e-06 7e-06
gnus-summary-make-tool-bar 1 7e-06 7e-06
gnus-undo-boundary 2 1.300...e-05 6.500...e-06
gnus-home-score-file 2 1.300...e-05 6.500...e-06
gnus-sorted-ndifference 2 1.2e-05 6e-06
gnus-remove-if 1 6e-06 6e-06
nnheader-translate-file-chars 4 2.3e-05 5.75e-06
gnus-visual-p 385 0.0021539999 5.594...e-06
nnoo-current-server 4 2.1e-05 5.25e-06
gnus-window-to-buffer-helper 2 1e-05 5e-06
gnus-use-long-file-name 5 2.5e-05 5e-06
gnus-extent-start-open 1 5e-06 5e-06
gnus-set-default-directory 1 5e-06 5e-06
gnus-summary-maybe-hide-threads 1 5e-06 5e-06
gnus-string-width-function 4 1.999...e-05 4.999...e-06
gnus-range-length 3 1.499...e-05 4.999...e-06
gnus-article-mark-to-type 18 8.400...e-05 4.666...e-06
gnus-article-sort-by-number 1709 0.0079709999 4.664...e-06
gnus-substring-function 2 9e-06 4.5e-06
gnus-make-score-map 2 9e-06 4.5e-06
gnus-group-remove-excess-properties 1 4e-06 4e-06
[-- Attachment #3: Type: text/plain, Size: 66 bytes --]
---- Profile emacs 21.4.1 ----
---- Profile emacs 22.0.50.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: prof22 --]
[-- Type: text/x-patch, Size: 18377 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 6.020546 6.020546
gnus-group-read-group 1 6.020517 6.020517
gnus-summary-read-group 1 6.020412 6.020412
gnus-summary-read-group-1 1 6.020391 6.020391
gnus-select-newsgroup 1 5.191268 5.191268
gnus-fetch-headers 1 5.185388 5.185388
gnus-get-newsgroup-headers-xover 1 3.477925 3.477925
nnfolder-existing-articles 1 3.298889 3.298889
gnus-cache-retrieve-headers 2 5.006239 2.5031195
gnus-retrieve-headers 4 10.011773999 2.5029434999
nnfolder-retrieve-headers 2 5.005167 2.5025835
nnfolder-read-folder 1 1.698323 1.698323
nnheader-find-file-noselect 1 1.103362 1.103362
gnus-summary-prepare 1 0.775316 0.775316
gnus-summary-prepare-threads 1 0.752412 0.752412
gnus-summary-prepare-unthreaded 1 0.7509669999 0.7509669999
nnfolder-possibly-change-group 3 1.699391 0.5664636666
gnus-summary-setup-buffer 1 0.029343 0.029343
gnus-summary-mode 1 0.028891 0.028891
gnus-sort-articles 1 0.022787 0.022787
gnus-summary-make-menu-bar 1 0.013248 0.013248
nnheader-insert-file-contents 1 0.00645 0.00645
gnus-summary-initial-limit 1 0.006385 0.006385
gnus-group-update-group 1 0.005912 0.005912
gnus-build-sparse-threads 1 0.003637 0.003637
gnus-update-summary-mark-positions 2 0.007131 0.0035655
nnfolder-retrieve-headers-with-nov 2 0.006919 0.0034595
gnus-byte-compile 2 0.0060599999 0.0030299999
gnus-set-mode-line 1 0.002806 0.002806
gnus-dribble-enter 1 0.002494 0.002494
gnus-group-set-mode-line 2 0.0049219999 0.0024609999
gnus-mode-line-buffer-identification 3 0.00738 0.00246
gnus-summary-insert-line 382 0.7495970000 0.0019622958
gnus-update-format-specifications 2 0.002872 0.001436
gnus-user-format-function-b 380 0.5427229999 0.0014282184
gnus-possibly-score-headers 1 0.001293 0.001293
gnus-update-missing-marks 1 0.001231 0.001231
gnus-all-score-files 1 0.001111 0.001111
gnus-articles-to-read 1 0.001076 0.001076
nnfolder-save-active 1 0.0009 0.0009
nnmail-save-active 1 0.00088 0.00088
gnus-score-find-bnews 1 0.000872 0.000872
gnus-make-sort-function 1 0.00081 0.00081
gnus-summary-auto-select-subject 1 0.000748 0.000748
gnus-summary-first-unread-subject 1 0.000733 0.000733
gnus-make-thread-indent-array 1 0.000696 0.000696
gnus-summary-first-subject 1 0.00068 0.00068
gnus-group-insert-group-line-info 1 0.000636 0.000636
gnus-killed-articles 1 0.0006 0.0006
gnus-request-group 1 0.000599 0.000599
gnus-parse-complex-format 3 0.0017730000 0.000591
gnus-group-insert-group-line 1 0.00059 0.00059
nnfolder-request-group 1 0.000537 0.000537
gnus-parse-format 4 0.002077 0.00051925
gnus-message 10 0.0050100000 0.000501
gnus-compute-unseen-list 1 0.000423 0.000423
gnus-score-score-files 1 0.000419 0.000419
gnus-inverse-list-range-intersection 1 0.00041 0.00041
nnmail-generate-active 1 0.000406 0.000406
gnus-list-range-difference 1 0.000399 0.000399
gnus-run-mode-hooks 1 0.000377 0.000377
gnus-cache-articles-in-group 2 0.000731 0.0003655
gnus-uncompress-range 1 0.000352 0.000352
gnus-summary-setup-default-charset 1 0.000332 0.000332
nnmail-within-headers-p 378 0.120649 0.0003191772
gnus-complex-form-to-spec 6 0.0018409999 0.0003068333
gnus-update-read-articles 1 0.000299 0.000299
gnus-agent-mode 1 0.000292 0.000292
nnmail-search-unix-mail-delim-backward 378 0.1081300000 0.0002860582
gnus-cache-file-name 2 0.000564 0.000282
gnus-configure-windows 1 0.000269 0.000269
gnus-agent-find-parameter 1 0.000224 0.000224
gnus-apply-kill-file 1 0.000218 0.000218
gnus-adjust-marked-articles 1 0.000204 0.000204
gnus-sorted-difference 2 0.000398 0.000199
nnmail-activate 3 0.0005909999 0.000197
gnus-parameter-charset 1 0.000196 0.000196
gnus-sorted-intersection 1 0.000186 0.000186
nnheader-nov-delete-outside-range 1 0.00017 0.00017
gnus-score-headers 1 0.000159 0.000159
gnus-group-highlight-line 1 0.00015 0.00015
gnus-agent-summary-make-menu-bar 1 0.000147 0.000147
gnus-summary-menu-split 1 0.000147 0.000147
gnus-group-decoded-name 8 0.001153 0.000144125
gnus-score-load-files 1 0.000144 0.000144
gnus-prin1-to-string 1 0.000137 0.000137
nnmail-message-id 1 0.000136 0.000136
gnus-score-load-file 1 0.000129 0.000129
gnus-all-windows-visible-p 1 0.000121 0.000121
nnheader-fold-continuation-lines 1 0.000119 0.000119
gnus-summary-buffer-name 2 0.000236 0.000118
gnus-check-server 1 0.000116 0.000116
gnus-parse-simple-format 10 0.0011160000 0.0001116000
nnheader-replace-regexp 1 0.000105 0.000105
gnus-continuum-version 4 0.00041 0.0001025
gnus-group-category 1 9.9e-05 9.9e-05
gnus-server-opened 1 9.9e-05 9.9e-05
gnus-parameter-ignored-charsets 1 9.8e-05 9.8e-05
gnus-sort-score-files 1 9.7e-05 9.7e-05
gnus-summary-make-local-variables 2 0.000182 9.1e-05
gnus-summary-from-or-to-or-newsgroups 382 0.0346630000 9.074...e-05
gnus-score-file-name 1 8.7e-05 8.7e-05
gnus-run-hooks 393 0.0336490000 8.562...e-05
gnus-get-newsgroup-headers 1 8.5e-05 8.5e-05
nnfolder-group-nov-pathname 2 0.000169 8.45e-05
gnus-get-buffer-window 1 8.2e-05 8.2e-05
gnus-summary-set-local-parameters 1 8.2e-05 8.2e-05
gnus-summary-set-display-table 1 8.2e-05 8.2e-05
nnfolder-group-pathname 6 0.0004430000 7.383...e-05
gnus-group-auto-expirable-p 1 7.3e-05 7.3e-05
gnus-group-name-decode 10 0.000728 7.280...e-05
gnus-summary-highlight-line 382 0.0274000000 7.172...e-05
gnus-group-find-parameter 9 0.000645 7.166...e-05
nnheader-find-nov-line 2 0.0001419999 7.099...e-05
gnus-configure-frame 2 0.000137 6.85e-05
gnus-get-unread-articles-in-group 1 6.7e-05 6.7e-05
gnus-newsgroup-kill-file 2 0.000129 6.45e-05
nnheader-init-server-buffer 3 0.0001880000 6.266...e-05
gnus-set-global-variables 3 0.000154 5.133...e-05
gnus-extract-address-components 382 0.018218 4.769...e-05
gnus-last-element 1 4.6e-05 4.6e-05
gnus-group-timestamp-delta 1 4.6e-05 4.6e-05
gnus-make-hashtable 2 9.1e-05 4.55e-05
gnus-agent-possibly-alter-active 2 9.1e-05 4.55e-05
gnus-score-load-score-alist 1 4.5e-05 4.5e-05
gnus-remove-from-range 2 8.6e-05 4.3e-05
gnus-agent-get-undownloaded-list 1 4.2e-05 4.2e-05
gnus-group-fast-parameter 7 0.0002909999 4.157...e-05
gnus-group-quit-config 1 4.1e-05 4.1e-05
gnus-tilde-max-form 2 7.9e-05 3.95e-05
nnfolder-server-opened 4 0.000156 3.9e-05
gnus-get-function 2 7.099...e-05 3.549...e-05
gnus-frames-on-display-list 1 3.5e-05 3.5e-05
gnus-group-name-charset 9 0.0003140000 3.488...e-05
gnus-summary-position-point 3 0.0001030000 3.433...e-05
gnus-ephemeral-group-p 2 6.500...e-05 3.250...e-05
gnus-short-group-name 1 3.2e-05 3.2e-05
gnus-turn-off-edit-menu 1 3.2e-05 3.2e-05
gnus-virtual-group-p 1 2.7e-05 2.7e-05
gnus-define-keys-1 1 2.5e-05 2.5e-05
nnheader-set-temp-buffer 1 2.5e-05 2.5e-05
nnheader-report 1 2.5e-05 2.5e-05
gnus-undo-register 1 2.4e-05 2.4e-05
gnus-correct-substring 52 0.0012320000 2.369...e-05
gnus-goto-colon 3 7e-05 2.333...e-05
nnoo-server-opened 4 9.3e-05 2.325e-05
gnus-pad-form 2 4.200...e-05 2.100...e-05
gnus-simplify-mode-line 1 2e-05 2e-05
gnus-summary-highlight-line-0 382 0.0068589999 1.795...e-05
gnus-group-group-name 2 3.3e-05 1.65e-05
gnus-get-buffer-create 3 4.8e-05 1.6e-05
gnus-put-text-property-excluding-characters-with-faces 383 0.0054540000 1.424...e-05
gnus-list-of-unread-articles 1 1.4e-05 1.4e-05
gnus-set-work-buffer 15 0.0001960000 1.306...e-05
gnus-newsgroup-savable-name 2 2.499...e-05 1.249...e-05
gnus-copy-sequence 5 6.2e-05 1.24e-05
gnus-group-get-parameter 14 0.0001709999 1.221...e-05
gnus-emacs-version 3 3.599...e-05 1.199...e-05
gnus-create-hash-size 2 2.2e-05 1.1e-05
gnus-server-add-address 1 1e-05 1e-05
gnus-summary-limit-children 413 0.0037769999 9.145...e-06
gnus-server-status 1 9e-06 9e-06
gnus-parameters-get-parameter 2 1.7e-05 8.5e-06
gnus-find-method-for-group 20 0.0001660000 8.300...e-06
gnus-score-file-regexp 2 1.6e-05 8e-06
gnus-undo-register-1 1 8e-06 8e-06
gnus-group-parameter-value 13 0.000104 8e-06
gnus-put-text-property 765 0.0060460000 7.903...e-06
gnus-agent-method-p 5 3.8e-05 7.6e-06
gnus-summary-make-tool-bar 1 7e-06 7e-06
gnus-score-find-alist 1 7e-06 7e-06
gnus-mouse-face-function 3 2.1e-05 7e-06
gnus-make-sort-function-1 1 7e-06 7e-06
gnus-add-text-properties 1 7e-06 7e-06
gnus-online 6 4.1e-05 6.833...e-06
gnus-undo-boundary 2 1.300...e-05 6.500...e-06
gnus-buffer-live-p 12 7.7e-05 6.416...e-06
gnus-windows-old-to-new 1 6e-06 6e-06
gnus-home-score-file 2 1.2e-05 6e-06
gnus-sorted-nintersection 1 6e-06 6e-06
gnus-sorted-ndifference 2 1.1e-05 5.5e-06
gnus-group-group-indentation 2 1.1e-05 5.5e-06
nnheader-translate-file-chars 4 2.1e-05 5.25e-06
gnus-article-mark-to-type 18 9.200...e-05 5.111...e-06
gnus-make-local-hook 1 5e-06 5e-06
gnus-set-default-directory 1 5e-06 5e-06
gnus-remove-if 1 5e-06 5e-06
nnoo-current-server 4 2e-05 5e-06
gnus-use-long-file-name 5 2.499...e-05 4.999...e-06
gnus-visual-p 385 0.0018470000 4.797...e-06
gnus-range-length 3 1.399...e-05 4.666...e-06
gnus-string-width-function 4 1.8e-05 4.5e-06
gnus-make-score-map 2 9e-06 4.5e-06
gnus-article-sort-by-number 1709 0.0074669999 4.369...e-06
gnus-window-to-buffer-helper 2 8e-06 4e-06
gnus-substring-function 2 8e-06 4e-06
gnus-extent-start-open 1 4e-06 4e-06
gnus-group-remove-excess-properties 1 4e-06 4e-06
gnus-summary-maybe-hide-threads 1 4e-06 4e-06
[-- Attachment #5: Type: text/plain, Size: 34 bytes --]
---- Profile emacs 22.0.50.1 ----
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 15:25 ` Elias Oltmanns
@ 2006-03-20 21:44 ` Romain Francoise
2006-03-20 23:33 ` Elias Oltmanns
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Romain Francoise @ 2006-03-20 21:44 UTC (permalink / raw)
Elias Oltmanns <oltmanns@uni-bonn.de> writes:
> So far I managed to rule out search-* and re-search-*.
How?
> Still, I have the feeling that it might be one of those seeking
> commands, however, I didn't manage to profile goto-char which was my
> next guess.
Quite unlikely... `goto-char' can take its sweet time to move around in
very large buffers, but *three seconds in one call*?
I think the problem with this function is that we call `search-forward'
and `re-search-forward' alternatively for each message, which duplicates
all the work of setting up Boyer-Moore tables, and possibly just
thrashes search caches between each call.
I get a very significant performance boost with this simple patch which
just merges the two searches into one: on a 65M file, calling the
function now takes 10 seconds instead of 23 on my (slow) machine. Could
you try it out?
--- nnfolder.el 09 Feb 2006 08:33:47 +0100 7.16
+++ nnfolder.el 20 Mar 2006 22:39:42 +0100
@@ -405,12 +405,10 @@ the group. Then the marks file will be
(when nnfolder-current-buffer
(set-buffer nnfolder-current-buffer)
(goto-char (point-min))
- (let ((marker (concat "\n" nnfolder-article-marker))
- (number "[0-9]+")
- numbers)
- (while (and (search-forward marker nil t)
- (re-search-forward number nil t))
- (let ((newnum (string-to-number (match-string 0))))
+ (let ((head (concat "\n" nnfolder-article-marker "\\([0-9]+\\)"))
+ numbers)
+ (while (re-search-forward head nil t)
+ (let ((newnum (string-to-number (match-string 1))))
(if (nnmail-within-headers-p)
(push newnum numbers))))
;; The article numbers are increasing, so this result is sorted.
--
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
| ever free! --Bryan W. Procter
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 21:44 ` Romain Francoise
@ 2006-03-20 23:33 ` Elias Oltmanns
2006-03-21 12:29 ` Romain Francoise
2006-03-20 23:36 ` Miles Bader
2006-03-22 1:07 ` James Cloos
2 siblings, 1 reply; 11+ messages in thread
From: Elias Oltmanns @ 2006-03-20 23:33 UTC (permalink / raw)
[-- Attachment #1: Type: text/plain, Size: 1308 bytes --]
Romain Francoise <romain@orebokech.com> wrote:
> Elias Oltmanns <oltmanns@uni-bonn.de> writes:
>
>> So far I managed to rule out search-* and re-search-*.
>
> How?
Of course, I didn't! Accidentally watched the wrong counter ... In
fact, there is a very significant difference in performance.
[...]
> I think the problem with this function is that we call
> `search-forward' and `re-search-forward' alternatively for each
> message, which duplicates all the work of setting up Boyer-Moore
> tables, and possibly just thrashes search caches between each call.
>
> I get a very significant performance boost with this simple patch
> which just merges the two searches into one: on a 65M file, calling
> the function now takes 10 seconds instead of 23 on my (slow)
> machine. Could you try it out?
Very nice. Performance increased quite a bit. However, whereas this
change improves performance under emacs 22, it hardly affects
performance under emacs 21 at all (stripped versions of new profiles
attached). Besides, there seem to be similar problems in other
functions as there are still striking differences in my profiles (note
nnfolder-existing-articles and gnus-get-newsgroup-headers-xover in
particular) and expiration still is much slower in emacs 22.
Regards,
Elias
---- Profile emacs 21.4.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: prof21 --]
[-- Type: text/x-patch, Size: 2375 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 3.531766 3.531766
gnus-group-read-group 1 3.531742 3.531742
gnus-summary-read-group 1 3.531643 3.531643
gnus-summary-read-group-1 1 3.531621 3.531621
gnus-select-newsgroup 1 2.524066 2.524066
gnus-fetch-headers 1 2.515859 2.515859
nnfolder-read-folder 1 1.900141 1.900141
nnheader-find-file-noselect 1 1.280613 1.280613
gnus-cache-retrieve-headers 2 2.268588 1.134294
gnus-retrieve-headers 4 4.5365540000 1.1341385000
nnfolder-retrieve-headers 2 2.267596 1.133798
gnus-summary-prepare 1 0.953777 0.953777
gnus-summary-prepare-threads 1 0.855174 0.855174
gnus-summary-prepare-unthreaded 1 0.85414 0.85414
nnfolder-possibly-change-group 3 1.90113 0.63371
gnus-get-newsgroup-headers-xover 1 0.605513 0.605513
nnfolder-existing-articles 1 0.358229 0.358229
gnus-sort-articles 1 0.098472 0.098472
gnus-summary-setup-buffer 1 0.02565 0.02565
gnus-summary-mode 1 0.025266 0.025266
gnus-summary-make-menu-bar 1 0.011286 0.011286
gnus-summary-initial-limit 1 0.008198 0.008198
nnheader-insert-file-contents 1 0.007637 0.007637
gnus-group-update-group 1 0.006604 0.006604
[-- Attachment #3: Type: text/plain, Size: 67 bytes --]
---- Profile emacs 21.4.1 ----
---- Profile emacs 22.0.50.1 ----
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: prof22 --]
[-- Type: text/x-patch, Size: 2390 bytes --]
Function Name Call Count Elapsed Time Average Time
====================================================== ========== ============ ============
gnus-group-select-group 1 3.943411 3.943411
gnus-group-read-group 1 3.943388 3.943388
gnus-summary-read-group 1 3.943293 3.943293
gnus-summary-read-group-1 1 3.943274 3.943274
gnus-select-newsgroup 1 2.989176 2.989176
gnus-fetch-headers 1 2.983292 2.983292
nnfolder-read-folder 1 1.700333 1.700333
gnus-cache-retrieve-headers 2 2.803667 1.4018335
gnus-retrieve-headers 4 5.606614 1.4016535
nnfolder-retrieve-headers 2 2.8025830000 1.4012915000
gnus-get-newsgroup-headers-xover 1 1.273701 1.273701
nnheader-find-file-noselect 1 1.105102 1.105102
nnfolder-existing-articles 1 1.094185 1.094185
gnus-summary-prepare 1 0.816648 0.816648
gnus-summary-prepare-threads 1 0.793226 0.793226
gnus-summary-prepare-unthreaded 1 0.792235 0.792235
nnfolder-possibly-change-group 3 1.701399 0.567133
gnus-summary-setup-buffer 1 0.1114420000 0.1114420000
gnus-summary-mode 1 0.1109719999 0.1109719999
gnus-summary-make-menu-bar 1 0.0953239999 0.0953239999
gnus-sort-articles 1 0.0233 0.0233
gnus-group-update-group 1 0.006611 0.006611
nnheader-insert-file-contents 1 0.006487 0.006487
gnus-summary-initial-limit 1 0.006373 0.006373
[-- Attachment #5: Type: text/plain, Size: 34 bytes --]
---- Profile emacs 22.0.50.1 ----
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 23:33 ` Elias Oltmanns
@ 2006-03-21 12:29 ` Romain Francoise
0 siblings, 0 replies; 11+ messages in thread
From: Romain Francoise @ 2006-03-21 12:29 UTC (permalink / raw)
Elias Oltmanns <oltmanns@uni-bonn.de> writes:
> Of course, I didn't! Accidentally watched the wrong counter ... In
> fact, there is a very significant difference in performance.
Yeah, so it all boils down to the fact that regexp search is slower in
Emacs 22, and since the nnfolder code keeps moving back and forth in the
file using searches, it's painfully obvious.
I seem to remember someone mentioning that problem on -pretest or -devel
a few weeks back, but the details are fuzzy. I'll ask around.
--
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
| ever free! --Bryan W. Procter
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 21:44 ` Romain Francoise
2006-03-20 23:33 ` Elias Oltmanns
@ 2006-03-20 23:36 ` Miles Bader
2006-03-21 12:13 ` Romain Francoise
2006-03-22 1:07 ` James Cloos
2 siblings, 1 reply; 11+ messages in thread
From: Miles Bader @ 2006-03-20 23:36 UTC (permalink / raw)
Romain Francoise <romain@orebokech.com> writes:
> - (while (and (search-forward marker nil t)
> - (re-search-forward number nil t))
> - (let ((newnum (string-to-number (match-string 0))))
> + (let ((head (concat "\n" nnfolder-article-marker "\\([0-9]+\\)"))
> + numbers)
> + (while (re-search-forward head nil t)
> + (let ((newnum (string-to-number (match-string 1))))
The replacement doesn't seem to do exactly the same thing: in the
original code, it starts searching for [0-9]+ after finding the article
marker, but there can be intervening characters (including newlines).
[I dunno whether the format of the means that the two should actually be
immediately adjacent,]
Also you probably want to regexp-quote nnfolder-article-marker.
-Miles
--
`Suppose Korea goes to the World Cup final against Japan and wins,' Moon said.
`All the past could be forgiven.' [NYT]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 23:36 ` Miles Bader
@ 2006-03-21 12:13 ` Romain Francoise
0 siblings, 0 replies; 11+ messages in thread
From: Romain Francoise @ 2006-03-21 12:13 UTC (permalink / raw)
Miles Bader <miles@gnu.org> writes:
> The replacement doesn't seem to do exactly the same thing: [...]
I know, but I think it's correct nonetheless; what we're looking for
here is a header inserted by Gnus in the mbox file, it looks like this:
X-Gnus-Article-Number: 6905 Mon, 20 Mar 2006 22:02:58 +0100
and is inserted like this:
(defun nnfolder-insert-newsgroup-line (group-art)
(save-excursion
(goto-char (point-min))
(unless (search-forward "\n\n" nil t)
(goto-char (point-max))
(insert "\n"))
(forward-char -1)
(insert (format (concat nnfolder-article-marker "%d %s\n")
(cdr group-art) (message-make-date)))))
So unless there is a historical reason why we need to skip over stuff
between nnfolder-article-marker and the article number, the new code is
correct.
> Also you probably want to regexp-quote nnfolder-article-marker.
Ah yes I had overlooked that, thanks. Although in that case it wouldn't
change anything (the marker is a plain string and it's not
customizable).
--
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
| ever free! --Bryan W. Procter
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders
2006-03-20 21:44 ` Romain Francoise
2006-03-20 23:33 ` Elias Oltmanns
2006-03-20 23:36 ` Miles Bader
@ 2006-03-22 1:07 ` James Cloos
2 siblings, 0 replies; 11+ messages in thread
From: James Cloos @ 2006-03-22 1:07 UTC (permalink / raw)
I see a similar slowdown with HEAD and nnmh folders.
Among other things, when starting up gnus HEAD reads through the
folders twice, stat(2)ing each message. Thus doubling the startup
time into the 2 to 4 hour range. (Yes, there are *lots* of folders.)
I've posted profiling data before. It hasn't changed significantly.
-JimC
--
James H. Cloos, Jr. <cloos@jhcloos.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2006-03-22 1:07 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-03-20 0:23 No Gnus v0.4 / emacs-cvs terribly slowly operating on large nnfolders Elias Oltmanns
2006-03-20 6:02 ` Daniel Pittman
2006-03-20 9:52 ` Elias Oltmanns
2006-03-20 11:30 ` Daniel Pittman
2006-03-20 15:25 ` Elias Oltmanns
2006-03-20 21:44 ` Romain Francoise
2006-03-20 23:33 ` Elias Oltmanns
2006-03-21 12:29 ` Romain Francoise
2006-03-20 23:36 ` Miles Bader
2006-03-21 12:13 ` Romain Francoise
2006-03-22 1:07 ` James Cloos
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).