Slow reading, gauge?

Slow reading, gauge?

[David Hedbor]

Sometimes displaying articles take a long time. This is on a PII-450
so it should be fast. Now I am sure this is because some feature I'm
using, but I don't know which one. Is there a way in emacs to gauge,
or profile, an action so that I can see which functions takes time?
-- 
[ Below is a random fortune, which is unrelated to the above message. ]
I am not afraid of tomorrow, for I have seen yesterday and I love today.
		-- William Allen White

Re: Slow reading, gauge?

[Kai.Grossjohann]

David Hedbor <david@hedbor.org> writes:

  > Sometimes displaying articles take a long time. This is on a PII-450
  > so it should be fast. Now I am sure this is because some feature I'm
  > using, but I don't know which one. Is there a way in emacs to gauge,
  > or profile, an action so that I can see which functions takes time?

Hm.  Does it happen with specific messages?  Are the messages
text/enriched or text/html?

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: Slow reading, gauge?

[David Hedbor]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:

> David Hedbor <david@hedbor.org> writes:
> 
>   > Sometimes displaying articles take a long time. This is on a PII-450
>   > so it should be fast. Now I am sure this is because some feature I'm
>   > using, but I don't know which one. Is there a way in emacs to gauge,
>   > or profile, an action so that I can see which functions takes time?
> 
> Hm.  Does it happen with specific messages?  Are the messages
> text/enriched or text/html?

After checking it some more it seems like it happens when I read
messages from people that aren't in the bbdb. Wonder why. I have the
following in my bbdb-notice-hook:

(defun bbdb-add-gnus-group (record)
  "Put the current gnus group in this record's posted-to field if it's not there"
  (or bbdb-readonly-p
      (let* ((group gnus-newsgroup-name)
	     (field (bbdb-record-getprop record 'posted-to))
	     (pos (string-match ":" group));; we don't want the nnm[hl]: part
	     )
	(setq group (substring group (cond
				      (pos (1+ pos))
				      (t 0))));; we didn't match on ":"
	(if (or
	     (not field);; the field isn't there yet.
	     (not (string-match (concat (regexp-quote group)
					"\\(\n\\|$\\)") field)))
	    (bbdb-annotate-notes record group 'posted-to)))))

(setq bbdb-auto-notes-alist 
      (list
       '("Organization" ("\\s-*\\(.*\\)\\s-*" company 1 t))
       '("Organisation" ("\\s-*\\(.*\\)\\s-*" company 1 t))
       '("X-Organization" ("\\s-*\\(.*\\)\\s-*" company 1 t))
       '("X-Organisation" ("\\s-*\\(.*\\)\\s-*" company 1 t))
       '("Web" ("\\s-*\\(.*\\)\\s-*" url 1))
       '("X-Web" ("\\s-*\\(.*\\)\\s-*" url 1))
       '("X-WWW-Homepage" ("\\s-*\\(.*\\)\\s-*" url 1))
       '("X-WWW-page" ("\\s-*\\(.*\\)\\s-*" url 1))
       '("X-Homepage" ("\\s-*\\(.*\\)\\s-*" url 0))
       ;; This is what Netscape puts in when sending a URL reference
       '("X-Url" ("\\s-*\\(.*\\)\\s-*" last-url 1))
       '("X-Mailer" ("\\s-*\\(.*\\)\\s-*" mailer 1 t))
       '("X-Newsreader" ("\\s-*\\(.*\\)\\s-*" mailer 1 t))
       '("X-Mua" ("\\s-*\\(.*\\)\\s-*" mailer 1 t))
       ))
(add-hook 'bbdb-notice-hook 'bbdb-auto-notes-hook)
(add-hook 'bbdb-notice-hook 'bbdb-add-gnus-group)

The code originally comes from this mailing list I believe. I was
never that slow before. Also the above is run even if the record isn't 
new. Might it be that adding all new entries to the database has
become slow due to the size of the database (14450 entries, 2.6MB)?

-- 
[ Below is a random fortune, which is unrelated to the above message. ]
When the blind lead the blind they will both fall over the cliff.
		-- Chinese proverb

Re: Slow reading, gauge?

[Colin Rafferty]

David Hedbor writes:
> Kai.Grossjohann@CS.Uni-Dortmund.DE writes:
>> David Hedbor <david@hedbor.org> writes:

>> > Sometimes displaying articles take a long time. This is on a PII-450
>> > so it should be fast. Now I am sure this is because some feature I'm
>> > using, but I don't know which one. Is there a way in emacs to gauge,
>> > or profile, an action so that I can see which functions takes time?
>> 
>> Hm.  Does it happen with specific messages?  Are the messages
>> text/enriched or text/html?

> After checking it some more it seems like it happens when I read
> messages from people that aren't in the bbdb. Wonder why. I have the
> following in my bbdb-notice-hook:

> [code elided]

> The code originally comes from this mailing list I believe. I was
> never that slow before. Also the above is run even if the record isn't 
> new. 

The code looks perfectly normal.

> Might it be that adding all new entries to the database has become
> slow due to the size of the database (14450 entries, 2.6MB)?

It might be.

If you do a `bbdb-create', does it feel exceptionally slow after your
final RET?

You have a very large .bbdb.

-- 
Colin

Re: Slow reading, gauge?

[tvaughan]

David Hedbor <david@hedbor.org> writes:

> Sometimes displaying articles take a long time. This is on a PII-450
> so it should be fast. Now I am sure this is because some feature I'm
> using, but I don't know which one. Is there a way in emacs to gauge,
> or profile, an action so that I can see which functions takes time?

What I've just come across is that it now takes the summary buffer a looong
time to display after I've selected an nnml mail group (these are the only
kind of groups I have). Though this is only with xemacs 21.0.67, and not
20.4. I too would like to know how to "profile an action". This problem
seems to have appeared after I did some futzing with my filters. I've put
my filters back to how they were, but no change.

Thanks,
Tom

-- 
Tom Vaughan <tvaughan at aventail dot com>

Re: Slow reading, gauge?

[David Hedbor]

Colin Rafferty <colin@xemacs.org> writes:

> It might be.
> 
> If you do a `bbdb-create', does it feel exceptionally slow after your
> final RET?

Yes. That is exactly what happens when I view a new record. 
 
> You have a very large .bbdb.

Indeed. I automatically entries from all I mails and news messages I
read.  

-- 
[ Below is a random fortune, which is unrelated to the above message. ]
This login session: $13.76, but for you $11.88.

Re: Slow reading, gauge?

[François Pinard]

David Hedbor <david@hedbor.org> writes:

> Colin Rafferty <colin@xemacs.org> writes:

> > It might be.

> > If you do a `bbdb-create', does it feel exceptionally slow after your
> > final RET?

> Yes. That is exactly what happens when I view a new record. 

> > You have a very large .bbdb.

> Indeed.  I automatically entries from all I mails and news messages I
> read.

I'm another poor soul who once felt in the BBDB trap! :-) My problem was
that I came to much depend on BBDB, and getting rid of it while keeping
needed information has been a long and painful undertaking.

A few friends told great good of write-only memory devices, and I thought
BBDB was a excellent opportunity for experimenting the principle.  I did
my experiment and learned my lesson! :-) Part of the problem is that all
is nice and dancing in the beginning.  It might take years before you
fully realize the extent of the nightmare!

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard

Re: Slow reading, gauge?

[Hrvoje Niksic]

François Pinard <pinard@iro.umontreal.ca> writes:

> I'm another poor soul who once felt in the BBDB trap! :-) My problem
> was that I came to much depend on BBDB, and getting rid of it while
> keeping needed information has been a long and painful undertaking.
> 
> A few friends told great good of write-only memory devices, and I
> thought BBDB was a excellent opportunity for experimenting the
> principle.  I did my experiment and learned my lesson! :-) Part of
> the problem is that all is nice and dancing in the beginning.  It
> might take years before you fully realize the extent of the
> nightmare!

I've never truly understood why you are warning everyone in sight
against BBDB?  What *is* the full extent of the nightmare, after all?

Re: Slow reading, gauge?

[Kai.Grossjohann]

David Hedbor <david@hedbor.org> writes:

  > Colin Rafferty <colin@xemacs.org> writes:
  > 
  > > You have a very large .bbdb.
  > 
  > Indeed. I automatically entries from all I mails and news messages I
  > read.  

I once cleaned up my .bbdb, set all the automatically-add things to
nil, now add people explicitly with `:', and I'm a happy camper.  My
.bbdb is 70k and contains 500 entries.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: Slow reading, gauge?

[Colin Rafferty]

David Hedbor writes:
> Colin Rafferty <colin@xemacs.org> writes:

>> It might be.
>> 
>> If you do a `bbdb-create', does it feel exceptionally slow after your
>> final RET?

> Yes. That is exactly what happens when I view a new record. 

Open a 2.6 mB file, go to somewhere in the middle of it, and type a
character.  See how long that takes.

This is a problem with extremely large files.  There's nothing BBDB
(or any emacs) can do about it.

>> You have a very large .bbdb.

> Indeed. I automatically entries from all I mails and news messages I
> read.  

My bbdb/news-auto-create-p is a function that is only true when I'm in
an nnml group.  I've done this since October '93, and with four
different jobs, and a dozen mailing lists, I have just under 5000
entries (800k), which is manageable.

You may want to think about not adding all the usenet kiddies.

-- 
Colin

Re: Slow reading, gauge?

[François Pinard]

Hrvoje Niksic <hniksic@srce.hr> écrit:

> François Pinard <pinard@iro.umontreal.ca> writes:

> I've never truly understood why you are warning everyone in sight
> against BBDB?

Because I like people.  You know me! :-)

> What *is* the full extent of the nightmare, after all?

Kai replies for me:

> I once cleaned up my .bbdb, set all the automatically-add things to nil,
> now add people explicitly with `:', and I'm a happy camper.  My .bbdb
> is 70k and contains 500 entries.

We ought to take good precautions to refrain BBDB to save everything it sees.
Or else, after a while, it all get so big that we get the feeling Emacs
run like in molasso (hmph, Flyspell barfs, I thought it was English: I
mean this blackish-brown thick-sticky syrup-of-sugar).  Cleaning is then
an almost impossible task, if you came to depend on BBDB in the meantime.

If one uses means so BBDB is parsimonious and keeps control all along, then
I guess it is workable.  Nobody warned me of this, so I try to warn others!

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard

Re: Slow reading, gauge?

[David Hedbor]

Colin Rafferty <colin@xemacs.org> writes:

> David Hedbor writes:
> > Colin Rafferty <colin@xemacs.org> writes:
> 
> >> It might be.
> >> 
> >> If you do a `bbdb-create', does it feel exceptionally slow after your
> >> final RET?
> 
> > Yes. That is exactly what happens when I view a new record. 
> 
> Open a 2.6 mB file, go to somewhere in the middle of it, and type a
> character.  See how long that takes.

It takes about no time at all. Even when I do this with a 13MB file it 
doesn't take any time.

> This is a problem with extremely large files.  There's nothing BBDB
> (or any emacs) can do about it.

Well, I have no such problem so I doubt that's it.

-- 
[ Below is a random fortune, which is unrelated to the above message. ]
God may be subtle, but he isn't plain mean.
		-- Albert Einstein

Re: Slow reading, gauge?

[Colin Rafferty]

David Hedbor writes:
> Colin Rafferty <colin@xemacs.org> writes:

>> Open a 2.6 mB file, go to somewhere in the middle of it, and type a
>> character.  See how long that takes.

> It takes about no time at all. Even when I do this with a 13MB file it 
> doesn't take any time.

Okay, so I didn't think it through completely.

The real problem is that in order to insert a new record, BBDB has to
search for the insertion location, which is linear.

You can see the difference by first inserting Hank Aaron, and then
inserting Pat Zachary.

-- 
Colin

Re: Slow reading, gauge?

[Jan Vroonhof]

Colin Rafferty <colin@xemacs.org> writes:

> This is a problem with extremely large files.  There's nothing BBDB
> (or any emacs) can do about it.

Use the XEmacs database extensions?

Jan