Let's get effective!

Let's get effective!

[Lars Magne Ingebrigtsen]

I'm on another one of my "Must go faster!  Faster I say!" flings, so
I'm profiling and converting `int-to-string's to `princ's and
defsubsting and stuff.  (Which means that 0.53 won't work, in all
probability.) 

But what I was wondering if y'all could (after 0.53 has been released)
`M-x elp-instrument-package RET gnus RET', read news for a while, and
see whether `M-x elp-results' gives you any data that you find odd.
("Odd" would be one function called 5 million times or another
function called `gnus-tiny-function' using 5 seconds per iteration.)

Let me know what you find out.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Ingebrigtsen

Re: Let's get effective!

[Steven L Baur]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

Lars> I'm on another one of my "Must go faster!  Faster I say!" flings, so
Lars> I'm profiling and converting `int-to-string's to `princ's and
Lars> defsubsting and stuff.  (Which means that 0.53 won't work, in all
Lars> probability.) 

O.K.

Good deal.  I trust you are optimizing for reduced memory usage also?

-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be proofread for $250/hour.
Andrea Seastrand: For your vote on the Telecom bill, I will vote for anyone
except you in November.

Re: Let's get effective!

[d. hall]

-----BEGIN PGP SIGNED MESSAGE-----

// thus on 12 Mar 1996 20:10:02 -0800, Steven virtually scripted:

>>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
Steven> [...]

Lars> I'm on another one of my "Must go faster!  Faster I say!" flings, so
Lars> I'm profiling and converting `int-to-string's to `princ's and
Lars> defsubsting and stuff.  (Which means that 0.53 won't work, in all
Lars> probability.)

Steven> O.K.

Steven> Good deal.  I trust you are optimizing for reduced memory usage
Steven> also?

*groan* I just went out and bought some more Ram.  Does this mean I wasted
my money now ? =)  I've been going over a lot of Emacs documentation, and
based upon the current overview format, I can't really see how Gnus would
be able to save more memory (I'm mean newsreaders like tin alone use up 4+
megs).  I'm guessing that there has to be a way to window the overview
file, either by disk caching or something else.

d.
-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Comment: Processed by Mailcrypt 3.4, an Emacs/PGP interface

iQCVAwUBMUZYKoX26urqpgG1AQFj+wP/TRn+DUmwICyMi34YGCrTUHfrjMOvb1uL
ErTBRw9HLq9g0hNoxyssbDC27yt8TDMXds+BOGr1vdRD4ZyxxAo1xg3c1sJNGYHl
/LZe81flto3QBd90K+4DYQ8nn6497kZQN73XZMFvhRG3Xr5k3qu8cv20XKcV115+
1d6BqYNeuh4=
=L8iY
-----END PGP SIGNATURE-----

Re: Let's get effective!

[Steven L Baur]

>>>>> "Joe" == Joe Hildebrand <hildjj@fuentez.com> writes:

Joe> Does emacs have an interface to mmap?  It is probably the best
Joe> solution to this problem, on those systems that support it.  It can
Joe> probably be emulated on those that don't.  Those that don't support it
Joe> will just need more memory, or a better OS.

XEmacs does with the relocating allocator.  Emacs does not.  There has
been discussion about this though in the linux-gcc mailing list.
There are patches now available that allow Emacs 19.30 to use Doug
Lea's mmap-based malloc in Linux libc-5.3.5.  It helps, but is not
worth writing home about.  I mailed rms a copy of the XEmacs ralloc.c
this past weekend.  Since it is already copyright assigned to the FSF
I would expect it has an excellent chance of appearing in a future
version of Emacs.

None of this means that effort should be relaxed on cutting down the
excess garbage in Gnus.  My tests indicate that Gnus easily defeats
mmap mallocs at present.

Regards,
-- 
steve@miranova.com baur
Unsolicited commercial e-mail will be proofread for $250/hour.
Andrea Seastrand: For your vote on the Telecom bill, I will vote for anyone
except you in November.

Re: Let's get effective!

[Joe Hildebrand]

> "Steven" == Steven L Baur <steve@miranova.com> writes:

   Steven> None of this means that effort should be relaxed on cutting
   Steven> down the excess garbage in Gnus.  My tests indicate that
   Steven> Gnus easily defeats mmap mallocs at present.

Well, that wasn't exactly what I meant.  I was trying to suggest that
gnus mmap the .overview file semi-directly, rather than going through
lots of malloc stuff.  The more I think about it, though, the less
sure I am of the efficacy of that plan.  At that point, it probably
makes more sense for the overview files to be in gdbm format, or
something.  

Hey!  Hadn't there been some talk about a nngdbm backend?  Didn't
someone do a gdbm interface for emacs, that they were going to give to
RMS?

-- 
Joe Hildebrand                  Fuentez Systems Concepts
hildjj@fuentez.com              Lead Software Engineer
	"Breakfast recapitulates phylogeny" - Spider Robinson

Re: Let's get effective!

[William Perry]

Joe Hildebrand wrote:
> 
> > "Steven" == Steven L Baur <steve@miranova.com> writes:
> 
>    Steven> None of this means that effort should be relaxed on cutting
>    Steven> down the excess garbage in Gnus.  My tests indicate that
>    Steven> Gnus easily defeats mmap mallocs at present.
> 
> Well, that wasn't exactly what I meant.  I was trying to suggest that
> gnus mmap the .overview file semi-directly, rather than going through
> lots of malloc stuff.  The more I think about it, though, the less
> sure I am of the efficacy of that plan.  At that point, it probably
> makes more sense for the overview files to be in gdbm format, or
> something.
> 
> Hey!  Hadn't there been some talk about a nngdbm backend?  Didn't
> someone do a gdbm interface for emacs, that they were going to give to
> RMS?

  That would be me.  I have GDBM and BerkeleyDB support in XEmacs currently.  I haven't
had a chance to backport them to the stock Emacs 19 yet, and am waiting to get some
generic waver papers signed by my current boss so anything I do is not encumbered.

  Anybody with the XEmacs 19.14beta code should be able to work on this stuff though.
I can try to pull out the patches I made if someone thinks they can find the time
before the end of the month to stick it in Emacs 19.

-Bill P.

Re: Let's get effective!

[Timo Metzemakers]

[Joe Hildebrand]

|  Hey!  Hadn't there been some talk about a nngdbm backend?  Didn't
|  someone do a gdbm interface for emacs, that they were going to give
|  to RMS?

I think I did such a thing (actually it was just a quick hack).  I
talked to RMS about it, but he was adamant about doing any
gdbm-related stuff in a subprocess.  So I dropped the matter.

Timo

Re: Let's get effective!

[Joe Hildebrand]

> "Timo" == Timo Metzemakers <metzemakers@labri.u-bordeaux.fr> writes:

   Timo> I think I did such a thing (actually it was just a quick
   Timo> hack).  I talked to RMS about it, but he was adamant about
   Timo> doing any gdbm-related stuff in a subprocess.  So I dropped
   Timo> the matter.

Couldn't we just write a sub-process that did this, then?  It's
interface would look a heckuva lot like an nntp server...  At that
point, though, I guess we have written an IMAP server.

-- 
Joe Hildebrand                  Fuentez Systems Concepts
hildjj@fuentez.com              Lead Software Engineer
	"Breakfast recapitulates phylogeny" - Spider Robinson

Re: Let's get effective!

[Lars Magne Ingebrigtsen]

dhall@illusion.apk.net (d. hall) writes:

> I'm guessing that there has to be a way to window the overview file,
> either by disk caching or something else.

No.  Gnus has to read info pertaining to all the articles to be shown
before showing any of the articles.  Threading, sorting, etc.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Let's get effective!

[Lars Magne Ingebrigtsen]

Steven L Baur <steve@miranova.com> writes:

> I trust you are optimizing for reduced memory usage also?

You'd be wrong to trust me to do anything.  :-)

Anyways, I've now hacked up alloc.c to defvar all the interesting
alloc variables and I've hacked up elp.el to profile using these
variables to monitor for consing, stringsing, vectoring and so on.  It
even looks like it might be working!  

We'll see what comes of it.

I've done some slight calculation on what how much memory Gnus should
theoretically use for each article in a group.  The answer is:

(+ (* (+ 6 10 20) 8) 200 200)
  => 688 bytes

That means that Emacs should grow with half a meg when entering a 1000
article group the first time.  I think we can all agree that Emacs
grows quite a bit more.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."