any tips for how to tune Gnus/emacs for low memory usage?

any tips for how to tune Gnus/emacs for low memory usage?

[Steinar Bang]

Platform: GNU Emacs 20.7.2, Oort Gnus v0.04 (CVS update from october 6)
          using the nnimap and nntp backends, with Gnus agent,
          Intel P133, debian woody linux w/the 2.2.19 kernel

I'm running Gnus inside of GNU emacs under linux and XFree86, on what
today is a small machine: P133 with 24 MB of RAM (top reports 22724k
total.  Presumably the rest of the memory is taken by the kernel).

My problem is that things start to get slow when the size of the emacs
process approaches the size of the physical memory, and the system
starts paging at the drop of a hat.  What drives memory consumption up
to this level is in my case, opening the newsgroup no.alt.motorsykler.

In this group I have ticked some old articles, and since I'm using the
agent they never expire.  The group has a lot of traffic, which means
that Gnus have fill in all the empty space between the article numbers
when opening the group.

In addition I do a lot of scoring in this group.  I guess the followup
scoring of some persons really add to the cost.  I also do adaptive
scoring based on reading, and deleting of articles.

Is there anything I can try to keep the size of the Emacs process
down?  Stay completely away from groups with sparse article numbers?
Stay away from adaptive/followup scoring?

Or are there some variables I could try tuning to keep the memory
usage down?


- Steinar

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Stainless Steel Rat]

* Steinar Bang <sb@dod.no>  on Sun, 25 Nov 2001
| I'm running Gnus inside of GNU emacs under linux and XFree86, on what
| today is a small machine: P133 with 24 MB of RAM (top reports 22724k
| total.  Presumably the rest of the memory is taken by the kernel).

You have barely enough memory for the X server, and not nearly enough for X
and Emacs.  Either don't run X, or add memory to the machine.
-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Steinar Bang]

>>>>> Stainless Steel Rat <ratinox@peorth.gweep.net>:

> You have barely enough memory for the X server, and not nearly
> enough for X and Emacs.

Running KDE or Gnome is out, but the X server itself isn't all that
much of a hog.  And I'm using icewm as the WM, which uses only around
800k.

No problem at all running X and emacs without swapping... until I've
used Gnus to visit one of the newsgroups that makes the emacs process
size ballon.

The two big culprits, according to top, are emacs/Gnus (after visiting
one of the problem groups), and opera.

I'll try out galeon when it moves out of debian unstable, and into
testing.  But I don't really have a replacement candidate for Gnus.

I'll also try out the 2.4-series to see if the VM is more efficient.

> Either don't run X, or add memory to the machine.

This machine is just a hobby machine.  I don't know if more memory is
even available from anywhere for old DEC laptops.

In any case it is a matter of pride to make linux run faster, and work
better, than Win95 did on the same machine.  Adding memory would be
cheating. 

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Karl Kleinpaste]

Steinar Bang <sb@dod.no> writes:
> No problem at all running X and emacs without swapping... until I've
> used Gnus to visit one of the newsgroups that makes the emacs process
> size ballon.

You might check on your settings of things like gnus-fetch-old-headers
and gnus-build-sparse-threads.  Using apropos (I have XEmacs with
hyper-apropos) to look at all the gnus.*header and gnus.*thread
possibilities is probably a good idea.

> In any case it is a matter of pride to make linux run faster, and work
> better, than Win95 did on the same machine.  Adding memory would be
> cheating. 

:-)

My poor old 486dx4/100 laptop has only 20M, and I've nearly given up
on X entirely.  Without ever using anything so complex as Gnus, it
swaps merely by moving the mouse between xterms and XEmacs.

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Stainless Steel Rat]

* Steinar Bang <sb@dod.no>  on Mon, 26 Nov 2001
| In any case it is a matter of pride to make linux run faster, and work
| better, than Win95 did on the same machine.  Adding memory would be
| cheating.

If you -did- run Emacs and Gnus on the same machine with Windows 95, it
would thrash greatly.  Believe me.  I wrote pop3.el on a Windows 95
notebook, a 50MHz 486 with 12MB physical memory.  That was about five years
ago, before Gnus got big :).

If you -really- want to trim it down that badly, fall back to Emacs 19 or
XEmacs 20 compiled without any features.  That should get your Emacs
footprint down to around 10-15MB.  Course, you won't have all the new
stuff, but it -is- the new stuff that is gobbling up memory.
-- 
Rat <ratinox@peorth.gweep.net>    \ If Happy Fun Ball begins to smoke, get
Minion of Nathan - Nathan says Hi! \ away immediately. Seek shelter and cover
PGP Key: at a key server near you!  \ head.
       That and five bucks will get you a small coffee at Starbucks.

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Daniel Pittman]

On Mon, 26 Nov 2001, Steinar Bang wrote:
>>>>>> Stainless Steel Rat <ratinox@peorth.gweep.net>:
> 
>> You have barely enough memory for the X server, and not nearly
>> enough for X and Emacs.
> 
> Running KDE or Gnome is out, but the X server itself isn't all that
> much of a hog.  And I'm using icewm as the WM, which uses only around
> 800k.

Quite. X, contrary to rumors, is rather light-weight. I happily ran it
on my P-133 with 24MB of ram for an awful while. :)

> No problem at all running X and emacs without swapping... until I've
> used Gnus to visit one of the newsgroups that makes the emacs process
> size ballon.
> 
> The two big culprits, according to top, are emacs/Gnus (after visiting
> one of the problem groups), and opera.

The best I ever found for keeping the size of the Emacs process that was
running stuff down was to keep Gnus distinct from my standard Emacs[1],
to ensure that W3 was *not* available under Gnus, and to make sure I
never kept ticked articles lying around for long.

Oh, and I had fairly small groups at the time, though that doesn't seem
to have made that much difference...

> I'll try out galeon when it moves out of debian unstable, and into
> testing.  

I doubt that you will be happy; this is a very light session:

 3633 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   2:59 galeon-bin
 3638 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   0:00 galeon-bin
 3639 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   0:04 galeon-bin
 3640 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   0:00 galeon-bin
 4788 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   0:00 galeon-bin
 6786 daniel     9   0 45040  41M 18224 S     0  0.0 14.5   0:00 galeon-bin

That's not exactly a light-weight browser, and it bloats relatively fast.

> But I don't really have a replacement candidate for Gnus.

I never met anything better. :0

> I'll also try out the 2.4-series to see if the VM is more efficient.

It's a heavier footprint than 2.2 but 2.4.16-pre1 and close to that are
fairly good. They swap sooner, feel decent interactively and don't have
too many harsh edges.

        Daniel

Footnotes: 
[1]  I did this briefly, but stopped for the lack of integration.

-- 
There is no female mind. The brain is not an organ of sex.
As well to speak of a female liver.
        -- Charlette Perkins Gilman, _Women and Economics_

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

Steinar Bang <sb@dod.no> writes:

> Running KDE or Gnome is out, but the X server itself isn't all that
> much of a hog.  And I'm using icewm as the WM, which uses only around
> 800k.

What's the smallest WM?

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[luis fernandes]

9wm. The look-alike to 9 1/2, which is the wm for Plan 9.

(The compressed shar file is 29857 bytes.)

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

luis fernandes <elf@ee.ryerson.ca> writes:

> 9wm. The look-alike to 9 1/2, which is the wm for Plan 9.
>
> (The compressed shar file is 29857 bytes.)

Thanks.  Here are some other small ones I've found after looking
today:

  wm name         tar.gz size
-----------------------------
  wm2             32807
  heliwm          19785
  flwm            61000
  aewm            29811
  larswm          61257
  w9wm            25000

Which ones have good mouseless operation?  (larswm seems to have
this, but it isn't soo small.)

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Kai Großjohann]

(Chris Beggy ) news@kippona.com writes:

> Which ones have good mouseless operation?  (larswm seems to have
> this, but it isn't soo small.)

Ion provides good mouseless operation, but is larger even than larswm.

kai
-- 
Simplification good!  Oversimplification bad!  (Larry Wall)

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Stainless Steel Rat]

* (Chris Beggy ) news@kippona.com  on Tue, 27 Nov 2001
| What's the smallest WM?

No window manager at all.  Have your .xsession or .xinitrc open up whatever
clients you need, specifiying geometry and location as required to avoid
overlapping windows.
-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball contains a liquid core,
Minion of Nathan - Nathan says Hi! \ which, if exposed due to rupture, should
PGP Key: at a key server near you!  \ not be touched, inhaled, or looked at.
       That and five bucks will get you a small coffee at Starbucks.

Re: any tips for how to tune Gnus/emacs for low memory usage?

[luis fernandes]

>>>>> "Rat" == Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

    Rat> * (Chris Beggy ) news@kippona.com on Tue, 27 Nov 2001 |
    Rat> What's the smallest WM?

    Rat> No window manager at all.

That's a good idea; just start a full-screen Emacs from your
~/.xsession

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Robin S. Socha]

* luis fernandes <elf@ee.ryerson.ca> writes:
> Stainless Steel Rat <ratinox@peorth.gweep.net> writes:
>>  (Chris Beggy ) news@kippona.com on Tue, 27 Nov 2001 

>>> What's the smallest WM?

>> No window manager at all.

> That's a good idea; just start a full-screen Emacs from your ~/.xsession

http://www.students.tut.fi/~tuomov/ion/

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Paul Jarc]

Stainless Steel Rat <ratinox@peorth.gweep.net> wrote:
> * (Chris Beggy ) news@kippona.com  on Tue, 27 Nov 2001
>| What's the smallest WM?
>
> No window manager at all.  Have your .xsession or .xinitrc open up whatever
> clients you need, specifiying geometry and location as required to avoid
> overlapping windows.

Now you've got me wondering about the possibility of just having a set
of shell commands to run out of an xterm to raise/lower windows, etc.
Hmm.


paul

Re: any tips for how to tune Gnus/emacs for low memory usage?

[luis fernandes]

>>>>> "robin" == Robin S Socha <robin-dated-1007148969.21ca13@socha.net> writes:
    robin> http://www.students.tut.fi/~tuomov/ion/

Ummm....more like:

http://ratpoison.sourceforge.net/

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

(Chris Beggy ) news@kippona.com writes:

> luis fernandes <elf@ee.ryerson.ca> writes:
>
>> 9wm. The look-alike to 9 1/2, which is the wm for Plan 9.
>>
>> (The compressed shar file is 29857 bytes.)
>
> Thanks.  Here are some other small ones I've found after looking
> today:

Here are some small ones (listed with the output of size) I
installed today:

  text    data     bss     dec     hex filename
  14241     572     132   14945    3a61 aewm
   6028     416    2736    9180    23dc xaw-palette (part of aewm)
   2852     316    2700    5868    16ec xaw-panel(part of aewm)
   2970     316    2712    5998    176e xaw-switch(part of aewm)
  24615    2304     348   27267    6a83 w9wm
  62936     736   11844   75516   126fc larswm
  61977   13068     432   75477   126d5 wm2
  16187     584     436   17207    4337 heliwm

I couldn't get heliwm launched with xinit, which was
disappointing. It doesn't require any pointing device at
all, and that interests me.

The others were interesting, with w9wm having good mouseless
behavior.  (C-TAB to cycle through, raise, and focus to windows.)

9wm is missing from this table because its ftp site was down.

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

luis fernandes <elf@ee.ryerson.ca> writes:

> http://ratpoison.sourceforge.net/

Thanks for this link! /ratpoison/ is really good from my
emacs-user point of view.  I get most of the eye candy I need
from my emacs fonts and xfonts.  It also seems to use
video memory very well, as well as screen real-estate ( I'm on a
laptop a lot.)

Remote X sessions and multiple emacs frames are handled correctly.

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Steinar Bang]

>>>>> luis fernandes <elf@ee.ryerson.ca>:

>>>>> Robin S Socha <robin-dated-1007148969.21ca13@socha.net> writes:

>> http://www.students.tut.fi/~tuomov/ion/

> Ummm....more like:

> http://ratpoison.sourceforge.net/

In debian woody, it was easy to try them out:
	apt-get install ion ratpoison
installed them, and made them available as KDM alternatives.

First impression of ratpoison: I never got anything to happend with
any of the commands listed in the man page.  I eventually crashed out
of the session, and back into KDM, but with the display trashed and no
cursor, and no response from Ctrl-Alt-Fx for switching between virtual
consoles. 

I logged in from a different machine and first tried restarting KDM,
which made no difference (ie. still trashed screen, and still no
cursor, and still frozen keyboard).  I stopped KDM, but the display
was still the frozen KDM display.  I did a halt on the machine, which
cut the connection to the other machine, but didn't change the
display.  I turned the power off, and then back on, and the laptop
booted normally.

I think ratpoison is probably not for me...:-)

First impression of ion: spartan, but the commands I needed were
simple and easy to remember (F3, F12, M-k n, M-k p).  I'm running ion
right now, with an emacs, a top running in an xterm, and opera.

The advantages for me with this approach, compared to just running
emacs in a virtual console, are:
 - things look better
 - the keyboard commands to emacs work better here than in the virtual
   console 
 - I can click on a URL to have it opened in opera.  One problem here,
   is that instead of being displayed in an existing opera process,
   like it happens in icewm, a new opera process is started when I
   click on a link

Disadvantages:
 - no clock always on screen like icewm, and KDE has
 - no load indicator always on screen like icewm has

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Kai Großjohann]

Steinar Bang <sb@dod.no> writes:

> Disadvantages:
>  - no clock always on screen like icewm, and KDE has
>  - no load indicator always on screen like icewm has

On the Ion list, some patches have been circulating which allow you
to reserve a strip of the screen for something else, such as a KDE
panel, or a Gnome panel.

kai
-- 
Simplification good!  Oversimplification bad!  (Larry Wall)

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> On the Ion list, some patches have been circulating which allow you

I compiled, installed, evaluated and took a look at the
output of ls -l and size for some of the small window managers:

     size     name         comment
--------------------------------------------------------------------
      17232  aewm*         simple, few knobs, plug-ins for menus
    1293351  blackbox*     not really small, coffemaker
     187976  fvwm2*        supports mouseless
     119812  fvwm95*       supports mouseless
      18968  heliwm*       couldn't get it to launch an xterm, my bad
      66496  larswm*       tiling, good mouseless, documentation
      23668  lwm*          simple
      57819  ratpoison*    max screen use, emacs-like, good mouseless
     127332  twm*          classic
      29436  w9wm*         virtual screens, not mouseless
     165757  wm2*          excellent aesthetics, not mouseless
     672256  wmx*          wm2 aesthetics, virtuals, good mouseless

   text	   data	    bss	    dec	    hex	filename
------------------------------------------------
  14241	    572	    132	  14945	   3a61	aewm
 160897	  26996	    336	 188229	  2df45	blackbox
 178635	   4456	  12096	 195187	  2fa73	fvwm2
 112149	   3024	   2916	 118089	  1cd49	fvwm95
  16187	    584	    436	  17207	   4337	heliwm
  62936	    736	  11844	  75516	  126fc	larswm
  20160	    832	    152	  21144	   5298	lwm
  33919	   1216	    448	  35583	   8aff	ratpoison
 118911	   5080	  82484	 206475	  3268b	twm
  24615	   2304	    348	  27267	   6a83	w9wm
  61977	  13068	    432	  75477	  126d5	wm2
  92111	  14720	    188	 107019	  1a20b	wmx

I don't understand why wm2 is so large, given that its feature
set is similar to aewm or lwm.

I'm interested in making best use of a small laptop screen,
mouseless navigation, freeing up resources for emacs :-)

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[news]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> * (Chris Beggy ) news@kippona.com  on Tue, 27 Nov 2001
> | What's the smallest WM?
>
> No window manager at all.  Have your .xsession or .xinitrc open up whatever
> clients you need, specifiying geometry and location as required to avoid
> overlapping windows.

Thanks.  This works well.  I know I can maximize with the right
-geometry options at launch or the right set-frame-height,width
settings in .emacs.

Is there any way to auto-maximize, and have emacs occupy the
whole screen regardless of screen size?  Or is that one of the
services of a window manager?

Chris

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Stainless Steel Rat]

* (Chris Beggy ) news@kippona.com  on Fri, 30 Nov 2001
| Is there any way to auto-maximize, and have emacs occupy the
| whole screen regardless of screen size?  Or is that one of the
| services of a window manager?

Client windows have geometries.  Minimize/maximize is a window manager
function.
-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 
       That and five bucks will get you a small coffee at Starbucks.

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Kevin Ryde]

prj@po.cwru.edu (Paul Jarc) writes:
>
> Now you've got me wondering about the possibility of just having a set
> of shell commands to run out of an xterm to raise/lower windows, etc.
> Hmm.

"xwit" does a few such things, apparently it lives in

        ftp://ftp.x.org/contrib/utilities

except that seems inaccessible right at this moment.  Also in a Debian
package

        http://packages.debian.org/xwit

Of course a real man would just netcat straight to localhost:6000 :-).

Re: any tips for how to tune Gnus/emacs for low memory usage?

[Steinar Bang]

>>>>> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann):

> Steinar Bang <sb@dod.no> writes:
>> Disadvantages:
>> - no clock always on screen like icewm, and KDE has
>> - no load indicator always on screen like icewm has

> On the Ion list, some patches have been circulating which allow you
> to reserve a strip of the screen for something else, such as a KDE
> panel, or a Gnome panel.

I'm afraid apt-get has turned me away from downloading, patching and
building. :-)

Gnus is the only package I use that lives from the CVS checkout.

But I put this in my ~/.emacs, and it takes me part of the way:

(setq display-time-24hr-format t)
(setq display-time-day-and-date t)
(display-time)