IMAP trouble - and my memory's gone, too...

IMAP trouble - and my memory's gone, too...

[Robin S. Socha]

0.04 from CVS as of  a couple of minutes ago:

,----
| nnimap: Group INBOX is not uidvalid.  Continue? (y or n)  
`----

And then:

,----
| (1) (memory/critical) Warning: past 75% of memory limit 
| Killing some buffers may delay running out of memory. 
| However, certainly by the time you receive the 95% warning, 
| you should clean up, kill this Emacs, and start a new one. 
|  
| (2) (memory/critical) Warning: past 85% of memory limit 
| Killing some buffers may delay running out of memory. 
| However, certainly by the time you receive the 95% warning, 
| you should clean up, kill this Emacs, and start a new one. 
`----

Like, HELP!!!1 ;-) The system is OpenBSD2.8-current, XEmacs is 21.1.14
IIRC. 
-- 
Robin S. Socha - Your Worst Network Nightmare(tm).
`In Germany, they are not referred to as network administrators. They
prefer to be called "Sons Of The Third Reich".' (Kate: www.katewerk.com)

Re: IMAP trouble - and my memory's gone, too...

[Simon Josefsson]

"Robin S. Socha" <robin@socha.net> writes:

> | nnimap: Group INBOX is not uidvalid.  Continue? (y or n)  

Answer yes.  I think then it will replace the uidvalidity value on the
group and then proceed.  Possibly some things are now corrupt (like
the NOV cache in ~/News/overview/nnimap.<group>), but probably not.

The proper solution is to make Gnus wipe out everything (flags, cache,
agent etc) it knows about the group and sync it from the server again,
but a backend can't tell Gnus to do that today.  So instead nnimap
asks the user a cryptic question, and whatever the user answers he
will suspect that he created the problem and not nnimap. :) Not good,
I know.

> | (2) (memory/critical) Warning: past 85% of memory limit 
> | Killing some buffers may delay running out of memory. 
> | However, certainly by the time you receive the 95% warning, 
> | you should clean up, kill this Emacs, and start a new one. 

Well, how large is your emacsen?  Perhaps the gnus-range functions
take a lot of memory if it has to operate on a huge range, like
expanding the range (1 . 10000000).  Also, imap.el caches some stuff
from the server.

Re: IMAP trouble - and my memory's gone, too...

[Robin S. Socha]

* Simon Josefsson <simon@josefsson.org> [010512 03:58]:
> "Robin S. Socha" <robin@socha.net> writes:
> 
> > | nnimap: Group INBOX is not uidvalid.  Continue? (y or n)  
> 
> Answer yes.  I think then it will replace the uidvalidity value on the
> group and then proceed.  Possibly some things are now corrupt (like
> the NOV cache in ~/News/overview/nnimap.<group>), but probably not.

OK. I have (see below) allegedly approx. 65000 messages.

> The proper solution is to make Gnus wipe out everything (flags, cache,
> agent etc) it knows about the group and sync it from the server again,
> but a backend can't tell Gnus to do that today.  So instead nnimap
> asks the user a cryptic question, and whatever the user answers he
> will suspect that he created the problem and not nnimap. :) Not good,
> I know.

Ok. So what do I do now?

> > | (2) (memory/critical) Warning: past 85% of memory limit 
> > | Killing some buffers may delay running out of memory. 
> > | However, certainly by the time you receive the 95% warning, 
> > | you should clean up, kill this Emacs, and start a new one. 
> 
> Well, how large is your emacsen?  Perhaps the gnus-range functions
> take a lot of memory if it has to operate on a huge range, like
> expanding the range (1 . 10000000).  Also, imap.el caches some stuff
> from the server.

That was 65k messages. HELP! I'm stuck with mutt and vim. Do something.
Please!

Re: IMAP trouble - and my memory's gone, too...

[Simon Josefsson]

"Robin S. Socha" <robin-dated-990090181.ee1ff5@socha.net> writes:

> > The proper solution is to make Gnus wipe out everything (flags, cache,
> > agent etc) it knows about the group and sync it from the server again,
> > but a backend can't tell Gnus to do that today.  So instead nnimap
> > asks the user a cryptic question, and whatever the user answers he
> > will suspect that he created the problem and not nnimap. :) Not good,
> > I know.
> 
> Ok. So what do I do now?

To be really sure, rm the NOV cache file and remove the uidvalidity
value (`G p').

> > > | (2) (memory/critical) Warning: past 85% of memory limit 
> > > | Killing some buffers may delay running out of memory. 
> > > | However, certainly by the time you receive the 95% warning, 
> > > | you should clean up, kill this Emacs, and start a new one. 
> > 
> > Well, how large is your emacsen?  Perhaps the gnus-range functions
> > take a lot of memory if it has to operate on a huge range, like
> > expanding the range (1 . 10000000).  Also, imap.el caches some stuff
> > from the server.
> 
> That was 65k messages. HELP! I'm stuck with mutt and vim. Do something.
> Please!

How large is the memory footprint of emacs?  When does the error
occur, when entering a summary buffer?  Assuming imap.el stores 500
byte for each mail (not unlikely), 65k messages needs 32MB only within
imap.el. And building a summary buffer, with threading and whatnot,
will probably take some memory as well.  Mutt/pine etc only need to
hold in memory what you're currently viewing, while Gnus want to hold
large parts of the group in memory, so there's no easy fix here.

I just entered a nnimap group with 8k messages, and my emacs growed by
10MB.  It took about 50 seconds.  Entering a nnimap group with 16k
messages crashed my emacs (not reproducable though), but on second try
emacs grow by about 15MB.