Lisp pointer size warning?

Lisp pointer size warning?

[Mattias Ahnberg]

Ever since I upgraded to Gnus 5.10.2 I've been getting these annoying
messages, I never got them before (I ran ognus 0.17 before upgrading).

    Warning: memory in use exceeds lisp pointer size
    Killing some buffers may delay running out of memory.
    However, certainly by the time you receive the 95% warning,
    you should clean up, kill this Emacs, and start a new one.

They are very bothersome. What can I do about it? Something that has
changed that affects them?

/ahnberg.

Re: Lisp pointer size warning?

[Kai Großjohann]

Mattias Ahnberg <mattias@ahnberg.pp.se> writes:

> Ever since I upgraded to Gnus 5.10.2 I've been getting these annoying
> messages, I never got them before (I ran ognus 0.17 before upgrading).
>
>     Warning: memory in use exceeds lisp pointer size
>     Killing some buffers may delay running out of memory.
>     However, certainly by the time you receive the 95% warning,
>     you should clean up, kill this Emacs, and start a new one.
>
> They are very bothersome. What can I do about it? Something that has
> changed that affects them?

I also get similar errors.  After them, Emacs usually becomes
unusable.

RMS advises to run Emacs under the debugger, to set a breakpoint in
memory_full, and to get a Lisp-level backtrace whenever the
breakpoint is hit, to find out why so much memory is needed.  Also
see the file etc/DEBUG for more information on debugging.  An Emacs
source tree is useful for this.

I've tried to do this, but got no real results.  The breakpoint
wasn't hit when I was looking, and when the memory was full, it was
too late to see anything interesting by attaching gdb to the Emacs
process.

Maybe you could try to find out something?

(I'm suspecting the Agent somehow -- I get the memory full thing
after doing `J s'.  But I have no idea what's happening.  Maybe I
b0rked the internal data structures and now Gnus is punishing me.)
-- 
This line is not blank.

Re: Lisp pointer size warning?

[Danny Siu]

I have been observing the similar memory warnings after I added: (add-hook
'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

With the above removed, I don't see the warning as often.  It only after I
do many (20+) 'J s' does the warning appear.  I see the Emacs memory usage
(with task manager on WinXP) increases 100-500K whenever I do 'J s'.  I
decided to clean up my News/Agent/* files and start anew with .newsrc.eld
(removed and re-subscribed all groups) but still ended up with memory
warnings after many 'J s'.

I use nnmail as primary and nnimap as secondary select method.

Kai Großjohann writes:

  Kai> Mattias Ahnberg <mattias@ahnberg.pp.se> writes:
  >> Ever since I upgraded to Gnus 5.10.2 I've been getting these annoying
  >> messages, I never got them before (I ran ognus 0.17 before upgrading).
  >>
  >> Warning: memory in use exceeds lisp pointer size Killing some buffers
  >> may delay running out of memory.  However, certainly by the time you
  >> receive the 95% warning, you should clean up, kill this Emacs, and
  >> start a new one.
  >>
  >> They are very bothersome. What can I do about it? Something that has
  >> changed that affects them?

  Kai> I also get similar errors.  After them, Emacs usually becomes
  Kai> unusable.

  Kai> RMS advises to run Emacs under the debugger, to set a breakpoint in
  Kai> memory_full, and to get a Lisp-level backtrace whenever the
  Kai> breakpoint is hit, to find out why so much memory is needed.  Also
  Kai> see the file etc/DEBUG for more information on debugging.  An Emacs
  Kai> source tree is useful for this.

  Kai> I've tried to do this, but got no real results.  The breakpoint
  Kai> wasn't hit when I was looking, and when the memory was full, it was
  Kai> too late to see anything interesting by attaching gdb to the Emacs
  Kai> process.

  Kai> Maybe you could try to find out something?

  Kai> (I'm suspecting the Agent somehow -- I get the memory full thing
  Kai> after doing `J s'.  But I have no idea what's happening.  Maybe I
  Kai> b0rked the internal data structures and now Gnus is punishing me.)
  Kai> -- This line is not blank.


-- 
Danny Siu

Re: Lisp pointer size warning?

[Kai Großjohann]

Danny Siu <dsiu@adobe.com> writes:

> I have been observing the similar memory warnings after I added: (add-hook
> 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

I have this, too.

> With the above removed, I don't see the warning as often.  It only after I
> do many (20+) 'J s' does the warning appear.  I see the Emacs memory usage
> (with task manager on WinXP) increases 100-500K whenever I do 'J s'.

I wonder where is that memory going?
-- 
This line is not blank.

Re: Lisp pointer size warning?

[Danny Siu]

Kai Großjohann writes:

  >> With the above removed, I don't see the warning as often.  It only
  >> after I do many (20+) 'J s' does the warning appear.  I see the Emacs
  >> memory usage (with task manager on WinXP) increases 100-500K whenever I
  >> do 'J s'.

  Kai> I wonder where is that memory going?  

Is there any lisp level stuff for tracing where the memory goes without
resorting into gdb?

-- 
Danny Siu

Re: Lisp pointer size warning?

[Dave Love]

Danny Siu <dsiu@adobe.com> writes:

> Is there any lisp level stuff for tracing where the memory goes without
> resorting into gdb?

Well, the first thing you want to do is to find out what type of
object is taking most of it and whether there are vast numbers of
things on the freelists for some reason:

(garbage-collect)
  => ((462989 . 39897) (28247 . 9) (1833 . 5746) 1467501 674844 (544 . 287) (8551 . 1062) (59665 . 5414))

Obviously you can grovel the obarray and, for instance, examine the
aggregate objects in all the value slots, but you probably can't do
the job fully at the Lisp level.  There is no specific heap profiling
facility anyhow, though there is execution profiling support which
could tell you the frequency with which the various allocators are
called, and setting `garbage-collection-messages' can show up
operations which cons even more than normal for Gnus.

Re: Lisp pointer size warning?

[Mattias Ahnberg]

>> "DS" == Danny Siu <dsiu@adobe.com> writes:

DS> I have been observing the similar memory warnings after I added: (add-hook
DS> 'gnus-select-article-hook 'gnus-agent-fetch-selected-article)

I don't have this set at all, it is set to nil.

DS> With the above removed, I don't see the warning as often.  It only
DS> after I do many (20+) 'J s' does the warning appear.

I never use the agent functions either, atleast not directly. I do not
know how "integrated" the agent has been made in 5.10.2 so I might be
using it more than I know indirectly.

I should point out that I mainly use Gnus for e-mail, nnfolder, if
that would matter for this.

/ahnberg.