[Caml-list] Debugging memory-intensive programs...

[Caml-list] Debugging memory-intensive programs...

[Florian Hars]

Timetravel considered harmful:

I just tried to debug a program that reads a lot of data and keeps
a significant part of it in memory because it died with a string
or array out of bounds exception halfway through it's input.
Here is part of what syslog said when I ran it in ocamldebug:

Nov  9 15:06:33 quetelet kernel: VM: do_try_to_free_pages failed for kswapd...
Nov  9 15:06:34 quetelet kernel: VM: do_try_to_free_pages failed for mozilla-bin...
Nov  9 15:06:57 quetelet last message repeated 4 times
Nov  9 15:06:57 quetelet kernel: VM: do_try_to_free_pages failed for apache...
Nov  9 15:07:01 quetelet kernel: VM: do_try_to_free_pages failed for kswapd...

[...]

Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for sshd...
Nov  9 15:14:41 quetelet last message repeated 4 times
Nov  9 15:14:41 quetelet kernel: VM: killing process sshd

[...]

Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for emacs...
Nov  9 15:14:41 quetelet last message repeated 16 times
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for bash...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for klogd...
Nov  9 15:14:41 quetelet last message repeated 15 times
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for proftpd...
Nov  9 15:14:41 quetelet last message repeated 15 times
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for XF86_Mach64...
Nov  9 15:14:41 quetelet last message repeated 15 times

[...]

Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for emacs...
Nov  9 15:14:41 quetelet last message repeated 16 times
Nov  9 15:14:41 quetelet kernel: VM: killing process emacs
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for bash...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for inetd...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for bash...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for editor...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for mutt...
Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for syslogd...

Guess I have to do something else... :-)

Yours, Florian Hars.
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr

Re: [Caml-list] Debugging memory-intensive programs...

[Gerd Stolpmann]

On 2001.11.09 15:38 Florian Hars wrote:
> Timetravel considered harmful:
> 
> I just tried to debug a program that reads a lot of data and keeps
> a significant part of it in memory because it died with a string
> or array out of bounds exception halfway through it's input.
> Here is part of what syslog said when I ran it in ocamldebug:

That's one of the memory management problems of Linux. It overcommits 
memory, i.e.
you can allocate more memory than you really have (including swap 
space). On the one
hand, this allows to better exhaust memory, but on the other hand, 
there is no
real criterion whether memory is out or not. In the best case, the 
process writing
to a memory page gets a segfault. The worst case is that the kernel 
needs itself
memory and begins to kill processes (this is what you see in the 
syslog).

See http://lwn.net/2000/0309/kernel.php3

Because ocamldebug checkpoints the program state by forking 
subprocesses, there are
a lot "copy-on-write" memory pages, so it is likely that you run into 
this problem.

Possible solutions:
- Increase swapspace
- Decrease the number of checkpoints of ocamldebug (set processcount, 
also set bigstep,
   set smallstep)


> Nov  9 15:06:33 quetelet kernel: VM: do_try_to_free_pages failed for
> kswapd...
> Nov  9 15:06:34 quetelet kernel: VM: do_try_to_free_pages failed for
> mozilla-bin...
> Nov  9 15:06:57 quetelet last message repeated 4 times
> Nov  9 15:06:57 quetelet kernel: VM: do_try_to_free_pages failed for
> apache...
> Nov  9 15:07:01 quetelet kernel: VM: do_try_to_free_pages failed for
> kswapd...
> 
> [...]
> 
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> sshd...
> Nov  9 15:14:41 quetelet last message repeated 4 times
> Nov  9 15:14:41 quetelet kernel: VM: killing process sshd
> 
> [...]
> 
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> emacs...
> Nov  9 15:14:41 quetelet last message repeated 16 times
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> bash...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> klogd...
> Nov  9 15:14:41 quetelet last message repeated 15 times
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> proftpd...
> Nov  9 15:14:41 quetelet last message repeated 15 times
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> XF86_Mach64...
> Nov  9 15:14:41 quetelet last message repeated 15 times
> 
> [...]
> 
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> emacs...
> Nov  9 15:14:41 quetelet last message repeated 16 times
> Nov  9 15:14:41 quetelet kernel: VM: killing process emacs
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> bash...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> inetd...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> bash...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> editor...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> mutt...
> Nov  9 15:14:41 quetelet kernel: VM: do_try_to_free_pages failed for
> syslogd...
> 
> Guess I have to do something else... :-)
> 
> Yours, Florian Hars.
> -------------------
> Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ:
> http://caml.inria.fr/FAQ/
> To unsubscribe, mail caml-list-request@inria.fr  Archives:
> http://caml.inria.fr
> 
-------------------
Bug reports: http://caml.inria.fr/bin/caml-bugs  FAQ: http://caml.inria.fr/FAQ/
To unsubscribe, mail caml-list-request@inria.fr  Archives: http://caml.inria.fr