gc crash

gc crash

[Warren Harris]

I've observed the following gc crash with my ocaml (3.11.2) application, after attempting to implement a memory-flushing routine with Gc.create_alarm:

Program received signal SIGSEGV, Segmentation fault.
0x00000000007c15a7 in caml_fl_allocate (wo_sz=<value optimized out>) at freelist.c:171
171     freelist.c: No such file or directory.
        in freelist.c
(gdb) bt
#0  0x00000000007c15a7 in caml_fl_allocate (wo_sz=<value optimized out>) at freelist.c:171
#1  0x00000000007c33f9 in caml_alloc_shr (wosize=5, tag=247) at memory.c:409
#2  0x00000000007c2628 in caml_oldify_one (v=140737353752592, p=0x7fffff400f08) at minor_gc.c:130
#3  0x00000000007c02d3 in caml_oldify_local_roots () at roots.c:203
#4  0x00000000007c2840 in caml_empty_minor_heap () at minor_gc.c:229
#5  0x00000000007c2969 in caml_minor_collection () at minor_gc.c:272
#6  0x00000000007c0e51 in caml_garbage_collection () at signals_asm.c:68
#7  0x00000000007cfc32 in caml_call_gc ()
#8  0x4178dfb350000000 in ?? ()
#9  0x41d303f4a85a6a55 in ?? ()
#10 0x40f2eddb4aeeae5c in ?? ()
#11 0x4171af416ff5f186 in ?? ()
#12 0x40ede93d997081cb in ?? ()
#13 0x3e1460aa5c6654d8 in ?? ()
#14 0x3e2318142703c7d0 in ?? ()
#15 0x4170000000000000 in ?? ()
#16 0x0000000000000000 in ?? ()

I rebuild ocaml with gcc -g to get this much information. My process grows to about 57MB before the crash occurs. I don't think it's an out-of-memory condition, because the app would grow to many GB before the memory flushing change was added. Unfortunately, my attempts to reproduce the problem with a small test case have failed so far (my app is rather large and uses ocsigen and lwt).

Perhaps the problem relates to the fact that my memory flusher iterates through some large Hashtbls acting as a cache, removing entries in order to reduce the footprint. (Note that this seems very similar to the bug described here: http://caml.inria.fr/mantis/print_bug_page.php?bug_id=2893 although I can't reproduce that specific problem either.)

My questions are:

- what is the best way to debug this problem?
- is it safe to call Hashtbl.remove from within Hashtable.iter and/or a gc alarm?
- is there a better way to be notified of memory pressure, or to build a cache in general? -- I've considered weak hashtables, but route seems to require another table to prevent all entries from being evacuated during gc (I don't want to rely exclusively on roots from active transactions within my app to prevent uncaching.)

Thanks for any help on these issues,

Warren