GC troubleshooting

GC troubleshooting

[Mike Lin]

Hi folks,

I'm not very experienced in troubleshooting fine GC issues, and
unfortunately I have a situation where the GC is really grinding me to
a halt. My program has a loop that needs to run about 20,000 times,
allocating a handful of lists, roughly between 10 and 100 in length,
in each iteration.

The first 8,000 or so iterations run fine, in a few minutes. Somewhere
around there, the GC starts to tear me a new one. I incompetently
played with different settings for the heap sizes, but I don't really
know what I'm doing. When I set the GC output to verbose, I see the
following type of output when it slows down:

$<>allocated_words = 77096
extra_heap_resources = 0u
amount of work to do = 1240u
ordered work = 0 words
computed work = 520396 words
Sweeping 520396 words
$<>allocated_words = 74450
extra_heap_resources = 0u
amount of work to do = 1198u
ordered work = 0 words
computed work = 502536 words
Sweeping 502536 words
$<>allocated_words = 71702
extra_heap_resources = 0u
amount of work to do = 1153u
ordered work = 0 words
computed work = 483988 words
Sweeping 483988 words
$<>allocated_words = 68841
extra_heap_resources = 0u
amount of work to do = 1107u
ordered work = 0 words
computed work = 464676 words
Sweeping 464676 words
...

These messages continue to occur so frequently that I think the GC is
doing much more work than the program itself.

The loop is doing some reasonable (I think) operations on these lists
- filter, fold_left, rev_append, rev_map, iter, and so forth. Each
iteration uses Printf to print a line to stdout. There are probably a
couple hundred megs of static data set (not being allocated or
deallocated) also present in memory, and being used by the loop.

Anyway, let me know if you have any ideas.

Mike