Performance-question

[Oliver Bandel <oliver@first.in-berlin.de>]

Hello,

I have a concrete (non philosophical) question
on the Performance of one of my scripts.

The general behaviour: it reads data fast first,
then slows down a lot (possibly after about 1/3 of the
data; it seems to completely have stopped then), then later speeds up
again.
Ths script needs nearly complete CPU-time and the system load goes up.


Now here some data:

=====================================================

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls   s/call   s/call  name
 44.45    163.08   163.08    26627     0.01     0.01  mark_slice
 22.31    244.92    81.84    38447     0.00     0.00  sweep_slice
 14.40    297.74    52.82      131     0.40     0.61  caml_compact_heap
  3.49    310.54    12.80 372416920     0.00     0.00  invert_pointer_at
  3.40    323.02    12.48 560604006     0.00     0.00  compact_allocate
  3.12    334.45    11.43  9725772     0.00     0.00  caml_fl_allocate
  0.92    337.81     3.36    25607     0.00     0.00  caml_add_to_heap

----------------------------------------------------------------------

minor_words: 218494242
promoted_words: 38303811
major_words: 3546606791
minor_collections: 64811
major_collections: 720
heap_words: 1044480
heap_chunks: 17
top_heap_words: 47322112
live_words: 666290
live_blocks: 103159
free_words: 378186
free_blocks: 360
largest_free: 60034
fragments: 4
compactions: 131

=====================================================


For me it looks like a lot of GC-actions, but I'm not sure in that
point. What do you think is slowing down here, and how to avoid
this behaviour?

Ciao,
   Oliver