Does the gc avoid collecting arrays of ints

Does the gc avoid collecting arrays of ints

[Till Varoquaux]

Looking at ocaml's source code I see that the values representing
arrays have a tag representing the type of their content (I'm guessing
boxed/unboxed). Does this mean that a bidimensional array containing
ints will only be explored in one direction during garbage collection?
If so, how do the compare to Bigarray's (I'm guessing they still are
slower).

Till

-- 
http://till-varoquaux.blogspot.com/

Re: [Caml-list] Does the gc avoid collecting arrays of ints

[Jon Harrop]

On Monday 23 July 2007 18:35:40 Till Varoquaux wrote:
> Looking at ocaml's source code I see that the values representing
> arrays have a tag representing the type of their content (I'm guessing
> boxed/unboxed). Does this mean that a bidimensional array containing
> ints will only be explored in one direction during garbage collection?
> If so, how do the compare to Bigarray's (I'm guessing they still are
> slower).

My gut feeling is that an array of arrays would be faster. You might also like 
to abstract away a single array behind the interface of a multidimensional 
array (particularly if you're on 64-bit).

However, there are some wierdnesses here. The GC treats the stack and arrays 
of pointers atomically, traversing all elements in one go. So having a single 
large array of boxed values (like an array of arrays or an array of lists in 
a Hashtbl) can cause significant stalls in the incremental GC.

I found this performance characteristic whilst optimizing Smoke's worst case 
performance, which is very important for such soft-real-time applications. I 
had "optimized" by memoizing stuff and things in a hash table, which turned 
out to slow the GC down enormously whenever it stumbled upon the hash table.

Incidentally, I get the distinct impression that Chistophe's question isn't 
going to get answered until the ICFP is done and dusted and everyone has 
recouperated. ;-)

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists/?e

Re: [Caml-list] Does the gc avoid collecting arrays of ints

[Till Varoquaux]

On 7/23/07, Jon Harrop <jon@ffconsultancy.com> wrote:
...
>
> My gut feeling is that an array of arrays would be faster. You might also like
> to abstract away a single array behind the interface of a multidimensional
> array (particularly if you're on 64-bit).
>
Array of arrayrequier you two jump through two indirections (they are
like **int instead of *int), bigarrays are based on a linear backend
whatever there dimension may be. Using single dimension arrays and
building an interface over it sounds like a good solution. I don't
feel like I have the time to benchmark.
For the record: I saw a tremendous performance boost going from two
dimensional arrays of tupple of ints to tupples of bigarrays. By
profilling I can tell I was loosing most of my time collecting local
values. This was probably due to:
_Manipulating a lot of tupples, wich needed to be collected.
_Assigning local variables to rows. This was probably a bad idea to start with..

> However, there are some wierdnesses here. The GC treats the stack and arrays
> of pointers atomically, traversing all elements in one go. So having a single
> large array of boxed values (like an array of arrays or an array of lists in
> a Hashtbl) can cause significant stalls in the incremental GC.
I thought OCaml did not have an incremental GC, and that minor and
major collections were both atomic.
Till
-- 
http://till-varoquaux.blogspot.com/

Re: [Caml-list] Does the gc avoid collecting arrays of ints

[Xavier Leroy]

For the record:

> I thought OCaml did not have an incremental GC, and that minor and
> major collections were both atomic.

The major collector is incremental: a slice of major GC is performed
at every minor GC.  If it weren't the case, you'd see big pauses at
major GC time.

- Xavier Leroy