no_scan_tag and int array

no_scan_tag and int array

[ygrek]

Hello,

Consider this code:

open Printf

let measure f =
  let t = Unix.gettimeofday () in
  let () = f () in
  printf "%.4f sec" (Unix.gettimeofday () -. t)

let () =
  let gc () = for i = 1 to 10 do Gc.full_major () done in
  let a = Array.make 4_000_000 0 in
  measure gc;
  printf " normal %u (%u)\n%!" (Array.length a) (Gc.stat ()).Gc.live_words;

  Obj.set_tag (Obj.repr a) (Obj.no_scan_tag);
  measure gc;
  printf " no_scan_tag %u (%u)\n%!" (Array.length a) (Gc.stat ()).Gc.live_words;

  measure gc;
  printf " no array (%u)\n%!" (Gc.stat ()).Gc.live_words;
  ()

Output looks like :

0.2281 sec normal 4000000 (4000165)
0.0002 sec no_scan_tag 4000000 (4000165)
0.0002 sec no array (164)

So, as expected, setting No_scan_tag on the array of integers prevents GC from uselessly 
scanning the huge chunk of memory. Looks like polymorphic array functions still work fine and
GC correctly reclaims array memory when it is not referenced anymore.
Apparantly this trick is not allowed for float array as they have a special tag set.
The question is - how safe is this? And even more, could the compiler itself set this tag?

-- 
 ygrek
 http://ygrek.org.ua

Re: [Caml-list] no_scan_tag and int array

[Goswin von Brederlow]

ygrek <ygrekheretix@gmail.com> writes:

> Hello,
>
> Consider this code:
>
> open Printf
>
> let measure f =
>   let t = Unix.gettimeofday () in
>   let () = f () in
>   printf "%.4f sec" (Unix.gettimeofday () -. t)
>
> let () =
>   let gc () = for i = 1 to 10 do Gc.full_major () done in
>   let a = Array.make 4_000_000 0 in
>   measure gc;
>   printf " normal %u (%u)\n%!" (Array.length a) (Gc.stat ()).Gc.live_words;
>
>   Obj.set_tag (Obj.repr a) (Obj.no_scan_tag);
>   measure gc;
>   printf " no_scan_tag %u (%u)\n%!" (Array.length a) (Gc.stat ()).Gc.live_words;
>
>   measure gc;
>   printf " no array (%u)\n%!" (Gc.stat ()).Gc.live_words;
>   ()
>
> Output looks like :
>
> 0.2281 sec normal 4000000 (4000165)
> 0.0002 sec no_scan_tag 4000000 (4000165)
> 0.0002 sec no array (164)
>
> So, as expected, setting No_scan_tag on the array of integers prevents GC from uselessly 
> scanning the huge chunk of memory. Looks like polymorphic array functions still work fine and

What polymorphic array function? I don't think Array.length is a good
test as it looks at the header of the array to see how many words the
array has.

> GC correctly reclaims array memory when it is not referenced anymore.
> Apparantly this trick is not allowed for float array as they have a special tag set.

But does the GC scan float arrays?

> The question is - how safe is this? And even more, could the compiler itself set this tag?

>From the GC side this should be perfectly save. But all the array
functions should be checked and not just Array.length before doing this.

And yes, the compiler could mark arrays with primitive types (int, char,
unit,...) and even type x = A | B | C for not scanning. This also
applies to records, tuples and (polymorphic) variant types that contain
no pointer.

It would break though if someone declares an int array and then stores a
pointer in there. But if you do something illegal then you get to keep
the segfaults.

MfG
        Goswin

Re: [Caml-list] no_scan_tag and int array

[ygrek]

On Sun, 07 Mar 2010 01:48:51 +0100
Goswin von Brederlow <goswin-v-b@web.de> wrote:

> What polymorphic array function? I don't think Array.length is a good
> test as it looks at the header of the array to see how many words the
> array has.

Sure, I've tested this elsewhere with Array.iter and other. Array.length here is just to
keep the array alive.

> > GC correctly reclaims array memory when it is not referenced anymore.
> > Apparantly this trick is not allowed for float array as they have a special tag set.
> 
> But does the GC scan float arrays?

No.

-- 
 ygrek
 http://ygrek.org.ua

Re: [Caml-list] no_scan_tag and int array

[Damien Doligez]

Hello,

On 2010-03-06, at 10:26, ygrek wrote:

> So, as expected, setting No_scan_tag on the array of integers prevents GC from uselessly 
> scanning the huge chunk of memory. Looks like polymorphic array functions still work fine and
> GC correctly reclaims array memory when it is not referenced anymore.
> Apparantly this trick is not allowed for float array as they have a special tag set.

The trick is not needed for float arrays, the GC already doesn't scan
them.

> The question is - how safe is this?

It's safe, and will be in the forseeable future.

BUT: you should use Abstract_tag and not No_scan_tag.  Abstract_tag means
"don't make assumptions about the contents of this block", while
No_scan_tag is just the min of all the tags that the GC is not supposed
to scan.  Right now they are equal, but a future version of OCaml might
have No_scan_tag = Double_array_tag, which would break your code.

> And even more, could the compiler itself set this tag?

This is a bit tricky because you have to make sure that the static
type of the array is "int array".  Unlike floats, a run-time test
at allocation will not work.

You should enter this as a feature wish in the BTS.

-- Damien