Big array with data managed by another value

Big array with data managed by another value

[Romain Beauxis]

	Hi all !

In our application, we use a custom block to carry a pointer to data allocated 
by C.

We would like to use this data with functions that use Bigarrays. Hence, we 
want a [custom_to_bigarray] function that would allocate a big array, and put 
our pointer as its data.

We can pass the CAML_BA_EXTERNAL flag to the bigarray, so that the Gc doesn't 
touch data pointer when deallocating the returned array. 

However, what I don't know how to do yet is to also tell the Gc that the 
original value should not be deallocated until we have finished dealing with 
the returned bigarray.

Is there a clean way to do that ?


Romain

Re: [Caml-list] Big array with data managed by another value

[Daniel Bünzli]

Le 14 nov. 08 à 15:03, Romain Beauxis a écrit :

> We can pass the CAML_BA_EXTERNAL flag to the bigarray, so that the  
> Gc doesn't
> touch data pointer when deallocating the returned array.
>
> However, what I don't know how to do yet is to also tell the Gc that  
> the
> original value should not be deallocated until we have finished  
> dealing with
> the returned bigarray.
>
> Is there a clean way to do that ?

I'm not sure I understand what you want to do (what is the original  
value ?).

1) If you are wondering about the original data of the allocated big  
array then there is no such data if you use alloc_bigarray with  
BIGARRAY_EXTERNAL.
2) If you are wondering about how you could deallocate the C pointer  
using your own function whenever the bigarray becomes unreachable then  
register the bigarray with Gc.finalise and an appropriate function.

Best,

Daniel

Re: [Caml-list] Big array with data managed by another value

[Daniel Bünzli]

Le 14 nov. 08 à 15:42, Daniel Bünzli a écrit :

> I'm not sure I understand what you want to do (what is the original  
> value ?).

I think I understood now, the original value was the custom block.

If you were using the custom block only for finalization issues (not  
serialization, hashing etc.) then one solution would be to replace  
your custom block type by the bigarray itself. An register the  
bigarray with Gc.finalise to deallocate the C pointer at the  
appropriate time. Then you have

type t = bigarray
let to_bigarray x = x

and no problems.

Best,

Daniel

Re: [Caml-list] Big array with data managed by another value

[Romain Beauxis]

Le Friday 14 November 2008 16:10:13 Daniel Bünzli, vous avez écrit :
> I think I understood now, the original value was the custom block.
>
> If you were using the custom block only for finalization issues (not  
> serialization, hashing etc.) then one solution would be to replace  
> your custom block type by the bigarray itself. An register the  
> bigarray with Gc.finalise to deallocate the C pointer at the  
> appropriate time. Then you have
>
> type t = bigarray
> let to_bigarray x = x
>
> and no problems.

Yep, that's my current solution.
However, I see little interest to be able to allocate a big array with 
CAML_BA_EXTERNAL then.
The only situation where this would be of some interest is for using a caml 
callback from C, where you don't want the callback to finalize your data.
But you would never be able to convert a part of a custom data type to big 
array.

Thanks for your anwsers.

R.

Re: [Caml-list] Big array with data managed by another value

[Daniel Bünzli]

Le 14 nov. 08 à 16:58, Romain Beauxis a écrit :

> Yep, that's my current solution.

Another less friendly solution is to provide the bigarray in a  
callback and document the fact that the bigarray is only valid for the  
call and should not escape the scope.

val with_bigarray : custom -> (bigarray -> 'a) -> 'a

let with_bigarray c f =
   let ba = barray_of c in
   try let r = f ba in barray_invalidate ba; r with e ->  
barray_invalidate ba; raise e

barray_invalidate should set the bigarray size to 0 so that if the  
client doesn't respect the policy it will only get an out of bounds  
error (unless he compiled with -unsafe).

Best,

Daniel