When can we ignore CAMLparam and CAMLreturn?

When can we ignore CAMLparam and CAMLreturn?

[Jianzhou Zhao]

Hi All,

Here is the code from LLVM-OCaml bindings.

/////////////////
/* llvalue -> GenericValue.t array -> ExecutionEngine.t -> GenericValue.t */
CAMLprim value llvm_ee_run_function(LLVMValueRef F, value Args,
                                   LLVMExecutionEngineRef EE) {
 unsigned NumArgs;
 LLVMGenericValueRef Result, *GVArgs;
 unsigned I;

 NumArgs = Wosize_val(Args);
 GVArgs = (LLVMGenericValueRef*) malloc(NumArgs * sizeof(LLVMGenericValueRef));
 for (I = 0; I != NumArgs; ++I)
   GVArgs[I] = Genericvalue_val(Field(Args, I));

 Result = LLVMRunFunction(EE, F, NumArgs, GVArgs);

 free(GVArgs);
 return alloc_generic_value(Result);
}
////////////////////////

In this code,  GenericValue.t is an Ocaml value with custom_operations
(say finalizing, hashing, serializing or deserializing a C type) as
shown in the below.

////////////////////////////
 #define Genericvalue_val(v)  (*(LLVMGenericValueRef *)(Data_custom_val(v)))

 static void llvm_finalize_generic_value(value GenVal) {
   LLVMDisposeGenericValue(Genericvalue_val(GenVal));
 }

 static struct custom_operations generic_value_ops = {
   (char *) "LLVMGenericValue",
   llvm_finalize_generic_value,
   custom_compare_default,
   custom_hash_default,
   custom_serialize_default,
   custom_deserialize_default
 };

 static value alloc_generic_value(LLVMGenericValueRef Ref) {
   value Val = alloc_custom(&generic_value_ops,
sizeof(LLVMGenericValueRef), 0, 1);
   Genericvalue_val(Val) = Ref;
   return Val;
 }
 ////////////////////////////////////

The 'llvm_ee_run_function' does not protect the Args parameter by
CAMLparam with CAMLreturn. Is this safe in this case, because we
allocated a GVArgs? The Ocaml manual suggests to use CAMLparam for any
value parameters
(http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc140). In
the other LLVM bindings files, the code sometimes also ignores
CAMLparam and CAMLreturn if Caml "value" refers to unit or string. Is
it also safe? for example.

/* string -> lltype -> llmodule -> llvalue */
CAMLprim LLVMValueRef llvm_define_function(value Name, LLVMTypeRef Ty,
LLVMModuleRef M) {
   LLVMValueRef Fn = LLVMAddFunction(M, String_val(Name), Ty);
   LLVMAppendBasicBlockInContext(LLVMGetTypeContext(Ty), Fn, "entry");
   return Fn;
}

I have been working on extending this Ocaml bindings, but got some
inconsistent results at runtime. I used Valgrind to check my program,
it reports some invalid writes or reads in C overlapping with some
memory space managed by Ocaml, but the report is not detailed enough.
so I doubt if the binding code does any thing wrong. Is there any tool
to help analyzing if the code manages memory between C and OCaml
bindings correctly?

Thanks
-- 
Jianzhou



-- 
Jianzhou

Re: [Caml-list] When can we ignore CAMLparam and CAMLreturn?

[Jacques Garrigue]

On 2010/10/26, at 1:19, Jianzhou Zhao wrote:

> Hi All,
> 
> Here is the code from LLVM-OCaml bindings.
> 
> /////////////////
> /* llvalue -> GenericValue.t array -> ExecutionEngine.t -> GenericValue.t */
> CAMLprim value llvm_ee_run_function(LLVMValueRef F, value Args,
>                                   LLVMExecutionEngineRef EE) {
> unsigned NumArgs;
> LLVMGenericValueRef Result, *GVArgs;
> unsigned I;
> 
> NumArgs = Wosize_val(Args);
> GVArgs = (LLVMGenericValueRef*) malloc(NumArgs * sizeof(LLVMGenericValueRef));
> for (I = 0; I != NumArgs; ++I)
>   GVArgs[I] = Genericvalue_val(Field(Args, I));
> 
> Result = LLVMRunFunction(EE, F, NumArgs, GVArgs);
> 
> free(GVArgs);
> return alloc_generic_value(Result);
> }
> ////////////////////////
> 
> The 'llvm_ee_run_function' does not protect the Args parameter by
> CAMLparam with CAMLreturn. Is this safe in this case, because we
> allocated a GVArgs? The Ocaml manual suggests to use CAMLparam for any
> value parameters (http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc140). 

The basic rule is that those macros are only needed if some allocation in the caml heap
occurs before accessing a caml value. Expressed another way, whenever allocation is
used all unprotected pointers to the caml heap should be considered as invalid.

Since in the above code there is no allocation in the caml heap before the Genericvalue_val's,
Args need not be protected. Similarly for the result, we are just returning the result of another
function, with no risk of corruption.

When in doubt, it is always safer to use CAMLparam/CAMLreturn, eventhough they will
generate a bit more code.

Jacques Garrigue

Re: [Caml-list] When can we ignore CAMLparam and CAMLreturn?

[Goswin von Brederlow]

Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> writes:

> On 2010/10/26, at 1:19, Jianzhou Zhao wrote:
>
>> Hi All,
>> 
>> Here is the code from LLVM-OCaml bindings.
>> 
>> /////////////////
>> /* llvalue -> GenericValue.t array -> ExecutionEngine.t -> GenericValue.t */
>> CAMLprim value llvm_ee_run_function(LLVMValueRef F, value Args,
>>                                   LLVMExecutionEngineRef EE) {
>> unsigned NumArgs;
>> LLVMGenericValueRef Result, *GVArgs;
>> unsigned I;
>> 
>> NumArgs = Wosize_val(Args);
>> GVArgs = (LLVMGenericValueRef*) malloc(NumArgs * sizeof(LLVMGenericValueRef));
>> for (I = 0; I != NumArgs; ++I)
>>   GVArgs[I] = Genericvalue_val(Field(Args, I));
>> 
>> Result = LLVMRunFunction(EE, F, NumArgs, GVArgs);
>> 
>> free(GVArgs);
>> return alloc_generic_value(Result);
>> }
>> ////////////////////////
>> 
>> The 'llvm_ee_run_function' does not protect the Args parameter by
>> CAMLparam with CAMLreturn. Is this safe in this case, because we
>> allocated a GVArgs? The Ocaml manual suggests to use CAMLparam for any
>> value parameters (http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc140). 
>
> The basic rule is that those macros are only needed if some allocation in the caml heap
> occurs before accessing a caml value. Expressed another way, whenever allocation is
> used all unprotected pointers to the caml heap should be considered as invalid.
>
> Since in the above code there is no allocation in the caml heap before the Genericvalue_val's,
> Args need not be protected. Similarly for the result, we are just returning the result of another
> function, with no risk of corruption.
>
> When in doubt, it is always safer to use CAMLparam/CAMLreturn, eventhough they will
> generate a bit more code.

As a special case: If you enter/leave_blocking_section() then some other
thread might (almost certainly will) do an allocation. So any ocaml
values you need in enter/leave_blocking_section() need to be copied to C
values beforehand and values you need after need to be protected.

MfG
        Goswin

Re: [Caml-list] When can we ignore CAMLparam and CAMLreturn?

[Jianzhou Zhao]

On Wed, Oct 27, 2010 at 5:36 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
> Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> writes:
>
>> On 2010/10/26, at 1:19, Jianzhou Zhao wrote:
>>
>>> Hi All,
>>>
>>> Here is the code from LLVM-OCaml bindings.
>>>
>>> /////////////////
>>> /* llvalue -> GenericValue.t array -> ExecutionEngine.t -> GenericValue.t */
>>> CAMLprim value llvm_ee_run_function(LLVMValueRef F, value Args,
>>>                                   LLVMExecutionEngineRef EE) {
>>> unsigned NumArgs;
>>> LLVMGenericValueRef Result, *GVArgs;
>>> unsigned I;
>>>
>>> NumArgs = Wosize_val(Args);
>>> GVArgs = (LLVMGenericValueRef*) malloc(NumArgs * sizeof(LLVMGenericValueRef));
>>> for (I = 0; I != NumArgs; ++I)
>>>   GVArgs[I] = Genericvalue_val(Field(Args, I));
>>>
>>> Result = LLVMRunFunction(EE, F, NumArgs, GVArgs);
>>>
>>> free(GVArgs);
>>> return alloc_generic_value(Result);
>>> }
>>> ////////////////////////
>>>
>>> The 'llvm_ee_run_function' does not protect the Args parameter by
>>> CAMLparam with CAMLreturn. Is this safe in this case, because we
>>> allocated a GVArgs? The Ocaml manual suggests to use CAMLparam for any
>>> value parameters (http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc140).
>>
>> The basic rule is that those macros are only needed if some allocation in the caml heap
>> occurs before accessing a caml value. Expressed another way, whenever allocation is
>> used all unprotected pointers to the caml heap should be considered as invalid.
>>
>> Since in the above code there is no allocation in the caml heap before the Genericvalue_val's,
>> Args need not be protected. Similarly for the result, we are just returning the result of another
>> function, with no risk of corruption.
>>
>> When in doubt, it is always safer to use CAMLparam/CAMLreturn, eventhough they will
>> generate a bit more code.
>
> As a special case: If you enter/leave_blocking_section() then some other
> thread might (almost certainly will) do an allocation. So any ocaml
> values you need in enter/leave_blocking_section() need to be copied to C
> values beforehand and values you need after need to be protected.
>
> MfG
>        Goswin
>

Does it imply that, if any Ocaml binding uses
enter/leave_blocking_section, the Ocaml runtime only switch contexts
at enter/leave_blocking_section(). If it allows other threads run at
any point, we might have to protect values in any function.

-- 
Jianzhou

Re: [Caml-list] When can we ignore CAMLparam and CAMLreturn?

[Goswin von Brederlow]

Jianzhou Zhao <jianzhou@seas.upenn.edu> writes:

> On Wed, Oct 27, 2010 at 5:36 AM, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> Jacques Garrigue <garrigue@math.nagoya-u.ac.jp> writes:
>>
>>> On 2010/10/26, at 1:19, Jianzhou Zhao wrote:
>>>
>>>> Hi All,
>>>>
>>>> Here is the code from LLVM-OCaml bindings.
>>>>
>>>> /////////////////
>>>> /* llvalue -> GenericValue.t array -> ExecutionEngine.t -> GenericValue.t */
>>>> CAMLprim value llvm_ee_run_function(LLVMValueRef F, value Args,
>>>>                                   LLVMExecutionEngineRef EE) {
>>>> unsigned NumArgs;
>>>> LLVMGenericValueRef Result, *GVArgs;
>>>> unsigned I;
>>>>
>>>> NumArgs = Wosize_val(Args);
>>>> GVArgs = (LLVMGenericValueRef*) malloc(NumArgs * sizeof(LLVMGenericValueRef));
>>>> for (I = 0; I != NumArgs; ++I)
>>>>   GVArgs[I] = Genericvalue_val(Field(Args, I));
>>>>
>>>> Result = LLVMRunFunction(EE, F, NumArgs, GVArgs);
>>>>
>>>> free(GVArgs);
>>>> return alloc_generic_value(Result);
>>>> }
>>>> ////////////////////////
>>>>
>>>> The 'llvm_ee_run_function' does not protect the Args parameter by
>>>> CAMLparam with CAMLreturn. Is this safe in this case, because we
>>>> allocated a GVArgs? The Ocaml manual suggests to use CAMLparam for any
>>>> value parameters (http://caml.inria.fr/pub/docs/manual-ocaml/manual032.html#toc140).
>>>
>>> The basic rule is that those macros are only needed if some allocation in the caml heap
>>> occurs before accessing a caml value. Expressed another way, whenever allocation is
>>> used all unprotected pointers to the caml heap should be considered as invalid.
>>>
>>> Since in the above code there is no allocation in the caml heap before the Genericvalue_val's,
>>> Args need not be protected. Similarly for the result, we are just returning the result of another
>>> function, with no risk of corruption.
>>>
>>> When in doubt, it is always safer to use CAMLparam/CAMLreturn, eventhough they will
>>> generate a bit more code.
>>
>> As a special case: If you enter/leave_blocking_section() then some other
>> thread might (almost certainly will) do an allocation. So any ocaml
>> values you need in enter/leave_blocking_section() need to be copied to C
>> values beforehand and values you need after need to be protected.
>>
>> MfG
>>        Goswin
>>
>
> Does it imply that, if any Ocaml binding uses
> enter/leave_blocking_section, the Ocaml runtime only switch contexts
> at enter/leave_blocking_section(). If it allows other threads run at
> any point, we might have to protect values in any function.
>
> -- 
> Jianzhou

It never switches threads preemptively. But I think the alloc function
potentially does switch threads. So any time you allocate some memory a
switch occurs if your thread has run enough. Something like that. This
happens all the time in the code, not just on
enter/leave_blocking_section() afaik.

MfG
        Goswin