[Caml-list] Question about register_global_root

[Caml-list] Question about register_global_root

[Mary F. Fernandez]

Hi Caml experts,
We are using register_global_root to register addresses in the malloc'd
C heap that contain Caml values.  We noticed that register_global_root
(comment below) expects a global C variable, but we assumed that it should
work with malloc'd addresses as well.  Is there any reason that it would not?

/* [register_global_root] registers a global C variable as a memory root
    for the duration of the program, or until [remove_global_root] is
    called. */

We are getting (somewhat random) core dumps after calling the same Caml
function repeatedly on the same Caml value (which is referenced
from the C heap).

Thanks,
Mary
-- 
Mary Fernandez, Principal Technical Staff Member
AT&T Labs - Research, 180 Park Ave., Room E243, Florham Park, NJ 07932-0971
phone: 973-360-8679,  fax: 973-360-8187
mff@research.att.com, http://www.research.att.com/~mff


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Alexander V. Voinov]

Hi All

Mary F. Fernandez wrote:

> We are getting (somewhat random) core dumps after calling the same Caml
> function repeatedly on the same Caml value (which is referenced
> from the C heap).

Me too. I didn't report this because I couldn't narrow it down to a smal 
reproduceable example. Just didn't have enough time. In my example 
register_global_root was used heavily.

Alexander


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Maas-Maarten Zeeman]

Hello,

I'm by far no Caml expert, but it works pretty well for me in 
OcamlExpat, an OCaml wrapper for Expat I wrote a while back. I use a 
malloced value to store a pointer to a tuple, which is used to store 
event handlers (ocaml functions).

This is what I did:

  /* Malloc a value for a tuple which will contain the callback
   * handlers and register it as global root.
   */
  handlers = (value *) malloc(sizeof(value));
  register_global_root(handlers);

  *handlers = alloc_tuple(NUM_HANDLERS);
  for(i = 0; i < NUM_HANDLERS; i++) {
    Field(*handlers, i) = Val_unit;
  }

One thing to keep in mind is that you have to be really careful, and use 
the right CAMLxxxx macro in the right place, otherwise things start 
crashing at seemingly random times. The reason for this is that the 
global root will correctly stay at the same memory location, but any 
ocaml values stored in the root can move during a garbage collection 
cycle. If you do not use the CAMLxxxx macro's correctly the values will 
change, and move in the middle of your c-function, which almost always 
results in a crashing program.

It is very easy to make mistakes here, but there is a way to (sort of) 
test if your c-extension behaves correctly. If you call "Gc.full_major 
()" in your tests you will know almost instantly if there is a problem 
related to garbage collection, and possible incorrect use of CAMLxxxx 
macros (at least that is the idea).

Regards,

Maas

> Hi Caml experts,
> We are using register_global_root to register addresses in the malloc'd
> C heap that contain Caml values.  We noticed that register_global_root
> (comment below) expects a global C variable, but we assumed that it 
> should
> work with malloc'd addresses as well.  Is there any reason that it 
> would not?
>
> /* [register_global_root] registers a global C variable as a memory root
>    for the duration of the program, or until [remove_global_root] is
>    called. */
>
> We are getting (somewhat random) core dumps after calling the same Caml
> function repeatedly on the same Caml value (which is referenced
> from the C heap).
>
> Thanks,
> Mary




-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Jerome Simeon]

Thanks Maas,
We managed to pin down that *!@#! bug after all. You mail was very
helpful.
- Jerome

On Wed, 2003-05-28 at 16:33, Maas-Maarten Zeeman wrote:
> Hello,
> 
> I'm by far no Caml expert, but it works pretty well for me in 
> OcamlExpat, an OCaml wrapper for Expat I wrote a while back. I use a 
> malloced value to store a pointer to a tuple, which is used to store 
> event handlers (ocaml functions).
> 
> This is what I did:
> 
>   /* Malloc a value for a tuple which will contain the callback
>    * handlers and register it as global root.
>    */
>   handlers = (value *) malloc(sizeof(value));
>   register_global_root(handlers);
> 
>   *handlers = alloc_tuple(NUM_HANDLERS);
>   for(i = 0; i < NUM_HANDLERS; i++) {
>     Field(*handlers, i) = Val_unit;
>   }
> 
> One thing to keep in mind is that you have to be really careful, and use 
> the right CAMLxxxx macro in the right place, otherwise things start 
> crashing at seemingly random times. The reason for this is that the 
> global root will correctly stay at the same memory location, but any 
> ocaml values stored in the root can move during a garbage collection 
> cycle. If you do not use the CAMLxxxx macro's correctly the values will 
> change, and move in the middle of your c-function, which almost always 
> results in a crashing program.
> 
> It is very easy to make mistakes here, but there is a way to (sort of) 
> test if your c-extension behaves correctly. If you call "Gc.full_major 
> ()" in your tests you will know almost instantly if there is a problem 
> related to garbage collection, and possible incorrect use of CAMLxxxx 
> macros (at least that is the idea).
> 
> Regards,
> 
> Maas
> 
> > Hi Caml experts,
> > We are using register_global_root to register addresses in the malloc'd
> > C heap that contain Caml values.  We noticed that register_global_root
> > (comment below) expects a global C variable, but we assumed that it 
> > should
> > work with malloc'd addresses as well.  Is there any reason that it 
> > would not?
> >
> > /* [register_global_root] registers a global C variable as a memory root
> >    for the duration of the program, or until [remove_global_root] is
> >    called. */
> >
> > We are getting (somewhat random) core dumps after calling the same Caml
> > function repeatedly on the same Caml value (which is referenced
> > from the C heap).
> >
> > Thanks,
> > Mary
> 
> 
-- 
Jerome Simeon <simeon@research.bell-labs.com>
Lucent Technologies

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Chris Hecker]

>  handlers = (value *) malloc(sizeof(value));
>  register_global_root(handlers);

Shouldn't you init *handlers to Val_unit before calling 
register_global_root() as well?

Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Maas-Maarten Zeeman]

  Chris Hecker wrote:

>
>>  handlers = (value *) malloc(sizeof(value));
>>  register_global_root(handlers);
>
>
> Shouldn't you init *handlers to Val_unit before calling 
> register_global_root() as well?

The manual explicitly tells that register_global_root should be called 
before any valid value is stored in it for the first time, like 
Val_unit. See Rule 4 of Interfacing C with Objective Caml in the manual.

So I think it is ok the way it is implemented.

Maas



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Chris Hecker]

>The manual explicitly tells that register_global_root should be called 
>before any valid value is stored in it for the first time, like Val_unit. 
>See Rule 4 of Interfacing C with Objective Caml in the manual.

Yes, but you don't store immediately, you call the alloc_tuple:

>  handlers = (value *) malloc(sizeof(value));
>  register_global_root(handlers);
>  *handlers = alloc_tuple(NUM_HANDLERS);

If alloc_tuple has to collect, then the gc will scan *handlers but it is 
uninitialized data.

If you called alloc_tuple above the register and stored it in a CAMLlocal 
variable, then just stuffed *handler immediately with that value after 
register I could see it working, but I don't see how the current code isn't 
a possible error.

Maybe I'm misunderstanding how the GC works or something?

Chris


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Damien Doligez]

On Sunday, June 1, 2003, at 12:50 PM, Maas-Maarten Zeeman wrote:

>  Chris Hecker wrote:
>
>>
>>>  handlers = (value *) malloc(sizeof(value));
>>>  register_global_root(handlers);
>>
>>
>> Shouldn't you init *handlers to Val_unit before calling 
>> register_global_root() as well?
>
> The manual explicitly tells that register_global_root should be called 
> before any valid value is stored in it for the first time, like 
> Val_unit. See Rule 4 of Interfacing C with Objective Caml in the 
> manual.
>
> So I think it is ok the way it is implemented.

I'm afraid Chris is right and the manual is wrong.

You can store Val_unit before you call register_global_root, because
Val_unit is not a pointer into the Caml heap.
You must store a valid value before the first allocation that follows
register_global_root.
You can store an allocated value (a pointer into the Caml heap) before
or after calling register_global_root, but it must be after the last
allocation that precedes register_global_root, and before the first
allocation that succeeds it.

At any rate, storing Val_unit before you call register_global_root is
safe and does not cost much CPU time.

-- Damien

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Jerome Simeon]

On Sun, 2003-06-01 at 15:53, Damien Doligez wrote:
> On Sunday, June 1, 2003, at 12:50 PM, Maas-Maarten Zeeman wrote:
> 
> >  Chris Hecker wrote:
> >
> >>
> >>>  handlers = (value *) malloc(sizeof(value));
> >>>  register_global_root(handlers);
> >>
> >>
> >> Shouldn't you init *handlers to Val_unit before calling 
> >> register_global_root() as well?
> >
> > The manual explicitly tells that register_global_root should be called 
> > before any valid value is stored in it for the first time, like 
> > Val_unit. See Rule 4 of Interfacing C with Objective Caml in the 
> > manual.
> >
> > So I think it is ok the way it is implemented.
> 
> I'm afraid Chris is right and the manual is wrong.
> 

Us poor developers could use a little improvement in the doc in this
area. Some additional example doing this feature in the doc would
help...
- Jerome

> You can store Val_unit before you call register_global_root, because
> Val_unit is not a pointer into the Caml heap.
> You must store a valid value before the first allocation that follows
> register_global_root.
> You can store an allocated value (a pointer into the Caml heap) before
> or after calling register_global_root, but it must be after the last
> allocation that precedes register_global_root, and before the first
> allocation that succeeds it.
> 
> At any rate, storing Val_unit before you call register_global_root is
> safe and does not cost much CPU time.
> 
> -- Damien
-- 
Jerome Simeon <simeon@research.bell-labs.com>
Lucent Technologies

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Maas-Maarten Zeeman]

 >
 >
 >Us poor developers could use a little improvement in the doc in this
 >area. Some additional example doing this feature in the doc would
 >help...
 >
For sure!

Btw. does anybody know if it is possible/allowed to call 'compact_heap'
inside a c stub? Somebody privately suggested to add a call to this
function to check for garbage collection problems.

The strange thing is that I'm getting reproducable crashes if I call it,
and also use String.sub, or String.copy in the ocaml program which calls
the stubs. Just passing a string is generally ok.

Maas



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Am I mad?: OCaml for scientific scripting

[Siegfried Gonzi]

How many of you guys are using OCaml for scripting? I mean I am faced 
now with the following problem: I have a Fortran program which relies 
more or less on input files and produces output files. This is good news 
because there is no need to write foreign function interfaces; all what 
is needed is to use a system command in order to start the program.

This sounds easy, but my brain is screaming whether I should really use 
OCaml for the task: scripting. It is by no means  a killer application 
but I am playing with the thought to make it available to a specialized 
community, eventually. So, it is science and I can use my tools what I 
think are best for the job, but believe it or not I am unsure:

a) Bigloo
b) Python
c) OCaml
[d) Clean but someone should immediately shot me for this stupid idea: 
Clean is dead, dead, dead, dead, dead and you will not get any help]

I wouldn't hesitate to use Python. I wouldn't hesitate to use Bigloo 
(Scheme), but I am really not that sure whether it would be fair to use 
Ocaml for that task. Why? May I really expect from a colleague that he 
settles on OCaml? Okay, nobody will ever ask  whether they are ready to 
settle on C++ and you may not forget: C++ is a huge language.  All the 
newer projects in science are more or less exclusively done in C++ (see 
for example ROOT in Cern). But is it legal to say he should also become 
interested in OCaml if he wants to use my software?

I estimate the learning curve for Ocaml as big as C++.

I am now a bit irritated about myself, because I have always thought the 
C++, Java,...imperative, and devil knows  bigots are ignorant and we 
know the stories from the managers who do not recognise the good 
software practise: functional programming (which in reality cannot show 
its cutting edge, because there is no such a thing).

I am not into mainstream and do not have problems to use my tools. For 
example I use Linux and do not even have access at my working place at 
the university to Windows.  But why is it that hard for me to use Ocaml? 
Look I had the chance to.
Others complain all day long that they are forced to use C++ and would 
feel like in heaven if they could use OCaml/Haskell/....

S. Gonzi

>
>
>



-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Am I mad?: OCaml for scientific scripting

[Siegfried Gonzi]

How many of you guys are using OCaml for scripting? I mean I am faced 
now with the following problem: I have a Fortran program which relies 
more or less on input files and produces output files. This is good news 
because there is no need to write foreign function interfaces; all what 
is needed is to use a system command in order to start the program.

This sounds easy, but my brain is screaming whether I should really use 
OCaml for the task: scripting. It is by no means  a killer application 
but I am playing with the thought to make it available to a specialized 
community, eventually. So, it is science and I can use my tools what I 
think are best for the job, but believe it or not I am unsure:

a) Bigloo
b) Python
c) OCaml
[d) Clean but someone should immediately shot me for this stupid idea: 
Clean is dead, dead, dead, dead, dead and you will not get any help]

I wouldn't hesitate to use Python. I wouldn't hesitate to use Bigloo 
(Scheme), but I am really not that sure whether it would be fair to use 
Ocaml for that task. Why? May I really expect from a colleague that he 
settles on OCaml? Okay, nobody will ever ask  whether they are ready to 
settle on C++ and you may not forget: C++ is a huge language.  All the 
newer projects in science are more or less exclusively done in C++ (see 
for example ROOT in Cern). But is it legal to say he should also become 
interested in OCaml if he wants to use my software?

I estimate the learning curve for Ocaml as big as C++.

I am now a bit irritated about myself, because I have always thought the 
C++, Java,...imperative, and devil knows  bigots are ignorant and we 
know the stories from the managers who do not recognise the good 
software practise: functional programming (which in reality cannot show 
its cutting edge, because there is no such a thing).

I am not into mainstream and do not have problems to use my tools. For 
example I use Linux and do not even have access at my working place at 
the university to Windows.  But why is it that hard for me to use Ocaml? 
Look I had the chance to.
Others complain all day long that they are forced to use C++ and would 
feel like in heaven if they could use OCaml/Haskell/....

S. Gonzi


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Harry Chomsky]

Damien Doligez wrote:
> On Sunday, June 1, 2003, at 12:50 PM, Maas-Maarten Zeeman wrote:
> > The manual explicitly tells that register_global_root should be called
> > before any valid value is stored in it for the first time, like
> > Val_unit. See Rule 4 of Interfacing C with Objective Caml in the
> > manual.
>
> I'm afraid Chris is right and the manual is wrong.
> [...]
> You must store a valid value before the first allocation that follows
> register_global_root.

I think that both Chris and the manual are right.  The manual actually
instructs the programmer to register a global root v "just before a valid
value is stored in v for the first time".  Key word: "just".  In other
words, as Damien says, no allocations are allowed between registration and
storing a value.

Apparently more than one reader of the manual has missed this detail.  Would
it be clearer to say "immediately before" instead of "just before"?

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Maas-Maarten Zeeman]

Harry Chomsky wrote:

>Damien Doligez wrote:
>  
>
>>On Sunday, June 1, 2003, at 12:50 PM, Maas-Maarten Zeeman wrote:
>>    
>>
>>>The manual explicitly tells that register_global_root should be called
>>>before any valid value is stored in it for the first time, like
>>>Val_unit. See Rule 4 of Interfacing C with Objective Caml in the
>>>manual.
>>>      
>>>
>>I'm afraid Chris is right and the manual is wrong.
>>[...]
>>You must store a valid value before the first allocation that follows
>>register_global_root.
>>    
>>
>
>I think that both Chris and the manual are right.  The manual actually
>instructs the programmer to register a global root v "just before a valid
>value is stored in v for the first time".  Key word: "just".  In other
>words, as Damien says, no allocations are allowed between registration and
>storing a value.
>
>Apparently more than one reader of the manual has missed this detail.  Would
>it be clearer to say "immediately before" instead of "just before"?
>  
>
Maybe it is better to be even more explicit.

Registration of a global variable v is achieved by calling 
register_global_root(&v), with v initialized to Val_unit, before 
allocating and storing a valid value in v for the first time.

On the other hand, some ocaml libraries (which are also examples) which 
use register_global_root, do not always follow this rule. i.e. 
install_signal_handler uses it like this:

  signal_handlers = alloc(NSIG, 0);
  register_global_root(&signal_handlers);

Is this also allowed?

All this also raises a question about the remove_global_root call, are 
there any special precautions I need to take just before or just after 
calling remove_global_root? Or can it be called at any time?

Maas


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Chris Hecker]

>  signal_handlers = alloc(NSIG, 0);
>  register_global_root(&signal_handlers);
>Is this also allowed?

If signal_handlers is CAMLlocal (or equivalent).  Although since 
register_global_root doesn't [currently] call the gc, it's probably okay as 
is, but why risk it?

Perhaps all of the c-callable runtime functions should have a special 
prefix or suffix if they can cause a collection, and then the docs can be 
clear about what you can do before/after/between/whatever these specially 
marked calls.  Of course, then you'd have to transition everybody to the 
new calls, which is a pain (or you could make it an #ifdef in the headers 
like STRICT on win32).

Chris

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Question about register_global_root

[Florian Douetteau]

As far as i understand the caml gc, it only scans blocks pointing to memory
blocks allocated through caml allocation functions
The reason of this is to allow pointers C values allocated C heap to be
pointed directly by caml values
(As shown in the unix-curses exemple at
http://caml.inria.fr/ocaml/htmlman/manual032.html#htoc224)

if you try to allocate C values in C malloc heap, it just won't work (your C
values won't be scanned, and those allocated in the original caml heap
only pointed by your caml values in C heap, will be reclaimed too soon)

Hope this helps ...


----- Original Message ----- 
From: "Mary F. Fernandez" <mff@research.att.com>
To: <caml-list@inria.fr>; "Jerome Simeon" <simeon@research.bell-labs.com>
Sent: Wednesday, May 28, 2003 9:45 PM
Subject: [Caml-list] Question about register_global_root


> Hi Caml experts,
> We are using register_global_root to register addresses in the malloc'd
> C heap that contain Caml values.  We noticed that register_global_root
> (comment below) expects a global C variable, but we assumed that it should
> work with malloc'd addresses as well.  Is there any reason that it would
not?
>
> /* [register_global_root] registers a global C variable as a memory root
>     for the duration of the program, or until [remove_global_root] is
>     called. */
>
> We are getting (somewhat random) core dumps after calling the same Caml
> function repeatedly on the same Caml value (which is referenced
> from the C heap).
>
> Thanks,
> Mary
> -- 
> Mary Fernandez, Principal Technical Staff Member
> AT&T Labs - Research, 180 Park Ave., Room E243, Florham Park, NJ
07932-0971
> phone: 973-360-8679,  fax: 973-360-8187
> mff@research.att.com, http://www.research.att.com/~mff
>
>
> -------------------
> To unsubscribe, mail caml-list-request@inria.fr Archives:
http://caml.inria.fr
> Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ:
http://caml.inria.fr/FAQ/
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners