RE: [Caml-list] Segmentation fault right after caml_alloc_custom

[Lin Hong <lhong@amnh.org>]

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Gerd Stolpmann [info@gerd-stolpmann.de]
Sent: Wednesday, August 01, 2012 5:24 PM
To: Mailing OCaML
Subject: AW: [Caml-list] Segmentation fault right after caml_alloc_custom

Am 01.08.2012 23:08:12 schrieb(en) Lin Hong:
> Hi Gerd,
>
> thank you for the reply.
>
> how do we know if some value is registered as local root or not?

Easy: It is your task as developer of a C external to register values,
so if you don't do it, it is not done.

> the code is long so I will just paste some of it first
>
> in that c file, the value we tried to access but failed was also
> allocated by caml_alloc_custom(ops, size, used, max).
>
> caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof (struct
> elt_arr)+sz, 1,alloc_custom_max);
>
> the ops was custom_operations with all standard options as follow:
>
> static struct custom_operations sankoff_custom_operations_eltarr = {
>     "http://www.amnh.org/poy/",
>     custom_finalize_default,
>     custom_compare_default,
>     custom_hash_default,
>     custom_serialize_default,
>     custom_deserialize_default
> };
>
>
> the size in caml_alloc_custom is set to the full size , that is the
> data structure itself, plus everything its pointers point to.

This is probably all ok. Did you use the CAMLparam and CAMLlocal macros?


yes,  we do, just like following one

value
sankoff_CAML_median_3(value a, value n, value l, value r) {
   CAMLparam4(a,n,l,r);
   CAMLlocal1(res);

   //load a,n,l,r to eapA,eapN,eapL and eapR
   eltarr_p eapA;
   eltarr_p eapN;
   eltarr_p eapL;
   eltarr_p eapR;
   Sankoff_eltarr_custom_val(eapA,a);
   sankoff_init_eltarr(eapA,eapA->num_states,eapA->num_elts,-1,-1,-1,-1,NULL,0);
   Sankoff_eltarr_custom_val(eapN,n);
   sankoff_init_eltarr(eapN,eapN->num_states,eapN->num_elts,-1,-1,-1,-1,NULL,0);
   Sankoff_eltarr_custom_val(eapL,l);
   sankoff_init_eltarr(eapL,eapL->num_states,eapL->num_elts,-1,-1,-1,-1,NULL,0);
   Sankoff_eltarr_custom_val(eapR,r);
   sankoff_init_eltarr(eapR,eapR->num_states,eapR->num_elts,-1,-1,-1,-1,NULL,0);


//we can print everything in eapA~eapR here, they look fine

   //create a new one for return
   res = caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof (struct elt_arr)+sz, 1,alloc_custom_max);

//if we print eapA~eapR at this point again, they are totally crap.

   neweltarr = Sankoff_eltarr_pointer(res);
   sankoff_median_3(eapA,eapN,eapL,eapR,neweltarr);

   CAMLreturn(res);
}

some explanations on functions:

[Sankoff_eltarr_custom_val] is defined in head file like this:

#define Sankoff_eltarr_pointer(a) ((struct elt_arr *) Data_custom_val(a))
#define Sankoff_eltarr_custom_val(to_asgn,a) to_asgn = Sankoff_eltarr_pointer(a)

[sankoff_init_eltarr] set pointers to their right position, since we are loading the full chunk of memory. not sure it's necessary, but there is no harm to make sure those pointers are correct.



Gerd


>
> we  also debug with valgrind,  the feed back suggest the memory we
> tried to access is indeed freed by caml gc
>
> ==64563== Invalid read of size 4
> ==64563==    at 0x38477B: sankoff_median_3 (in ../poy.native)
> ==64563==    by 0x386125: sankoff_CAML_median_3 (in ../poy.native)
> ==64563==    by 0x13177D: camlSankCS__median_3_196 (in ../poy.native)
> ==64563==  Address 0x8bb0c04 is 81,108 bytes inside a block of size
> 512,016 free'd
> ==64563==    at 0xAFD7BF: free (vg_replace_malloc.c:325)
> ==64563==    by 0x3E5AC2: caml_free_for_heap (in ../poy.native)
> ==64563==    by 0x3E5C31: caml_shrink_heap (in ../poy.native)
> ==64563==    by 0x3F16FC: caml_compact_heap (in ../poy.native)
> ==64563==    by 0x3F1A89: caml_compact_heap_maybe (in ../poy.native)
> ==64563==    by 0x3E4EBC: caml_major_collection_slice (in
> ../poy.native)
> ==64563==    by 0x3E5692: caml_minor_collection (in ../poy.native)
> ==64563==    by 0x3E57E0: caml_check_urgent_gc (in ../poy.native)
> ==64563==    by 0x3F209C: caml_alloc_custom (in ../poy.native)
>
>
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: Gerd Stolpmann [gerd@edgespring.com]
> Sent: Wednesday, August 01, 2012 4:13 PM
> To: Lin Hong
> Cc: Mailing OCaML
> Subject: Re: [Caml-list] Segmentation fault right after
> caml_alloc_custom
>
> You did not send code, so we can only speculate. A reason for this
> behavior could be that a value is not registered as local root, so the
> GC thinks it is unused.
>
> That you see the problem only on some platforms can be explained by
> the
> different implementations of malloc, and how quickly freed memory is
> really deallocated (i.e. given back to the kernel). I wouldn't give
> much on this observation.
>
> Gerd
>
> Am 01.08.2012 22:05:19 schrieb(en) Lin Hong:
> > Hi all,
> >
> > we have a weird problem here. In a c file, right after one call of
> > 'caml_alloc_custom', some old information we have in memory is gone,
> > then we have the segmentation fault because the code try to access
> > those informations later.
> >
> >  turn on debug message with OCAMLRUNPARAM, get this right before it
> > crash.
> >
> > ......
> > $Starting new major GC cycle
> > Marking 9223372036854775807 words
> > Subphase = 10
> > Sweeping 9223372036854775807 words
> > Compacting heap...
> > Shrinking heap to 37696k bytes
> > Shrinking heap to 36704k bytes
> > Shrinking heap to 35712k bytes
> > Shrinking heap to 34720k bytes
> > Shrinking heap to 33728k bytes
> > Shrinking heap to 32736k bytes
> > Shrinking heap to 31744k bytes
> > Shrinking heap to 30752k bytes
> > done.
> >
> > is it possible that the ocaml garbage collection free memory that is
> > still in use?
> >
> > the data cause this is kind of big, it doesn't happen right away,and
> > it doesn't happen on every machine (linux with ocaml4 is fine,
> > ocaml3.xx is not. mac with ocaml4 still shows the problem though).
> >
> > any suggestion is appreciated.
> >
> >
> > Thanks,
> > Lin
> >
> >
> > =======================
> > American Museum of Natural History
> > POY website :
> > https://code.google.com/p/poy/
> > http://research.amnh.org/scicomp/scripts/download.php
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa-roc.inria.fr/wws/info/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> >
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>

--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------
--
Caml-list mailing list.  Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs