RE: [Caml-list] Segmentation fault right after caml_alloc_custom

[Lin Hong <lhong@amnh.org>]

if we reload custom block again after caml_alloc_custom for the new one, it went fine. 

looks like you are right. the free() call is just because GC is moving memory around. I will try allocate pointer only and see what happen.

thank you very much.


Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: Gerd Stolpmann [info@gerd-stolpmann.de]
Sent: Wednesday, August 01, 2012 6:11 PM
To: Lin Hong
Cc: Mailing OCaML
Subject: AW: [Caml-list] Segmentation fault right after caml_alloc_custom

Hmmm, this looks all correct. You mention that there are pointers
inside eltarr. Note that memory allocated with caml_alloc_custom can be
moved around by the GC. So if you have pointers in the custom block
pointing to another part of the block, these will become meaningless
after such a move.

The more standard solution is to put eltarr_p into the custom block,
not eltarr as a whole.

Gerd

Am 01.08.2012 23:44:45 schrieb(en) Lin Hong:
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: caml-list-request@inria.fr [caml-list-request@inria.fr] on
> behalf of Gerd Stolpmann [info@gerd-stolpmann.de]
> Sent: Wednesday, August 01, 2012 5:24 PM
> To: Mailing OCaML
> Subject: AW: [Caml-list] Segmentation fault right after
> caml_alloc_custom
>
> Am 01.08.2012 23:08:12 schrieb(en) Lin Hong:
> > Hi Gerd,
> >
> > thank you for the reply.
> >
> > how do we know if some value is registered as local root or not?
>
> Easy: It is your task as developer of a C external to register values,
> so if you don't do it, it is not done.
>
> > the code is long so I will just paste some of it first
> >
> > in that c file, the value we tried to access but failed was also
> > allocated by caml_alloc_custom(ops, size, used, max).
> >
> > caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof (struct
> > elt_arr)+sz, 1,alloc_custom_max);
> >
> > the ops was custom_operations with all standard options as follow:
> >
> > static struct custom_operations sankoff_custom_operations_eltarr = {
> >     "http://www.amnh.org/poy/",
> >     custom_finalize_default,
> >     custom_compare_default,
> >     custom_hash_default,
> >     custom_serialize_default,
> >     custom_deserialize_default
> > };
> >
> >
> > the size in caml_alloc_custom is set to the full size , that is the
> > data structure itself, plus everything its pointers point to.
>
> This is probably all ok. Did you use the CAMLparam and CAMLlocal
> macros?
>
>
> yes,  we do, just like following one
>
> value
> sankoff_CAML_median_3(value a, value n, value l, value r) {
>    CAMLparam4(a,n,l,r);
>    CAMLlocal1(res);
>
>    //load a,n,l,r to eapA,eapN,eapL and eapR
>    eltarr_p eapA;
>    eltarr_p eapN;
>    eltarr_p eapL;
>    eltarr_p eapR;
>    Sankoff_eltarr_custom_val(eapA,a);
>
> sankoff_init_eltarr(eapA,eapA->num_states,eapA->num_elts,-1,-1,-1,-1,NULL,0);
>    Sankoff_eltarr_custom_val(eapN,n);
>
> sankoff_init_eltarr(eapN,eapN->num_states,eapN->num_elts,-1,-1,-1,-1,NULL,0);
>    Sankoff_eltarr_custom_val(eapL,l);
>
> sankoff_init_eltarr(eapL,eapL->num_states,eapL->num_elts,-1,-1,-1,-1,NULL,0);
>    Sankoff_eltarr_custom_val(eapR,r);
>
> sankoff_init_eltarr(eapR,eapR->num_states,eapR->num_elts,-1,-1,-1,-1,NULL,0);
>
>
> //we can print everything in eapA~eapR here, they look fine
>
>    //create a new one for return
>    res = caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof
> (struct elt_arr)+sz, 1,alloc_custom_max);
>
> //if we print eapA~eapR at this point again, they are totally crap.
>
>    neweltarr = Sankoff_eltarr_pointer(res);
>    sankoff_median_3(eapA,eapN,eapL,eapR,neweltarr);
>
>    CAMLreturn(res);
> }
>
> some explanations on functions:
>
> [Sankoff_eltarr_custom_val] is defined in head file like this:
>
> #define Sankoff_eltarr_pointer(a) ((struct elt_arr *)
> Data_custom_val(a))
> #define Sankoff_eltarr_custom_val(to_asgn,a) to_asgn =
> Sankoff_eltarr_pointer(a)
>
> [sankoff_init_eltarr] set pointers to their right position, since we
> are loading the full chunk of memory. not sure it's necessary, but
> there is no harm to make sure those pointers are correct.
>
>
>
> Gerd
>
>
> >
> > we  also debug with valgrind,  the feed back suggest the memory we
> > tried to access is indeed freed by caml gc
> >
> > ==64563== Invalid read of size 4
> > ==64563==    at 0x38477B: sankoff_median_3 (in ../poy.native)
> > ==64563==    by 0x386125: sankoff_CAML_median_3 (in ../poy.native)
> > ==64563==    by 0x13177D: camlSankCS__median_3_196 (in
> ../poy.native)
> > ==64563==  Address 0x8bb0c04 is 81,108 bytes inside a block of size
> > 512,016 free'd
> > ==64563==    at 0xAFD7BF: free (vg_replace_malloc.c:325)
> > ==64563==    by 0x3E5AC2: caml_free_for_heap (in ../poy.native)
> > ==64563==    by 0x3E5C31: caml_shrink_heap (in ../poy.native)
> > ==64563==    by 0x3F16FC: caml_compact_heap (in ../poy.native)
> > ==64563==    by 0x3F1A89: caml_compact_heap_maybe (in ../poy.native)
> > ==64563==    by 0x3E4EBC: caml_major_collection_slice (in
> > ../poy.native)
> > ==64563==    by 0x3E5692: caml_minor_collection (in ../poy.native)
> > ==64563==    by 0x3E57E0: caml_check_urgent_gc (in ../poy.native)
> > ==64563==    by 0x3F209C: caml_alloc_custom (in ../poy.native)
> >
> >
> >
> > Cheers,
> > Lin Hong
> > American Museum of Natural History
> > POY website :
> > https://code.google.com/p/poy/
> > http://research.amnh.org/scicomp/scripts/download.php
> >
> > ________________________________________
> > From: Gerd Stolpmann [gerd@edgespring.com]
> > Sent: Wednesday, August 01, 2012 4:13 PM
> > To: Lin Hong
> > Cc: Mailing OCaML
> > Subject: Re: [Caml-list] Segmentation fault right after
> > caml_alloc_custom
> >
> > You did not send code, so we can only speculate. A reason for this
> > behavior could be that a value is not registered as local root, so
> the
> > GC thinks it is unused.
> >
> > That you see the problem only on some platforms can be explained by
> > the
> > different implementations of malloc, and how quickly freed memory is
> > really deallocated (i.e. given back to the kernel). I wouldn't give
> > much on this observation.
> >
> > Gerd
> >
> > Am 01.08.2012 22:05:19 schrieb(en) Lin Hong:
> > > Hi all,
> > >
> > > we have a weird problem here. In a c file, right after one call of
> > > 'caml_alloc_custom', some old information we have in memory is
> gone,
> > > then we have the segmentation fault because the code try to access
> > > those informations later.
> > >
> > >  turn on debug message with OCAMLRUNPARAM, get this right before
> it
> > > crash.
> > >
> > > ......
> > > $Starting new major GC cycle
> > > Marking 9223372036854775807 words
> > > Subphase = 10
> > > Sweeping 9223372036854775807 words
> > > Compacting heap...
> > > Shrinking heap to 37696k bytes
> > > Shrinking heap to 36704k bytes
> > > Shrinking heap to 35712k bytes
> > > Shrinking heap to 34720k bytes
> > > Shrinking heap to 33728k bytes
> > > Shrinking heap to 32736k bytes
> > > Shrinking heap to 31744k bytes
> > > Shrinking heap to 30752k bytes
> > > done.
> > >
> > > is it possible that the ocaml garbage collection free memory that
> is
> > > still in use?
> > >
> > > the data cause this is kind of big, it doesn't happen right
> away,and
> > > it doesn't happen on every machine (linux with ocaml4 is fine,
> > > ocaml3.xx is not. mac with ocaml4 still shows the problem though).
> > >
> > > any suggestion is appreciated.
> > >
> > >
> > > Thanks,
> > > Lin
> > >
> > >
> > > =======================
> > > American Museum of Natural History
> > > POY website :
> > > https://code.google.com/p/poy/
> > > http://research.amnh.org/scicomp/scripts/download.php
> > >
> > > --
> > > Caml-list mailing list.  Subscription management and archives:
> > > https://sympa-roc.inria.fr/wws/info/caml-list
> > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > > Bug reports: http://caml.inria.fr/bin/caml-bugs
> > >
> > >
> >
> >
> > --
> > Caml-list mailing list.  Subscription management and archives:
> > https://sympa-roc.inria.fr/wws/info/caml-list
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> >
> >
>
> --
> ------------------------------------------------------------
> Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
> Creator of GODI and camlcity.org.
> Contact details:        http://www.camlcity.org/contact.html
> Company homepage:       http://www.gerd-stolpmann.de
> ------------------------------------------------------------
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa-roc.inria.fr/wws/info/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>



--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------