From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by sympa.inria.fr (Postfix) with ESMTPS id BB0C97ED5C for ; Thu, 2 Aug 2012 03:01:45 +0200 (CEST) Received-SPF: None (mail1-smtp-roc.national.inria.fr: no sender authenticity information available from domain of lhong@amnh.org) identity=pra; client-ip=216.73.244.167; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="lhong@amnh.org"; x-sender="lhong@amnh.org"; x-conformance=sidf_compatible Received-SPF: None (mail1-smtp-roc.national.inria.fr: no sender authenticity information available from domain of lhong@amnh.org) identity=mailfrom; client-ip=216.73.244.167; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="lhong@amnh.org"; x-sender="lhong@amnh.org"; x-conformance=sidf_compatible Received-SPF: None (mail1-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-mx-002.amnh.org) identity=helo; client-ip=216.73.244.167; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="lhong@amnh.org"; x-sender="postmaster@mail-mx-002.amnh.org"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApIBAMXQGVDYSfSnmWdsb2JhbAArGqd3kR4iAQEBAQEICwsHFCeCIAEBAQQnEzQLEAIBCA4DBAEBCwMREDITCggCBA4FCAEJCYdjAwwBCimzRwOJWItJhilgA5ZbiBeBWgiHX4FWBw X-IronPort-AV: E=Sophos;i="4.77,697,1336341600"; d="scan'208";a="168842311" Received: from mail-mx-002.amnh.org ([216.73.244.167]) by mail1-smtp-roc.national.inria.fr with ESMTP; 02 Aug 2012 03:01:44 +0200 X-ASG-Debug-ID: 1343869301-04264159b4421930001-jBxyQn Received: from amnh.org ([172.16.8.237]) by mail-mx-002.amnh.org with ESMTP id X5Ld1dKugju41zIc; Wed, 01 Aug 2012 21:01:41 -0400 (EDT) X-Barracuda-Envelope-From: lhong@amnh.org Received: from MAIL-MBX-004.internal.amnh.org ([fe80::7070:e0a1:bc56:c30a]) by MAIL-CASHT-002.internal.amnh.org ([fe80::f48f:c9ac:48a3:c7ed%10]) with mapi id 14.01.0289.001; Wed, 1 Aug 2012 21:01:41 -0400 From: Lin Hong X-Barracuda-Apparent-Source-IP: fe80::7070:e0a1:bc56:c30a To: Gerd Stolpmann CC: Mailing OCaML Thread-Topic: [Caml-list] Segmentation fault right after caml_alloc_custom X-ASG-Orig-Subj: RE: [Caml-list] Segmentation fault right after caml_alloc_custom Thread-Index: AQHNcDKQQXA9cZOHwESkX6hsHwCJmJdFsoiJ Date: Thu, 2 Aug 2012 01:01:40 +0000 Message-ID: References: <1343856245.3275.4@samsung> ,<1343859074.3275.6@samsung> In-Reply-To: <1343859074.3275.6@samsung> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.8.250] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[172.16.8.237] X-Barracuda-Start-Time: 1343869301 X-Barracuda-URL: http://spam.amnh.org:80/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at amnh.org X-Barracuda-Spam-Score: 0.50 X-Barracuda-Spam-Status: No, SCORE=0.50 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=4.0 KILL_LEVEL=9.0 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.104419 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M Subject: RE: [Caml-list] Segmentation fault right after caml_alloc_custom if we reload custom block again after caml_alloc_custom for the new one, it= went fine.=20 looks like you are right. the free() call is just because GC is moving memo= ry around. I will try allocate pointer only and see what happen. thank you very much. Cheers, Lin Hong American Museum of Natural History POY website : https://code.google.com/p/poy/ http://research.amnh.org/scicomp/scripts/download.php ________________________________________ From: Gerd Stolpmann [info@gerd-stolpmann.de] Sent: Wednesday, August 01, 2012 6:11 PM To: Lin Hong Cc: Mailing OCaML Subject: AW: [Caml-list] Segmentation fault right after caml_alloc_custom Hmmm, this looks all correct. You mention that there are pointers inside eltarr. Note that memory allocated with caml_alloc_custom can be moved around by the GC. So if you have pointers in the custom block pointing to another part of the block, these will become meaningless after such a move. The more standard solution is to put eltarr_p into the custom block, not eltarr as a whole. Gerd Am 01.08.2012 23:44:45 schrieb(en) Lin Hong: > > Cheers, > Lin Hong > American Museum of Natural History > POY website : > https://code.google.com/p/poy/ > http://research.amnh.org/scicomp/scripts/download.php > > ________________________________________ > From: caml-list-request@inria.fr [caml-list-request@inria.fr] on > behalf of Gerd Stolpmann [info@gerd-stolpmann.de] > Sent: Wednesday, August 01, 2012 5:24 PM > To: Mailing OCaML > Subject: AW: [Caml-list] Segmentation fault right after > caml_alloc_custom > > Am 01.08.2012 23:08:12 schrieb(en) Lin Hong: > > Hi Gerd, > > > > thank you for the reply. > > > > how do we know if some value is registered as local root or not? > > Easy: It is your task as developer of a C external to register values, > so if you don't do it, it is not done. > > > the code is long so I will just paste some of it first > > > > in that c file, the value we tried to access but failed was also > > allocated by caml_alloc_custom(ops, size, used, max). > > > > caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof (struct > > elt_arr)+sz, 1,alloc_custom_max); > > > > the ops was custom_operations with all standard options as follow: > > > > static struct custom_operations sankoff_custom_operations_eltarr =3D { > > "http://www.amnh.org/poy/", > > custom_finalize_default, > > custom_compare_default, > > custom_hash_default, > > custom_serialize_default, > > custom_deserialize_default > > }; > > > > > > the size in caml_alloc_custom is set to the full size , that is the > > data structure itself, plus everything its pointers point to. > > This is probably all ok. Did you use the CAMLparam and CAMLlocal > macros? > > > yes, we do, just like following one > > value > sankoff_CAML_median_3(value a, value n, value l, value r) { > CAMLparam4(a,n,l,r); > CAMLlocal1(res); > > //load a,n,l,r to eapA,eapN,eapL and eapR > eltarr_p eapA; > eltarr_p eapN; > eltarr_p eapL; > eltarr_p eapR; > Sankoff_eltarr_custom_val(eapA,a); > > sankoff_init_eltarr(eapA,eapA->num_states,eapA->num_elts,-1,-1,-1,-1,NULL= ,0); > Sankoff_eltarr_custom_val(eapN,n); > > sankoff_init_eltarr(eapN,eapN->num_states,eapN->num_elts,-1,-1,-1,-1,NULL= ,0); > Sankoff_eltarr_custom_val(eapL,l); > > sankoff_init_eltarr(eapL,eapL->num_states,eapL->num_elts,-1,-1,-1,-1,NULL= ,0); > Sankoff_eltarr_custom_val(eapR,r); > > sankoff_init_eltarr(eapR,eapR->num_states,eapR->num_elts,-1,-1,-1,-1,NULL= ,0); > > > //we can print everything in eapA~eapR here, they look fine > > //create a new one for return > res =3D caml_alloc_custom (&sankoff_custom_operations_eltarr,sizeof > (struct elt_arr)+sz, 1,alloc_custom_max); > > //if we print eapA~eapR at this point again, they are totally crap. > > neweltarr =3D Sankoff_eltarr_pointer(res); > sankoff_median_3(eapA,eapN,eapL,eapR,neweltarr); > > CAMLreturn(res); > } > > some explanations on functions: > > [Sankoff_eltarr_custom_val] is defined in head file like this: > > #define Sankoff_eltarr_pointer(a) ((struct elt_arr *) > Data_custom_val(a)) > #define Sankoff_eltarr_custom_val(to_asgn,a) to_asgn =3D > Sankoff_eltarr_pointer(a) > > [sankoff_init_eltarr] set pointers to their right position, since we > are loading the full chunk of memory. not sure it's necessary, but > there is no harm to make sure those pointers are correct. > > > > Gerd > > > > > > we also debug with valgrind, the feed back suggest the memory we > > tried to access is indeed freed by caml gc > > > > =3D=3D64563=3D=3D Invalid read of size 4 > > =3D=3D64563=3D=3D at 0x38477B: sankoff_median_3 (in ../poy.native) > > =3D=3D64563=3D=3D by 0x386125: sankoff_CAML_median_3 (in ../poy.nati= ve) > > =3D=3D64563=3D=3D by 0x13177D: camlSankCS__median_3_196 (in > ../poy.native) > > =3D=3D64563=3D=3D Address 0x8bb0c04 is 81,108 bytes inside a block of = size > > 512,016 free'd > > =3D=3D64563=3D=3D at 0xAFD7BF: free (vg_replace_malloc.c:325) > > =3D=3D64563=3D=3D by 0x3E5AC2: caml_free_for_heap (in ../poy.native) > > =3D=3D64563=3D=3D by 0x3E5C31: caml_shrink_heap (in ../poy.native) > > =3D=3D64563=3D=3D by 0x3F16FC: caml_compact_heap (in ../poy.native) > > =3D=3D64563=3D=3D by 0x3F1A89: caml_compact_heap_maybe (in ../poy.na= tive) > > =3D=3D64563=3D=3D by 0x3E4EBC: caml_major_collection_slice (in > > ../poy.native) > > =3D=3D64563=3D=3D by 0x3E5692: caml_minor_collection (in ../poy.nati= ve) > > =3D=3D64563=3D=3D by 0x3E57E0: caml_check_urgent_gc (in ../poy.nativ= e) > > =3D=3D64563=3D=3D by 0x3F209C: caml_alloc_custom (in ../poy.native) > > > > > > > > Cheers, > > Lin Hong > > American Museum of Natural History > > POY website : > > https://code.google.com/p/poy/ > > http://research.amnh.org/scicomp/scripts/download.php > > > > ________________________________________ > > From: Gerd Stolpmann [gerd@edgespring.com] > > Sent: Wednesday, August 01, 2012 4:13 PM > > To: Lin Hong > > Cc: Mailing OCaML > > Subject: Re: [Caml-list] Segmentation fault right after > > caml_alloc_custom > > > > You did not send code, so we can only speculate. A reason for this > > behavior could be that a value is not registered as local root, so > the > > GC thinks it is unused. > > > > That you see the problem only on some platforms can be explained by > > the > > different implementations of malloc, and how quickly freed memory is > > really deallocated (i.e. given back to the kernel). I wouldn't give > > much on this observation. > > > > Gerd > > > > Am 01.08.2012 22:05:19 schrieb(en) Lin Hong: > > > Hi all, > > > > > > we have a weird problem here. In a c file, right after one call of > > > 'caml_alloc_custom', some old information we have in memory is > gone, > > > then we have the segmentation fault because the code try to access > > > those informations later. > > > > > > turn on debug message with OCAMLRUNPARAM, get this right before > it > > > crash. > > > > > > ...... > > > $Starting new major GC cycle > > > Marking 9223372036854775807 words > > > Subphase =3D 10 > > > Sweeping 9223372036854775807 words > > > Compacting heap... > > > Shrinking heap to 37696k bytes > > > Shrinking heap to 36704k bytes > > > Shrinking heap to 35712k bytes > > > Shrinking heap to 34720k bytes > > > Shrinking heap to 33728k bytes > > > Shrinking heap to 32736k bytes > > > Shrinking heap to 31744k bytes > > > Shrinking heap to 30752k bytes > > > done. > > > > > > is it possible that the ocaml garbage collection free memory that > is > > > still in use? > > > > > > the data cause this is kind of big, it doesn't happen right > away,and > > > it doesn't happen on every machine (linux with ocaml4 is fine, > > > ocaml3.xx is not. mac with ocaml4 still shows the problem though). > > > > > > any suggestion is appreciated. > > > > > > > > > Thanks, > > > Lin > > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > American Museum of Natural History > > > POY website : > > > https://code.google.com/p/poy/ > > > http://research.amnh.org/scicomp/scripts/download.php > > > > > > -- > > > Caml-list mailing list. Subscription management and archives: > > > https://sympa-roc.inria.fr/wws/info/caml-list > > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > > > > > > > > > > -- > > Caml-list mailing list. Subscription management and archives: > > https://sympa-roc.inria.fr/wws/info/caml-list > > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > > > > > > -- > ------------------------------------------------------------ > Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de > Creator of GODI and camlcity.org. > Contact details: http://www.camlcity.org/contact.html > Company homepage: http://www.gerd-stolpmann.de > ------------------------------------------------------------ > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa-roc.inria.fr/wws/info/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs > > > -- ------------------------------------------------------------ Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de Creator of GODI and camlcity.org. Contact details: http://www.camlcity.org/contact.html Company homepage: http://www.gerd-stolpmann.de ------------------------------------------------------------