From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: * X-Spam-Status: No, score=1.4 required=5.0 tests=HTML_MESSAGE, MSGID_MULTIPLE_AT autolearn=disabled version=3.1.3 X-Original-To: caml-list@yquem.inria.fr Delivered-To: caml-list@yquem.inria.fr Received: from mail4-relais-sop.national.inria.fr (mail4-relais-sop.national.inria.fr [192.134.164.105]) by yquem.inria.fr (Postfix) with ESMTP id B0972BBAF for ; Sun, 9 Aug 2009 09:59:02 +0200 (CEST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqcBAK0efkrLOwGUkWdsb2JhbACCVZZqgQwBAQEBCQsKBxMEsieEGAWCJw X-IronPort-AV: E=Sophos;i="4.43,348,1246831200"; d="scan'208,217";a="44488492" Received: from outbound.icp-qv1-irony-out3.iinet.net.au ([203.59.1.148]) by mail4-smtp-sop.national.inria.fr with ESMTP; 09 Aug 2009 09:58:58 +0200 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: As8EABoffkrSVAql/2dsb2JhbACCVZZqs3SEGAWCJw X-IronPort-AV: E=Sophos;i="4.43,348,1246809600"; d="scan'208,217";a="489029996" Received: from unknown (HELO ivanz27oq2f1va) ([210.84.10.165]) by outbound.icp-qv1-irony-out3.iinet.net.au with ESMTP; 09 Aug 2009 15:58:54 +0800 From: "ivan chollet" To: "'David Allsopp'" Cc: , "'Edgar Friendly'" References: <000301ca184b$032b8cc0$0982a640$@chollet@free.fr> <000701ca184d$29507e90$7bf17bb0$@metastack.com> In-Reply-To: <000701ca184d$29507e90$7bf17bb0$@metastack.com> Subject: RE: [Caml-list] Re: ocaml sefault in bytecode: unanswered questions Date: Sun, 9 Aug 2009 09:58:43 +0200 Message-ID: <001001ca18c7$37b22220$a7166660$@chollet@free.fr> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0011_01CA18D7.FB3AF220" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcoYSwIujdfw2t6+QYu8dTW6a2iSLQAALp+gABHYngA= Content-Language: fr X-Spam: no; 0.00; ocaml:01 bytecode:01 real-world:01 myfun:01 iter:01 myfun:01 iteration:01 allocations:01 iter:01 iterating:01 iteration:01 ocamlrun:01 ocaml:01 bytecode:01 pointer:01 This is a multi-part message in MIME format. ------=_NextPart_000_0011_01CA18D7.FB3AF220 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Definitely. Actually I had my real-world case in mind, so let me explain further = with the following snippet: =20 let myfun =3D doSomeWork (); myref :=3D List.filter somefilterfunction = !myref in List.iter myfun !myref =20 In this case, a new linked list is created in each iteration of the List.filter. (that is, a new list allocation) Then, if doSomeWork () does a lot of work and lots of allocations, the = GC will be called on a regular basis while in function myfun.=20 Then List.iter is tail-recursive, so it doesn=92t push its successive arguments on the stack. So the successively created myref become = unreachable while still iterating on them. So my question is, how does the GC know whether all these myref created throughout the iteration are collectable or not? I=92m curious about how = these myref are tagged/untagged by the garbage collector. Maybe pointing me = the relevant portions of the ocamlrun source code would be nice. =20 Anyway no worries, once I get a bit more free I=92ll just try to read = about this topic by myself. Also I=92ll try to send you some source code for = that. All this will take me a little while, so see you next time! =20 =20 =20 From: David Allsopp [mailto:dra-news@metastack.com]=20 Sent: samedi 8 ao=FBt 2009 19:25 To: 'ivan chollet' Cc: caml-list@yquem.inria.fr Subject: RE: [Caml-list] Re: ocaml sefault in bytecode: unanswered = questions =20 When you pass a value to a function, you create a pointer to that value = in the OCaml runtime =96 the GC can=92t collect the old value until = List.iter completes because the value is still live (internally, it=92s part of a = local root but, in practice, as List.iter is implemented in OCaml directly = it=92s because an OCaml function parameter references the value). Note that in = this example: =20 let a =3D [1; 2; 3] and b =3D [4; 5; 6] and c =3D [7; 8; 9] in let myref =3D ref a in (* No allocations are done after here *) myref :=3D a; myref :=3D b; myref :=3D c;; =20 the assignments to [myref] do not result in any memory being allocated = at all (my point is that action of assigning to a reference does not = implicitly result in an allocation). =20 =20 David =20 From: caml-list-bounces@yquem.inria.fr [mailto:caml-list-bounces@yquem.inria.fr] On Behalf Of ivan chollet Sent: 08 August 2009 18:10 To: ivan.chollet@free.fr Cc: caml-list@yquem.inria.fr Subject: [Caml-list] Re: ocaml sefault in bytecode: unanswered questions =20 Yes it was a freebsd 6.4 with ocaml 3.10.2 I=92ll run the program on linux later and see how it goes. =20 Thanks for your advices regarding debugging. I pretty much tried all of these though=85 the thing is my error is not an ocaml error at runtime = but an error of the ocaml runtime. And to analyze a core dump of ocamlrun, I = just thought my best bet was gdb. Whatever. =20 OK I=92ll try to provide you with a minimal ocaml code that produce an ocamlrun error. Might take a little while as I=92m not free. In the meantime, I=92ve got a newbie question regarding ocaml garbage collector and the same List.iter stuff: Say you do a =93List.iter myfun !myref=94, where !myref is a list = (hehe=85), and where myfun is a function that does reallocations of myref (that is affectations like myref :=3D [some new or old objects]). The pointers = myref that are generated through this process are destroyed each time a new reallocation of myref is done. Of course the underlying linked lists = that are not referenced anymore shouldn=92t be collected by the GC before the = end of the main =93List.iter=94, otherwise it=92s iterating through a linked = list that has been garbage collected. My question is: does the GC know that it cannot collect the unreferenced myref pointers before the end of the List.iter?=20 Sorry, I just wanted to ask this question to rule it out. =20 Thanks again. =20 =20 =20 =20 On 07-08-2009, ivan chollet wrote: >=20 > This GDB was configured as "i386-marcel-freebsd"...(no debugging = symbols > found)... >=20 > Not very informative. So here are my questions: =20 I suppose you are running freebsd ? Which version of freebsd, of ocaml ? = =20 >=20 > =20 >=20 > - What is the best way to produce and analyze core dumps in ocaml? > Should I compile in bytecode or native? Is there any special gdb = "trick" > that gives you more information? Is there any special "trick" while > compiling the ocaml runtime to make it throw more information? >=20 =20 gdb is not the perfect tool to debug ocaml program. You should give a try to ocamldebug which is a better option for bytecode (see below for options). Bytecode is more informative when coming to reporting backtrace (at least with old version of ocaml).=20 =20 Compile every program with "-g" option (just like gcc).=20 =20 If you have compiled everything with "-g" option, you can also use the environment variable OCAMLRUNPARAM=3D"b" to get a backtrace for your exception, at runtime. =20 > - Then, my main question is actually: in bytecode, what can produce > segfaults? My ocaml code is completely standard, excepted that I use = the > Marshal module. So my question is rather: outside the Marshal module, = what > can cause segfault? =20 Some part of the bytecode are just standard C, everything can cause a segfault just as C. These errors are not very common but it is possible that some case are not well handled on freebsd. Most probably a porting issue. =20 Marshal module can easily trigger a segfault when you map the loaded = data to a type which doesn't match the dumped data. =20 Example:=20 List.length (Marshal.from_string (Marshal.to_string 1234 []) 0);; =20 Here the integer value is marshalled and then unmarshalled as a list -> segfault. =20 >=20 > - Slightly unrelated question: I have been able to generate > segfaults by running ocaml code that: 1) iterates recursively through = a list > reference 2) changes the reference while still iterating on it. For example, > you just do a "List.iter myfun !myref", and within the function myfun, = you > do stuff like "myref :=3D List.filter somefilterfunction !myref". It = is not > good to program like this, but for some reason I thought ocaml would = not > segfault on that. Is this expected behavior? If it's not, I'll be = happy to > provide some simple source code that illustrates it. (nevermind I have > actually cleaned all my code base from these dirty uses of references) >=20 =20 Could you provide a minimal example code for this error ? I don't think this should generate a segfault. =20 =20 Regards Sylvain Le Gall =20 ------=_NextPart_000_0011_01CA18D7.FB3AF220 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Definitely.

Actually = I had my real-world case in mind, so let me explain further with the following = snippet:

 

let = myfun =3D doSomeWork (); myref :=3D List.filter somefilterfunction !myref = in

List.iter myfun !myref

 

In this = case, a new linked list is created in each iteration of the List.filter. (that is, a = new list allocation)

Then, if = doSomeWork () does a lot of work and lots of allocations, the GC will be called on = a regular basis while in function myfun.

Then = List.iter is tail-recursive, so it doesn’t push its successive arguments on the = stack. So the successively created myref become unreachable while still = iterating on them.

So my = question is, how does the GC know whether all these myref created throughout the = iteration are collectable or not? I’m curious about how these myref are = tagged/untagged by the garbage collector. Maybe pointing me the relevant portions of the = ocamlrun source code would be nice.

 

Anyway = no worries, once I get a bit more free I’ll just try to read about this topic = by myself. Also I’ll try to send you some source code for that. All = this will take me a little while, so see you next time!

 

 

 

From: David Allsopp [mailto:dra-news@metastack.com]
Sent: samedi 8 ao=FBt 2009 19:25
To: 'ivan chollet'
Cc: caml-list@yquem.inria.fr
Subject: RE: [Caml-list] Re: ocaml sefault in bytecode: = unanswered questions

 

When you pass a value to a = function, you create a pointer to that value in the OCaml runtime – the GC = can’t collect the old value until List.iter completes because the value is = still live (internally, it’s part of a local root but, in practice, as = List.iter is implemented in OCaml directly it’s because an OCaml function = parameter references the value). Note that in this example:

 

let a =3D [1; 2; = 3]

and b =3D [4; 5; = 6]

and c =3D [7; 8; 9] = in

let myref =3D ref a = in

(* No allocations are done after = here *)

  myref :=3D = a;

  myref :=3D = b;

  myref :=3D = c;;

 

the assignments to [myref] do = not result in any memory being allocated at all (my point is that action of assigning = to a reference does not implicitly result in an = allocation).

 

 

David

 

From: caml-list-bounces@yquem.inria.fr [mailto:caml-list-bounces@yquem.inria.fr] On Behalf Of ivan = chollet
Sent: 08 August 2009 18:10
To: ivan.chollet@free.fr
Cc: caml-list@yquem.inria.fr
Subject: [Caml-list] Re: ocaml sefault in bytecode: unanswered = questions

 

Yes it was a freebsd 6.4 with ocaml 3.10.2

I’ll run the program on linux later and see how it = goes.

 

Thanks for your advices regarding debugging. I pretty much tried all of these though… the thing is my error is not an ocaml error at runtime but = an error of the ocaml runtime. And to analyze a core dump of ocamlrun, I = just thought my best bet was gdb. Whatever.

 

OK I’ll try to provide you with a minimal ocaml code that produce an ocamlrun error. Might take a little while as I’m not = free.

In the meantime, I’ve got a newbie question regarding ocaml garbage collector and the same List.iter stuff:

Say you do a “List.iter myfun !myref”, where !myref is a list (hehe…), and where myfun is a function that does reallocations of = myref (that is affectations like myref :=3D [some new or old objects]). The = pointers myref that are generated through this process are destroyed each time a = new reallocation of myref is done. Of course the underlying linked lists = that are not referenced anymore shouldn’t be collected by the GC before the = end of the main “List.iter”, otherwise it’s iterating through = a linked list that has been garbage collected.

My question is: does the GC know that it cannot collect the unreferenced = myref pointers before the end of the List.iter?

Sorry, I just wanted to ask this question to rule it out.

 

Thanks again.

 

 

 

 

On 07-08-2009, ivan chollet <ivan.chollet@free.fr> = wrote:

> 

> This GDB was configured as "i386-marcel-freebsd"...(no = debugging symbols

> found)...

> 

> Not very informative. So here are my questions:

 

I suppose you are running freebsd ? Which version of freebsd, of ocaml ? =

 

> 

> 

> 

> -          What is the best = way to produce and analyze core dumps in ocaml?

> Should I compile in bytecode or native? Is there any special gdb "trick"

> that gives you more information? Is there any special "trick" = while

> compiling the ocaml runtime to make it throw more = information?

> 

 

gdb is not the perfect tool to debug ocaml program. You should give = a

try to ocamldebug which is a better option for bytecode (see below = for

options). Bytecode is more informative when coming to = reporting

backtrace (at least with old version of ocaml).

 

Compile every program with "-g" option (just like gcc). =

 

If you have compiled everything with "-g" option, you can also = use the

environment variable OCAMLRUNPARAM=3D"b" to get a backtrace for = your

exception, at runtime.

 

> -          Then, my main = question is actually: in bytecode, what can produce

> segfaults? My ocaml code is completely standard, excepted that I use = the

> Marshal module. So my question is rather: outside the Marshal module, = what

> can cause segfault?

 

Some part of the bytecode are just standard C, everything can cause = a

segfault just as C. These errors are not very common but it is = possible

that some case are not well handled on freebsd. Most probably a = porting

issue.

 

Marshal module can easily trigger a segfault when you map the loaded = data

to a type which doesn't match the dumped data.

 

Example:

List.length (Marshal.from_string (Marshal.to_string 1234 []) = 0);;

 

Here the integer value is marshalled and then unmarshalled as a list = ->

segfault.

 

> 

> -          Slightly = unrelated question: I have been able to generate

> segfaults by running ocaml code that: 1) iterates recursively through a = list

> reference 2) changes the reference while still iterating on it. For = example,

> you just do a "List.iter myfun !myref", and within the = function myfun, you

> do stuff like "myref :=3D List.filter somefilterfunction = !myref". It is not

> good to program like this, but for some reason I thought ocaml would = not

> segfault on that. Is this expected behavior? If it's not, I'll be happy = to

> provide some simple source code that illustrates it. (nevermind I = have

> actually cleaned all my code base from these dirty uses of = references)

> 

 

Could you provide a minimal example code for this error ? I don't = think

this should generate a segfault.

 

 

Regards

Sylvain Le Gall

 

------=_NextPart_000_0011_01CA18D7.FB3AF220--