RE: [Caml-list] Segmentation fault from Array.append Ocaml 4.0.x -- walk around

RE: [Caml-list] Segmentation fault from Array.append Ocaml 4.0.x -- walk around

[Lin Hong]

Hi, all

we avoid the problem by creating our version of Array.append. Our version of Array.append looks just like the one from Ocaml 3.12, instead of operating directly on memory, we call get & set function to do the job.

still have no idea why Array.append from Ocaml4.0.x cause the seg fault, but we will stick to our walk around solution till new version come out.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Lin Hong [lhong@amnh.org]
Sent: Monday, November 19, 2012 12:04 PM
To: Damien Doligez; caml users
Subject: RE: [Caml-list] Segmentation fault from Array.append

Hi,

we recompiled Ocaml with debug version of runtime, then compiled our program into byte code, run the same data-set , the problem disappear.

but with native version of our program, the segmentation fault still shows.

As for native version, if we run it with Gc.compact() being called around Array.append, the problem won't show.

also gdb point us to some major_slice() function, so we call Gc.major_slice() before Array.append, this also make the problem disappear.


still don't know what's going on here, any other suggestions are highly appreciated.


Thanks.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Damien Doligez [damien.doligez@inria.fr]
Sent: Wednesday, October 17, 2012 9:33 AM
To: caml users
Subject: Re: [Caml-list] Segmentation fault from Array.append

Hello,

> From: Daniel Bünzli [daniel.buenzli@erratique.ch]
>
> http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/


I have posted a comment to that blog.  For the archive, here's what it says:

Starting with OCaml 4.00.0, there is an easy way to activate the debug version of the runtime:

1. configure OCaml with "-with-debug-runtime"
2. compile and install OCaml
3. compile your program with "-runtime-variant d"

This will compile your program with a version of the runtime which has assertions all over the place, and does a thorough check of the heap structure at each major GC (and at each compaction). If you then follow Rich’s advice, you should be able to narrow down the source of the heap corruption quite easily.

-- Damien


--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Segmentation fault from Array.append Ocaml 4.0.x -- walk around

[Damien Doligez]

[-- Attachment #1: Type: text/plain, Size: 3590 bytes --]

Hello Lin Hong,

I have found a bug in the C code for Array.append. A patch
against 4.00.1 is attached to this mail.

I couldn't make a small repro case for this bug, so I would
appreciate if you could try this patch an tell me whether it
fixes the problem for you.

Cheers,

-- Damien



On 2012-11-28, at 21:02, Lin Hong wrote:

> Hi, all
> 
> we avoid the problem by creating our version of Array.append. Our version of Array.append looks just like the one from Ocaml 3.12, instead of operating directly on memory, we call get & set function to do the job.
> 
> still have no idea why Array.append from Ocaml4.0.x cause the seg fault, but we will stick to our walk around solution till new version come out.
> 
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
> 
> ________________________________________
> From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Lin Hong [lhong@amnh.org]
> Sent: Monday, November 19, 2012 12:04 PM
> To: Damien Doligez; caml users
> Subject: RE: [Caml-list] Segmentation fault from Array.append
> 
> Hi,
> 
> we recompiled Ocaml with debug version of runtime, then compiled our program into byte code, run the same data-set , the problem disappear.
> 
> but with native version of our program, the segmentation fault still shows.
> 
> As for native version, if we run it with Gc.compact() being called around Array.append, the problem won't show.
> 
> also gdb point us to some major_slice() function, so we call Gc.major_slice() before Array.append, this also make the problem disappear.
> 
> 
> still don't know what's going on here, any other suggestions are highly appreciated.
> 
> 
> Thanks.
> 
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
> 
> ________________________________________
> From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Damien Doligez [damien.doligez@inria.fr]
> Sent: Wednesday, October 17, 2012 9:33 AM
> To: caml users
> Subject: Re: [Caml-list] Segmentation fault from Array.append
> 
> Hello,
> 
>> From: Daniel Bünzli [daniel.buenzli@erratique.ch]
>> 
>> http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/
> 
> 
> I have posted a comment to that blog.  For the archive, here's what it says:
> 
> Starting with OCaml 4.00.0, there is an easy way to activate the debug version of the runtime:
> 
> 1. configure OCaml with "-with-debug-runtime"
> 2. compile and install OCaml
> 3. compile your program with "-runtime-variant d"
> 
> This will compile your program with a version of the runtime which has assertions all over the place, and does a thorough check of the heap structure at each major GC (and at each compaction). If you then follow Rich’s advice, you should be able to narrow down the source of the heap corruption quite easily.
> 
> -- Damien
> 
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

[-- Attachment #2: 0000-array-append-segfault.patch --]
[-- Type: application/octet-stream, Size: 705 bytes --]

Index: byterun/array.c
===================================================================
--- byterun/array.c	(revision 13107)
+++ byterun/array.c	(working copy)
@@ -321,11 +321,12 @@
            count--, src++, pos++) {
         caml_initialize(&Field(res, pos), *src);
       }
-      /* Many caml_initialize in a row can create a lot of old-to-young
-         refs.  Give the minor GC a chance to run if it needs to. */
-      res = caml_check_urgent_gc(res);
     }
     Assert(pos == size);
+
+    /* Many caml_initialize in a row can create a lot of old-to-young
+       refs.  Give the minor GC a chance to run if it needs to. */
+    res = caml_check_urgent_gc(res);
   }
   CAMLreturn (res);
 }

RE: [Caml-list] Segmentation fault from Array.append Ocaml 4.0.x -- walk around

[Lin Hong]

Hi Damien,

our test finish without problem, looks like your fix for array.append works.  thanks again.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: Damien Doligez [damien.doligez@inria.fr]
Sent: Tuesday, December 04, 2012 10:20 AM
To: Lin Hong
Cc: caml users
Subject: Re: [Caml-list] Segmentation fault from Array.append Ocaml 4.0.x -- walk around

Hello Lin Hong,

I have found a bug in the C code for Array.append. A patch
against 4.00.1 is attached to this mail.

I couldn't make a small repro case for this bug, so I would
appreciate if you could try this patch an tell me whether it
fixes the problem for you.

Cheers,

-- Damien



On 2012-11-28, at 21:02, Lin Hong wrote:

> Hi, all
>
> we avoid the problem by creating our version of Array.append. Our version of Array.append looks just like the one from Ocaml 3.12, instead of operating directly on memory, we call get & set function to do the job.
>
> still have no idea why Array.append from Ocaml4.0.x cause the seg fault, but we will stick to our walk around solution till new version come out.
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Lin Hong [lhong@amnh.org]
> Sent: Monday, November 19, 2012 12:04 PM
> To: Damien Doligez; caml users
> Subject: RE: [Caml-list] Segmentation fault from Array.append
>
> Hi,
>
> we recompiled Ocaml with debug version of runtime, then compiled our program into byte code, run the same data-set , the problem disappear.
>
> but with native version of our program, the segmentation fault still shows.
>
> As for native version, if we run it with Gc.compact() being called around Array.append, the problem won't show.
>
> also gdb point us to some major_slice() function, so we call Gc.major_slice() before Array.append, this also make the problem disappear.
>
>
> still don't know what's going on here, any other suggestions are highly appreciated.
>
>
> Thanks.
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Damien Doligez [damien.doligez@inria.fr]
> Sent: Wednesday, October 17, 2012 9:33 AM
> To: caml users
> Subject: Re: [Caml-list] Segmentation fault from Array.append
>
> Hello,
>
>> From: Daniel Bünzli [daniel.buenzli@erratique.ch]
>>
>> http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/
>
>
> I have posted a comment to that blog.  For the archive, here's what it says:
>
> Starting with OCaml 4.00.0, there is an easy way to activate the debug version of the runtime:
>
> 1. configure OCaml with "-with-debug-runtime"
> 2. compile and install OCaml
> 3. compile your program with "-runtime-variant d"
>
> This will compile your program with a version of the runtime which has assertions all over the place, and does a thorough check of the heap structure at each major GC (and at each compaction). If you then follow Rich’s advice, you should be able to narrow down the source of the heap corruption quite easily.
>
> -- Damien
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs