[Caml-list] Segmentation fault from Array.append

[Caml-list] Segmentation fault from Array.append

[Lin Hong]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

Hi, all

we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function of Array. gdb traceback give us this:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
0x0000000100488840 in mark_slice ()
(gdb) backtrace
#0  0x0000000100488840 in mark_slice ()
#1  0x0000000100489216 in caml_major_collection_slice ()
#2  0x000000010048984c in caml_check_urgent_gc ()
#3  0x000000010048f26d in caml_array_gather ()
#4  0x000000010048f42e in caml_array_append ()
#5  0x00000001003ef216 in .L363 ()
Previous frame inner to this frame (gdb could not unwind past this frame)
(gdb)

looks like the error is from array.c

we don't see this problem with Ocaml3.X.X, they have different array function in the cside anyway.

Does anyone else have similar problem like this?


Thanks in advance for any suggestions.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

[-- Attachment #2: Type: text/html, Size: 3126 bytes --]

Re: [Caml-list] Segmentation fault from Array.append

[David House]

Segfaults in the GC are often the fault of random heap corruption. Are
you using any C stubs that might be at fault here?

Is the fault reproducable?

On Tue, Oct 16, 2012 at 4:08 PM, Lin Hong <lhong@amnh.org> wrote:
> Hi, all
>
> we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function
> of Array. gdb traceback give us this:
>
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
> 0x0000000100488840 in mark_slice ()
> (gdb) backtrace
> #0  0x0000000100488840 in mark_slice ()
> #1  0x0000000100489216 in caml_major_collection_slice ()
> #2  0x000000010048984c in caml_check_urgent_gc ()
> #3  0x000000010048f26d in caml_array_gather ()
> #4  0x000000010048f42e in caml_array_append ()
> #5  0x00000001003ef216 in .L363 ()
> Previous frame inner to this frame (gdb could not unwind past this frame)
> (gdb)
>
> looks like the error is from array.c
>
> we don't see this problem with Ocaml3.X.X, they have different array
> function in the cside anyway.
>
> Does anyone else have similar problem like this?
>
>
> Thanks in advance for any suggestions.
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php

RE: [Caml-list] Segmentation fault from Array.append

[Lin Hong]

if by "c stubs" you mean some function that holding up place but does nothing, we are not doing any of that.

and yes it's reproducible on one mac, I will test it on other machine with Ocaml4.X.X


Thanks,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: David House [dhouse@janestreet.com]
Sent: Tuesday, October 16, 2012 11:10 AM
To: Lin Hong
Cc: caml-list@inria.fr
Subject: Re: [Caml-list] Segmentation fault from Array.append

Segfaults in the GC are often the fault of random heap corruption. Are
you using any C stubs that might be at fault here?

Is the fault reproducable?

On Tue, Oct 16, 2012 at 4:08 PM, Lin Hong <lhong@amnh.org> wrote:
> Hi, all
>
> we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function
> of Array. gdb traceback give us this:
>
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
> 0x0000000100488840 in mark_slice ()
> (gdb) backtrace
> #0  0x0000000100488840 in mark_slice ()
> #1  0x0000000100489216 in caml_major_collection_slice ()
> #2  0x000000010048984c in caml_check_urgent_gc ()
> #3  0x000000010048f26d in caml_array_gather ()
> #4  0x000000010048f42e in caml_array_append ()
> #5  0x00000001003ef216 in .L363 ()
> Previous frame inner to this frame (gdb could not unwind past this frame)
> (gdb)
>
> looks like the error is from array.c
>
> we don't see this problem with Ocaml3.X.X, they have different array
> function in the cside anyway.
>
> Does anyone else have similar problem like this?
>
>
> Thanks in advance for any suggestions.
>
> Cheers,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php

Re: [Caml-list] Segmentation fault from Array.append

[David House]

By "c stubs" I really mean c functions that you're calling from ocaml.
It's very easy to get such things wrong and stamp all over your heap.

On Tue, Oct 16, 2012 at 4:41 PM, Lin Hong <lhong@amnh.org> wrote:
>
>
> if by "c stubs" you mean some function that holding up place but does nothing, we are not doing any of that.
>
> and yes it's reproducible on one mac, I will test it on other machine with Ocaml4.X.X
>
>
> Thanks,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: David House [dhouse@janestreet.com]
> Sent: Tuesday, October 16, 2012 11:10 AM
> To: Lin Hong
> Cc: caml-list@inria.fr
> Subject: Re: [Caml-list] Segmentation fault from Array.append
>
> Segfaults in the GC are often the fault of random heap corruption. Are
> you using any C stubs that might be at fault here?
>
> Is the fault reproducable?
>
> On Tue, Oct 16, 2012 at 4:08 PM, Lin Hong <lhong@amnh.org> wrote:
>> Hi, all
>>
>> we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function
>> of Array. gdb traceback give us this:
>>
>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>> Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
>> 0x0000000100488840 in mark_slice ()
>> (gdb) backtrace
>> #0  0x0000000100488840 in mark_slice ()
>> #1  0x0000000100489216 in caml_major_collection_slice ()
>> #2  0x000000010048984c in caml_check_urgent_gc ()
>> #3  0x000000010048f26d in caml_array_gather ()
>> #4  0x000000010048f42e in caml_array_append ()
>> #5  0x00000001003ef216 in .L363 ()
>> Previous frame inner to this frame (gdb could not unwind past this frame)
>> (gdb)
>>
>> looks like the error is from array.c
>>
>> we don't see this problem with Ocaml3.X.X, they have different array
>> function in the cside anyway.
>>
>> Does anyone else have similar problem like this?
>>
>>
>> Thanks in advance for any suggestions.
>>
>> Cheers,
>> Lin Hong
>> American Museum of Natural History
>> POY website :
>> https://code.google.com/p/poy/
>> http://research.amnh.org/scicomp/scripts/download.php

RE: [Caml-list] Segmentation fault from Array.append

[Lin Hong]

oh yes, we have c functions called from ocaml.

if any of them screw heap up, why gdb debug didn't say anything. is there some debug option that we can get a closer look at heap?


Thanks,


Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: David House [dhouse@janestreet.com]
Sent: Tuesday, October 16, 2012 11:42 AM
To: Lin Hong
Cc: caml-list@inria.fr
Subject: Re: [Caml-list] Segmentation fault from Array.append

By "c stubs" I really mean c functions that you're calling from ocaml.
It's very easy to get such things wrong and stamp all over your heap.

On Tue, Oct 16, 2012 at 4:41 PM, Lin Hong <lhong@amnh.org> wrote:
>
>
> if by "c stubs" you mean some function that holding up place but does nothing, we are not doing any of that.
>
> and yes it's reproducible on one mac, I will test it on other machine with Ocaml4.X.X
>
>
> Thanks,
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: David House [dhouse@janestreet.com]
> Sent: Tuesday, October 16, 2012 11:10 AM
> To: Lin Hong
> Cc: caml-list@inria.fr
> Subject: Re: [Caml-list] Segmentation fault from Array.append
>
> Segfaults in the GC are often the fault of random heap corruption. Are
> you using any C stubs that might be at fault here?
>
> Is the fault reproducable?
>
> On Tue, Oct 16, 2012 at 4:08 PM, Lin Hong <lhong@amnh.org> wrote:
>> Hi, all
>>
>> we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function
>> of Array. gdb traceback give us this:
>>
>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>> Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
>> 0x0000000100488840 in mark_slice ()
>> (gdb) backtrace
>> #0  0x0000000100488840 in mark_slice ()
>> #1  0x0000000100489216 in caml_major_collection_slice ()
>> #2  0x000000010048984c in caml_check_urgent_gc ()
>> #3  0x000000010048f26d in caml_array_gather ()
>> #4  0x000000010048f42e in caml_array_append ()
>> #5  0x00000001003ef216 in .L363 ()
>> Previous frame inner to this frame (gdb could not unwind past this frame)
>> (gdb)
>>
>> looks like the error is from array.c
>>
>> we don't see this problem with Ocaml3.X.X, they have different array
>> function in the cside anyway.
>>
>> Does anyone else have similar problem like this?
>>
>>
>> Thanks in advance for any suggestions.
>>
>> Cheers,
>> Lin Hong
>> American Museum of Natural History
>> POY website :
>> https://code.google.com/p/poy/
>> http://research.amnh.org/scicomp/scripts/download.php

Re: [Caml-list] Segmentation fault from Array.append

[David House]

I don't know enough about gdb to determine that, sorry.

If it's reproducible, you should be able to get a minimal test case,
and then just read carefully the c functions that remain. Keep in mind
the ocaml manual:

http://caml.inria.fr/pub/docs/manual-ocaml/manual033.html#toc146

On Tue, Oct 16, 2012 at 4:50 PM, Lin Hong <lhong@amnh.org> wrote:
>
> oh yes, we have c functions called from ocaml.
>
> if any of them screw heap up, why gdb debug didn't say anything. is there some debug option that we can get a closer look at heap?
>
>
> Thanks,
>
>
> Lin Hong
> American Museum of Natural History
> POY website :
> https://code.google.com/p/poy/
> http://research.amnh.org/scicomp/scripts/download.php
>
> ________________________________________
> From: David House [dhouse@janestreet.com]
> Sent: Tuesday, October 16, 2012 11:42 AM
> To: Lin Hong
> Cc: caml-list@inria.fr
> Subject: Re: [Caml-list] Segmentation fault from Array.append
>
> By "c stubs" I really mean c functions that you're calling from ocaml.
> It's very easy to get such things wrong and stamp all over your heap.
>
> On Tue, Oct 16, 2012 at 4:41 PM, Lin Hong <lhong@amnh.org> wrote:
>>
>>
>> if by "c stubs" you mean some function that holding up place but does nothing, we are not doing any of that.
>>
>> and yes it's reproducible on one mac, I will test it on other machine with Ocaml4.X.X
>>
>>
>> Thanks,
>> Lin Hong
>> American Museum of Natural History
>> POY website :
>> https://code.google.com/p/poy/
>> http://research.amnh.org/scicomp/scripts/download.php
>>
>> ________________________________________
>> From: David House [dhouse@janestreet.com]
>> Sent: Tuesday, October 16, 2012 11:10 AM
>> To: Lin Hong
>> Cc: caml-list@inria.fr
>> Subject: Re: [Caml-list] Segmentation fault from Array.append
>>
>> Segfaults in the GC are often the fault of random heap corruption. Are
>> you using any C stubs that might be at fault here?
>>
>> Is the fault reproducable?
>>
>> On Tue, Oct 16, 2012 at 4:08 PM, Lin Hong <lhong@amnh.org> wrote:
>>> Hi, all
>>>
>>> we got a Segmentation fault with Ocaml 4.0.0 and 4.0.1 from append function
>>> of Array. gdb traceback give us this:
>>>
>>> Program received signal EXC_BAD_ACCESS, Could not access memory.
>>> Reason: KERN_INVALID_ADDRESS at address: 0x0000000107b65000
>>> 0x0000000100488840 in mark_slice ()
>>> (gdb) backtrace
>>> #0  0x0000000100488840 in mark_slice ()
>>> #1  0x0000000100489216 in caml_major_collection_slice ()
>>> #2  0x000000010048984c in caml_check_urgent_gc ()
>>> #3  0x000000010048f26d in caml_array_gather ()
>>> #4  0x000000010048f42e in caml_array_append ()
>>> #5  0x00000001003ef216 in .L363 ()
>>> Previous frame inner to this frame (gdb could not unwind past this frame)
>>> (gdb)
>>>
>>> looks like the error is from array.c
>>>
>>> we don't see this problem with Ocaml3.X.X, they have different array
>>> function in the cside anyway.
>>>
>>> Does anyone else have similar problem like this?
>>>
>>>
>>> Thanks in advance for any suggestions.
>>>
>>> Cheers,
>>> Lin Hong
>>> American Museum of Natural History
>>> POY website :
>>> https://code.google.com/p/poy/
>>> http://research.amnh.org/scicomp/scripts/download.php

Re: [Caml-list] Segmentation fault from Array.append

[Daniel Bünzli]

If you're having a problem with your bindings, heap corruptors are usually quite easy to locate. See here :  

http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/

Best,

Daniel

RE: [Caml-list] Segmentation fault from Array.append

[Lin Hong]

thanks , I will try it out.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: Daniel Bünzli [daniel.buenzli@erratique.ch]
Sent: Tuesday, October 16, 2012 2:02 PM
To: Lin Hong
Cc: David House; caml-list@inria.fr
Subject: Re: [Caml-list] Segmentation fault from Array.append

If you're having a problem with your bindings, heap corruptors are usually quite easy to locate. See here :

http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/

Best,

Daniel

Re: [Caml-list] Segmentation fault from Array.append

[Damien Doligez]

Hello,

> From: Daniel Bünzli [daniel.buenzli@erratique.ch]
> 
> http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/


I have posted a comment to that blog.  For the archive, here's what it says:

Starting with OCaml 4.00.0, there is an easy way to activate the debug version of the runtime:

1. configure OCaml with "-with-debug-runtime"
2. compile and install OCaml
3. compile your program with "-runtime-variant d"

This will compile your program with a version of the runtime which has assertions all over the place, and does a thorough check of the heap structure at each major GC (and at each compaction). If you then follow Rich’s advice, you should be able to narrow down the source of the heap corruption quite easily.

-- Damien

RE: [Caml-list] Segmentation fault from Array.append

[Lin Hong]

Hi,

we recompiled Ocaml with debug version of runtime, then compiled our program into byte code, run the same data-set , the problem disappear. 

but with native version of our program, the segmentation fault still shows. 

As for native version, if we run it with Gc.compact() being called around Array.append, the problem won't show. 

also gdb point us to some major_slice() function, so we call Gc.major_slice() before Array.append, this also make the problem disappear. 


still don't know what's going on here, any other suggestions are highly appreciated. 


Thanks.

Cheers,
Lin Hong
American Museum of Natural History
POY website :
https://code.google.com/p/poy/
http://research.amnh.org/scicomp/scripts/download.php

________________________________________
From: caml-list-request@inria.fr [caml-list-request@inria.fr] on behalf of Damien Doligez [damien.doligez@inria.fr]
Sent: Wednesday, October 17, 2012 9:33 AM
To: caml users
Subject: Re: [Caml-list] Segmentation fault from Array.append

Hello,

> From: Daniel Bünzli [daniel.buenzli@erratique.ch]
>
> http://rwmj.wordpress.com/2010/01/22/tip-tracking-down-ocaml-heap-corruptors/


I have posted a comment to that blog.  For the archive, here's what it says:

Starting with OCaml 4.00.0, there is an easy way to activate the debug version of the runtime:

1. configure OCaml with "-with-debug-runtime"
2. compile and install OCaml
3. compile your program with "-runtime-variant d"

This will compile your program with a version of the runtime which has assertions all over the place, and does a thorough check of the heap structure at each major GC (and at each compaction). If you then follow Rich’s advice, you should be able to narrow down the source of the heap corruption quite easily.

-- Damien


--
Caml-list mailing list.  Subscription management and archives:
https://sympa.inria.fr/sympa/arc/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs