[Caml-list] Only bytecode version of executable allocating huge amounts

[Caml-list] Only bytecode version of executable allocating huge amounts

[Jon French]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

Hi all,

I'm wondering if anyone on this list might have encountered this issue before, since it's got me pretty stumped.

We have an OCaml project ( https://github.com/rems-project/sail ) which tries to allocate huge amounts of memory, but only when compiled to bytecode. The native version works perfectly fine.

strace for an example run ends in:

> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail", O_RDONLY|O_CLOEXEC) = 3
> lseek(3, 0, SEEK_CUR) = 0
> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> brk(0x5729f32c4000) = 0x55603f2f4000
> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
> ) = 28
> exit_group(2) = ?
> +++ exited with 2 +++

which naturally fails since I don't have 2 TB of memory available. 

The exact amount of the allocation varies from input to input and even run to run, but is always that approximate size. There's no characteristic pattern of heap or stack running out of control if I turn on memory debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run in ocamldebug, it also doesn't appear to be failing at a location which is sensible to be allocating such amounts of memory -- and the location also varies with the input. And on tiny inputs it doesn't seem to try and make the allocation at all.

Are we actually seeing a compiler/runtime bug here? Is there something I'm missing?

Thanks,

Jon French
Computer Laboratory
University of Cambridge

[-- Attachment #2: Type: text/html, Size: 2815 bytes --]

Re: [Caml-list] Only bytecode version of executable allocating huge amounts

[rixed]

Can it be another instance of https://github.com/ocaml/opam/issues/3636 ?

On Tue, May 28, 2019, at 14:27, Jon French wrote:
> Hi all,
> 
> I'm wondering if anyone on this list might have encountered this issue 
> before, since it's got me pretty stumped.
> 
> We have an OCaml project ( https://github.com/rems-project/sail ) which 
> tries to allocate huge amounts of memory, but only when compiled to 
> bytecode. The native version works perfectly fine.
> 
> strace for an example run ends in:
> 
> > openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail", O_RDONLY|O_CLOEXEC) = 3
> > lseek(3, 0, SEEK_CUR) = 0
> > read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
> > mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> > brk(0x5729f32c4000) = 0x55603f2f4000
> > mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> > write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
> > ) = 28
> > exit_group(2) = ?
> > +++ exited with 2 +++
> 
> which naturally fails since I don't have 2 TB of memory available. 
> 
> The exact amount of the allocation varies from input to input and even 
> run to run, but is always that approximate size. There's no 
> characteristic pattern of heap or stack running out of control if I 
> turn on memory debugging info in $OCAMLRUNPARAM, just that one huge 
> allocation. When run in ocamldebug, it also doesn't appear to be 
> failing at a location which is sensible to be allocating such amounts 
> of memory -- and the location also varies with the input. And on tiny 
> inputs it doesn't seem to try and make the allocation at all.
> 
> Are we actually seeing a compiler/runtime bug here? Is there something 
> I'm missing?
> 
> Thanks,
> 
> Jon French
> Computer Laboratory
> University of Cambridge

Re: [Caml-list] Only bytecode version of executable allocating huge amounts

[Xavier Leroy]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

On Tue, May 28, 2019 at 2:27 PM Jon French <jf451@cam.ac.uk> wrote:

> Hi all,
>
> I'm wondering if anyone on this list might have encountered this issue
> before, since it's got me pretty stumped.
>
> We have an OCaml project ( https://github.com/rems-project/sail ) which
> tries to allocate huge amounts of memory, but only when compiled to
> bytecode. The native version works perfectly fine.
>

This is unusual indeed.

Do you have a repro case that you could share publicly?  If so, please
submit an issue at https://github.com/ocaml/ocaml/issues and attach the
repro.

The first thing I would do is to increase the verbosity of the GC: set the
OCAMLRUNPARAM environment variable to the value "v=255".  The resulting GC
traces can give an idea of what's going on.

Kind regards,

- Xavier Leroy




>
> strace for an example run ends in:
>
> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail",
> O_RDONLY|O_CLOEXEC) = 3
> lseek(3, 0, SEEK_CUR)                   = 0
> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> brk(0x5729f32c4000)                     = 0x55603f2f4000
> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
> ) = 28
> exit_group(2)                           = ?
> +++ exited with 2 +++
>
>
> which naturally fails since I don't have 2 TB of memory available.
>
> The exact amount of the allocation varies from input to input and even run
> to run, but is always that approximate size. There's no characteristic
> pattern of heap or stack running out of control if I turn on memory
> debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run
> in ocamldebug, it also doesn't appear to be failing at a location which is
> sensible to be allocating such amounts of memory -- and the location also
> varies with the input. And on tiny inputs it doesn't seem to try and make
> the allocation at all.
>
> Are we actually seeing a compiler/runtime bug here? Is there something I'm
> missing?
>
> Thanks,
>
> Jon French
> Computer Laboratory
> University of Cambridge
>

[-- Attachment #2: Type: text/html, Size: 3536 bytes --]

Re: [Caml-list] Only bytecode version of executable allocating huge amounts

[Ivan Gotovchits]

[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]

My (common) suspect is Zarith. It uses a different backend when compiled to
bytecode, so I would start my investigation from this point :)

Hope it helps,
Ivan

On Tue, May 28, 2019 at 8:27 AM Jon French <jf451@cam.ac.uk> wrote:

> Hi all,
>
> I'm wondering if anyone on this list might have encountered this issue
> before, since it's got me pretty stumped.
>
> We have an OCaml project ( https://github.com/rems-project/sail ) which
> tries to allocate huge amounts of memory, but only when compiled to
> bytecode. The native version works perfectly fine.
>
> strace for an example run ends in:
>
> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail",
> O_RDONLY|O_CLOEXEC) = 3
> lseek(3, 0, SEEK_CUR)                   = 0
> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> brk(0x5729f32c4000)                     = 0x55603f2f4000
> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
> ) = 28
> exit_group(2)                           = ?
> +++ exited with 2 +++
>
>
> which naturally fails since I don't have 2 TB of memory available.
>
> The exact amount of the allocation varies from input to input and even run
> to run, but is always that approximate size. There's no characteristic
> pattern of heap or stack running out of control if I turn on memory
> debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run
> in ocamldebug, it also doesn't appear to be failing at a location which is
> sensible to be allocating such amounts of memory -- and the location also
> varies with the input. And on tiny inputs it doesn't seem to try and make
> the allocation at all.
>
> Are we actually seeing a compiler/runtime bug here? Is there something I'm
> missing?
>
> Thanks,
>
> Jon French
> Computer Laboratory
> University of Cambridge
>

[-- Attachment #2: Type: text/html, Size: 2942 bytes --]

Re: [Caml-list] Only bytecode version of executable allocating huge amounts

[Fabrice Le Fessant]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

A common difference between bytecode and native programs is that the native
compiler computes the liveness of variables in the stack, so that they are
not scanned anymore afterwards (or coalesced with other variables). For
example, if you have a data structure that is allocated before a
computation, but not used in the computation, the structure might be kept
alive in the bytecode, but garbage collected in native code.

Le mar. 28 mai 2019 à 19:23, Ivan Gotovchits <ivg@ieee.org> a écrit :

> My (common) suspect is Zarith. It uses a different backend when compiled
> to bytecode, so I would start my investigation from this point :)
>
> Hope it helps,
> Ivan
>
> On Tue, May 28, 2019 at 8:27 AM Jon French <jf451@cam.ac.uk> wrote:
>
>> Hi all,
>>
>> I'm wondering if anyone on this list might have encountered this issue
>> before, since it's got me pretty stumped.
>>
>> We have an OCaml project ( https://github.com/rems-project/sail ) which
>> tries to allocate huge amounts of memory, but only when compiled to
>> bytecode. The native version works perfectly fine.
>>
>> strace for an example run ends in:
>>
>> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail",
>> O_RDONLY|O_CLOEXEC) = 3
>> lseek(3, 0, SEEK_CUR)                   = 0
>> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
>> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>> brk(0x5729f32c4000)                     = 0x55603f2f4000
>> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
>> ) = 28
>> exit_group(2)                           = ?
>> +++ exited with 2 +++
>>
>>
>> which naturally fails since I don't have 2 TB of memory available.
>>
>> The exact amount of the allocation varies from input to input and even
>> run to run, but is always that approximate size. There's no characteristic
>> pattern of heap or stack running out of control if I turn on memory
>> debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run
>> in ocamldebug, it also doesn't appear to be failing at a location which is
>> sensible to be allocating such amounts of memory -- and the location also
>> varies with the input. And on tiny inputs it doesn't seem to try and make
>> the allocation at all.
>>
>> Are we actually seeing a compiler/runtime bug here? Is there something
>> I'm missing?
>>
>> Thanks,
>>
>> Jon French
>> Computer Laboratory
>> University of Cambridge
>>
> --
Fabrice LE FESSANT
CEO, OCamlPro SAS

[-- Attachment #2: Type: text/html, Size: 3857 bytes --]

Re: [Caml-list] Only bytecode version of executable allocating huge amounts

[Jon French]

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]

Thanks all for your replies. I've opened a github issue here: https://github.com/ocaml/ocaml/issues/8699

On Tue, 28 May 2019, at 18:58, Fabrice Le Fessant wrote:
> A common difference between bytecode and native programs is that the native compiler computes the liveness of variables in the stack, so that they are not scanned anymore afterwards (or coalesced with other variables). For example, if you have a data structure that is allocated before a computation, but not used in the computation, the structure might be kept alive in the bytecode, but garbage collected in native code.
> 
> Le mar. 28 mai 2019 à 19:23, Ivan Gotovchits <ivg@ieee.org> a écrit :
>> My (common) suspect is Zarith. It uses a different backend when compiled to bytecode, so I would start my investigation from this point :) 
>> 
>> Hope it helps,
>> Ivan
>> 
>> On Tue, May 28, 2019 at 8:27 AM Jon French <jf451@cam.ac.uk> wrote:
>>> __
>>> Hi all,
>>> 
>>> I'm wondering if anyone on this list might have encountered this issue before, since it's got me pretty stumped.
>>> 
>>> We have an OCaml project ( https://github.com/rems-project/sail ) which tries to allocate huge amounts of memory, but only when compiled to bytecode. The native version works perfectly fine.
>>> 
>>> strace for an example run ends in:
>>> 
>>>> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail", O_RDONLY|O_CLOEXEC) = 3
>>>> lseek(3, 0, SEEK_CUR) = 0
>>>> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
>>>> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>>>> brk(0x5729f32c4000) = 0x55603f2f4000
>>>> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>>>> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
>>>> ) = 28
>>>> exit_group(2) = ?
>>>> +++ exited with 2 +++
>>> 
>>> which naturally fails since I don't have 2 TB of memory available. 
>>> 
>>> The exact amount of the allocation varies from input to input and even run to run, but is always that approximate size. There's no characteristic pattern of heap or stack running out of control if I turn on memory debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run in ocamldebug, it also doesn't appear to be failing at a location which is sensible to be allocating such amounts of memory -- and the location also varies with the input. And on tiny inputs it doesn't seem to try and make the allocation at all.
>>> 
>>> Are we actually seeing a compiler/runtime bug here? Is there something I'm missing?
>>> 
>>> Thanks,
>>> 
>>> Jon French
>>> Computer Laboratory
>>> University of Cambridge
> -- 
> Fabrice LE FESSANT
> CEO, OCamlPro SAS

[-- Attachment #2: Type: text/html, Size: 4773 bytes --]