caml-list - the Caml user's mailing list
 help / color / Atom feed
From: Fabrice Le Fessant <fabrice.le_fessant@ocamlpro.com>
To: Ivan Gotovchits <ivg@ieee.org>
Cc: Jon French <jf451@cam.ac.uk>, caml-list <caml-list@inria.fr>
Subject: Re: [Caml-list] Only bytecode version of executable allocating huge amounts
Date: Tue, 28 May 2019 19:58:03 +0200
Message-ID: <CAHvkLrP+SAWHc72oP4r9LBSPPiJhhAcy=bpJDR66dM=qJLqDkQ@mail.gmail.com> (raw)
In-Reply-To: <CALdWJ+xdkibtwpyvvyA+chApq1uNoN-QoPjt0crjLOSvhLM6Fw@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

A common difference between bytecode and native programs is that the native
compiler computes the liveness of variables in the stack, so that they are
not scanned anymore afterwards (or coalesced with other variables). For
example, if you have a data structure that is allocated before a
computation, but not used in the computation, the structure might be kept
alive in the bytecode, but garbage collected in native code.

Le mar. 28 mai 2019 à 19:23, Ivan Gotovchits <ivg@ieee.org> a écrit :

> My (common) suspect is Zarith. It uses a different backend when compiled
> to bytecode, so I would start my investigation from this point :)
>
> Hope it helps,
> Ivan
>
> On Tue, May 28, 2019 at 8:27 AM Jon French <jf451@cam.ac.uk> wrote:
>
>> Hi all,
>>
>> I'm wondering if anyone on this list might have encountered this issue
>> before, since it's got me pretty stumped.
>>
>> We have an OCaml project ( https://github.com/rems-project/sail ) which
>> tries to allocate huge amounts of memory, but only when compiled to
>> bytecode. The native version works perfectly fine.
>>
>> strace for an example run ends in:
>>
>> openat(AT_FDCWD, "/home/ojno/work/sail2/lib/vector_dec.sail",
>> O_RDONLY|O_CLOEXEC) = 3
>> lseek(3, 0, SEEK_CUR)                   = 0
>> read(3, "$ifndef _VECTOR_DEC\n$define _VEC"..., 65536) = 7031
>> mmap(NULL, 1965819695104, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>> brk(0x5729f32c4000)                     = 0x55603f2f4000
>> mmap(NULL, 1965819826176, PROT_READ|PROT_WRITE,
>> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>> write(2, "Fatal error: out of memory.\n", 28Fatal error: out of memory.
>> ) = 28
>> exit_group(2)                           = ?
>> +++ exited with 2 +++
>>
>>
>> which naturally fails since I don't have 2 TB of memory available.
>>
>> The exact amount of the allocation varies from input to input and even
>> run to run, but is always that approximate size. There's no characteristic
>> pattern of heap or stack running out of control if I turn on memory
>> debugging info in $OCAMLRUNPARAM, just that one huge allocation. When run
>> in ocamldebug, it also doesn't appear to be failing at a location which is
>> sensible to be allocating such amounts of memory -- and the location also
>> varies with the input. And on tiny inputs it doesn't seem to try and make
>> the allocation at all.
>>
>> Are we actually seeing a compiler/runtime bug here? Is there something
>> I'm missing?
>>
>> Thanks,
>>
>> Jon French
>> Computer Laboratory
>> University of Cambridge
>>
> --
Fabrice LE FESSANT
CEO, OCamlPro SAS

[-- Attachment #2: Type: text/html, Size: 3857 bytes --]

  reply index

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-28 11:32 Jon French
2019-05-28 16:49 ` rixed
2019-05-28 17:16 ` Xavier Leroy
2019-05-28 17:23 ` Ivan Gotovchits
2019-05-28 17:58   ` Fabrice Le Fessant [this message]
2019-05-29 15:22     ` Jon French

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHvkLrP+SAWHc72oP4r9LBSPPiJhhAcy=bpJDR66dM=qJLqDkQ@mail.gmail.com' \
    --to=fabrice.le_fessant@ocamlpro.com \
    --cc=caml-list@inria.fr \
    --cc=ivg@ieee.org \
    --cc=jf451@cam.ac.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

caml-list - the Caml user's mailing list

Archives are clonable:
	git clone --mirror http://inbox.vuxu.org/caml-list
	git clone --mirror https://inbox.ocaml.org/caml-list

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://inbox.vuxu.org/vuxu.archive.caml-list


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git