Re: [Caml-list] unmarshaling large data from string on 32 bits

[Gabriel Scherer <gabriel.scherer@gmail.com>]

> One possible work-around is to use an alternative implementation of the demarshaler (there is such a pure OCaml implementation in Frama-C).

Is this implementation publicly available somewhere?

On Thu, Feb 5, 2015 at 9:56 AM, Alain Frisch <alain.frisch@lexifi.com> wrote:
> Hello,
>
> Be aware when using the generic demarshaling on 32 bit systems with large
> data (even when they fit in a string):  this will expand the heap (adding
> more pages to it) on every demarshaling, and unless you arrange so that the
> compacter runs often enough (calling manually Gc.compact for instance),
> you'll end up eating all the memory.
>
> This is documented here:
>
> http://caml.inria.fr/mantis/view.php?id=5813
>
> One possible work-around is to use an alternative implementation of the
> demarshaler (there is such a pure OCaml implementation in Frama-C). Another
> is to avoid the generic marshaling, either by writing a manual version for
> your specific data type or by generating it from your type definitions (à la
> bin-prot, I assume).
>
>
> Alain
>
>
>
> On 02/02/2015 11:32 AM, Enrico Tassi wrote:
>>
>> Hello, I've just discovered that on 32 bits systems strings are
>> limited to 16M.  I'm using strings as buffers holding data to
>> be unmarshaled.  I could use another data structure, like a Buffer.t,
>> but I see no API for unmarshaling from a Buffer.t.
>>
>> Is there another way? Is there code out there implementing that?
>>
>> Best,
>>
>
>
> --
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs