Re: [Caml-list] unmarshaling large data from string on 32 bits

[Alain Frisch <alain.frisch@lexifi.com>]

On 02/05/2015 10:01 AM, Gabriel Scherer wrote:
>> One possible work-around is to use an alternative implementation of the demarshaler (there is such a pure OCaml implementation in Frama-C).
>
> Is this implementation publicly available somewhere?

In the Frama-C distribution, external/unmarshal.ml

-- Alain

>
> On Thu, Feb 5, 2015 at 9:56 AM, Alain Frisch <alain.frisch@lexifi.com> wrote:
>> Hello,
>>
>> Be aware when using the generic demarshaling on 32 bit systems with large
>> data (even when they fit in a string):  this will expand the heap (adding
>> more pages to it) on every demarshaling, and unless you arrange so that the
>> compacter runs often enough (calling manually Gc.compact for instance),
>> you'll end up eating all the memory.
>>
>> This is documented here:
>>
>> http://caml.inria.fr/mantis/view.php?id=5813
>>
>> One possible work-around is to use an alternative implementation of the
>> demarshaler (there is such a pure OCaml implementation in Frama-C). Another
>> is to avoid the generic marshaling, either by writing a manual version for
>> your specific data type or by generating it from your type definitions (à la
>> bin-prot, I assume).
>>
>>
>> Alain
>>
>>
>>
>> On 02/02/2015 11:32 AM, Enrico Tassi wrote:
>>>
>>> Hello, I've just discovered that on 32 bits systems strings are
>>> limited to 16M.  I'm using strings as buffers holding data to
>>> be unmarshaled.  I could use another data structure, like a Buffer.t,
>>> but I see no API for unmarshaling from a Buffer.t.
>>>
>>> Is there another way? Is there code out there implementing that?
>>>
>>> Best,
>>>
>>
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>