From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 22DE87F6CC for ; Thu, 5 Feb 2015 10:01:49 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of gabriel.scherer@gmail.com) identity=pra; client-ip=209.85.223.171; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of gabriel.scherer@gmail.com designates 209.85.223.171 as permitted sender) identity=mailfrom; client-ip=209.85.223.171; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="gabriel.scherer@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-ie0-f171.google.com) identity=helo; client-ip=209.85.223.171; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="gabriel.scherer@gmail.com"; x-sender="postmaster@mail-ie0-f171.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CxAQAzMNNUlKvfVdFAGoNYWQSCfbABj02FcQKBHAdDAQEBAQERAQEBAQcLCwkSMIQMAQEBAwESER0BGxILAQMBCwYFCw0CAgkdAgIiAREBBQEKEgYTEhCHdgEDCQgNN7MAPjGLLoFrgneLOAoZJwMKVIUCAQEBAQEFAQEBAQEXAQUOgRONfyUzB4JogUEFhEwKjg6FWoEXNo1fgXQSI4EMCYQRPTEBgQGBQAEBAQ X-IPAS-Result: A0CxAQAzMNNUlKvfVdFAGoNYWQSCfbABj02FcQKBHAdDAQEBAQERAQEBAQcLCwkSMIQMAQEBAwESER0BGxILAQMBCwYFCw0CAgkdAgIiAREBBQEKEgYTEhCHdgEDCQgNN7MAPjGLLoFrgneLOAoZJwMKVIUCAQEBAQEFAQEBAQEXAQUOgRONfyUzB4JogUEFhEwKjg6FWoEXNo1fgXQSI4EMCYQRPTEBgQGBQAEBAQ X-IronPort-AV: E=Sophos;i="5.09,522,1418079600"; d="scan'208";a="120359160" Received: from mail-ie0-f171.google.com ([209.85.223.171]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 05 Feb 2015 10:01:48 +0100 Received: by mail-ie0-f171.google.com with SMTP id tr6so8630486ieb.2; Thu, 05 Feb 2015 01:01:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=9mM9qjPUmeeBLZz1VZvR1lW7cBA/OPgR0j80+K7zbIA=; b=tizqn4K3T+2X28JnptKy6JsOmCz2r9uXn9Z2FuRrz+LxJq4b3m1o415yblWL936r1d vQoud1HSo9CJmjcxFhqiaNPIAN+gfMqRurYnLIeVyk8tDQpFbRQfMarQf468pgnCxDFD ZX+2BYA+PnXSCJ74e1eZ4+40QNSDkhmpVwmxug5h4wD4opu8KppoyAn7BRZ4foYyxKdk BYz1z79an1qnQrEyQsexsWHEEqYJF5H7xSsVpelMCWa6YRnZdDG/lUoG8/eGDlI0+2/X KfwEmIBrAD8HD9DlXa1BWI8hHn/skKO1Yhn7Tvhmj9hQg2hT9ebUCIyR8xVNU5S+zYXM ecmA== X-Received: by 10.42.83.147 with SMTP id h19mr5956161icl.95.1423126907049; Thu, 05 Feb 2015 01:01:47 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.13.195 with HTTP; Thu, 5 Feb 2015 01:01:06 -0800 (PST) In-Reply-To: <54D33020.6010707@lexifi.com> References: <20150202103256.GA30053@birba.invalid> <54D33020.6010707@lexifi.com> From: Gabriel Scherer Date: Thu, 5 Feb 2015 10:01:06 +0100 Message-ID: To: Alain Frisch Cc: Enrico Tassi , caml users Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Caml-list] unmarshaling large data from string on 32 bits > One possible work-around is to use an alternative implementation of the d= emarshaler (there is such a pure OCaml implementation in Frama-C). Is this implementation publicly available somewhere? On Thu, Feb 5, 2015 at 9:56 AM, Alain Frisch wrot= e: > Hello, > > Be aware when using the generic demarshaling on 32 bit systems with large > data (even when they fit in a string): this will expand the heap (adding > more pages to it) on every demarshaling, and unless you arrange so that t= he > compacter runs often enough (calling manually Gc.compact for instance), > you'll end up eating all the memory. > > This is documented here: > > http://caml.inria.fr/mantis/view.php?id=3D5813 > > One possible work-around is to use an alternative implementation of the > demarshaler (there is such a pure OCaml implementation in Frama-C). Anoth= er > is to avoid the generic marshaling, either by writing a manual version for > your specific data type or by generating it from your type definitions (= =C3=A0 la > bin-prot, I assume). > > > Alain > > > > On 02/02/2015 11:32 AM, Enrico Tassi wrote: >> >> Hello, I've just discovered that on 32 bits systems strings are >> limited to 16M. I'm using strings as buffers holding data to >> be unmarshaled. I could use another data structure, like a Buffer.t, >> but I see no API for unmarshaling from a Buffer.t. >> >> Is there another way? Is there code out there implementing that? >> >> Best, >> > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs