From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 234DD7F6CC for ; Thu, 5 Feb 2015 09:56:03 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of alain.frisch@lexifi.com) identity=pra; client-ip=85.233.204.164; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="alain.frisch@lexifi.com"; x-sender="alain.frisch@lexifi.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of alain.frisch@lexifi.com) identity=mailfrom; client-ip=85.233.204.164; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="alain.frisch@lexifi.com"; x-sender="alain.frisch@lexifi.com"; x-conformance=sidf_compatible Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mx20.yaziba.net) identity=helo; client-ip=85.233.204.164; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="alain.frisch@lexifi.com"; x-sender="postmaster@mx20.yaziba.net"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0DqAAA8L9NUnKTM6VVAGoNYWcJPhXECgSNDAQEBAQERAQEBAQEGDQkJFC6EDQEFOEARCxgJFg8JAwIBAgFFBgEMCAEBiC0JN9VvAQsgjyBfhCkFkmSFWoEXNoRyiG2DPAKEEW4BgQGBQAEBAQ X-IPAS-Result: A0DqAAA8L9NUnKTM6VVAGoNYWcJPhXECgSNDAQEBAQERAQEBAQEGDQkJFC6EDQEFOEARCxgJFg8JAwIBAgFFBgEMCAEBiC0JN9VvAQsgjyBfhCkFkmSFWoEXNoRyiG2DPAKEEW4BgQGBQAEBAQ X-IronPort-AV: E=Sophos;i="5.09,522,1418079600"; d="scan'208";a="120357501" Received: from mx20.yaziba.net ([85.233.204.164]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/ADH-AES256-SHA; 05 Feb 2015 09:56:02 +0100 Received: from mta10.int.yaziba.net (unknown [10.4.20.30]) by mx20.yaziba.net (mx10.yaziba.net) with ESMTP id 241461A74C6; Thu, 5 Feb 2015 09:56:01 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mta10.int.yaziba.net (Postfix) with ESMTP id 106BCCA659; Thu, 5 Feb 2015 09:56:01 +0100 (CET) X-Virus-Scanned: amavisd-new at mta10.int.yaziba.net Received: from mta10.int.yaziba.net ([127.0.0.1]) by localhost (mta10.int.yaziba.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r1JtZeeM5ih6; Thu, 5 Feb 2015 09:56:00 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mta10.int.yaziba.net (Postfix) with ESMTP id D455FCA65A; Thu, 5 Feb 2015 09:56:00 +0100 (CET) X-Virus-Scanned: amavisd-new at mta10.int.yaziba.net Received: from mta10.int.yaziba.net ([127.0.0.1]) by localhost (mta10.int.yaziba.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id X_s-v2RUtx1O; Thu, 5 Feb 2015 09:56:00 +0100 (CET) Received: from [10.0.48.165] (unknown [185.23.92.144]) by mta10.int.yaziba.net (Postfix) with ESMTPSA id 9D168CA657; Thu, 5 Feb 2015 09:56:00 +0100 (CET) Message-ID: <54D33020.6010707@lexifi.com> Date: Thu, 05 Feb 2015 09:56:00 +0100 From: Alain Frisch Organization: LexiFi User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Enrico Tassi , caml-list@inria.fr References: <20150202103256.GA30053@birba.invalid> In-Reply-To: <20150202103256.GA30053@birba.invalid> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-DRWEB-SCAN: ok X-VRSPAM-SCORE: -100 X-VRSPAM-STATE: legit X-VRSPAM-CAUSE: gggruggvucftvghtrhhoucdtuddrfeejkedrhedvgddvgecutefuodetggdotefrucfrrhhofhhilhgvmecuggftfghnshhusghstghrihgsvgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhepkfffhfhofgggvffufhgjtgfgsehtqhgrtddtfeehnecuhfhrohhmpeetlhgrihhnucfhrhhishgthhcuoegrlhgrihhnrdhfrhhishgthheslhgvgihifhhirdgtohhmqeenucffohhmrghinhepihhnrhhirgdrfhhr X-VRSPAM-EXTCAUSE: mhhouggvpehsmhhtphhouhht Subject: Re: [Caml-list] unmarshaling large data from string on 32 bits Hello, Be aware when using the generic demarshaling on 32 bit systems with=20 large data (even when they fit in a string): this will expand the heap=20 (adding more pages to it) on every demarshaling, and unless you arrange=20 so that the compacter runs often enough (calling manually Gc.compact for=20 instance), you'll end up eating all the memory. This is documented here: http://caml.inria.fr/mantis/view.php?id=3D5813 One possible work-around is to use an alternative implementation of the=20 demarshaler (there is such a pure OCaml implementation in Frama-C).=20 Another is to avoid the generic marshaling, either by writing a manual=20 version for your specific data type or by generating it from your type=20 definitions (=E0 la bin-prot, I assume). Alain On 02/02/2015 11:32 AM, Enrico Tassi wrote: > Hello, I've just discovered that on 32 bits systems strings are > limited to 16M. I'm using strings as buffers holding data to > be unmarshaled. I could use another data structure, like a Buffer.t, > but I see no API for unmarshaling from a Buffer.t. > > Is there another way? Is there code out there implementing that? > > Best, >