From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 74B0D7F6CC for ; Thu, 5 Feb 2015 13:23:10 +0100 (CET) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of fabrissimo@gmail.com) identity=pra; client-ip=209.85.216.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="fabrissimo@gmail.com"; x-sender="fabrissimo@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of fabrissimo@gmail.com designates 209.85.216.177 as permitted sender) identity=mailfrom; client-ip=209.85.216.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="fabrissimo@gmail.com"; x-sender="fabrissimo@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-qc0-f177.google.com) identity=helo; client-ip=209.85.216.177; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="fabrissimo@gmail.com"; x-sender="postmaster@mail-qc0-f177.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CxAQBjX9NUlLHYVdFag1hZBIJ9r36PTYVxgSAHQwEBAQEBEQEBAQEHCwsJEjCEDQEBBBIRHQEtCwEDDAEFBQsNAgIJHQICIhIBBQEKBAENBhMSEId3AxENsx4+MYsuhGKLTycDCoVWAQsaAQUOgROOABISgyKBQQWETAqODoVagRc2jV+BdBIjgRVbgzY9MQGBAoE/AQEB X-IPAS-Result: A0CxAQBjX9NUlLHYVdFag1hZBIJ9r36PTYVxgSAHQwEBAQEBEQEBAQEHCwsJEjCEDQEBBBIRHQEtCwEDDAEFBQsNAgIJHQICIhIBBQEKBAENBhMSEId3AxENsx4+MYsuhGKLTycDCoVWAQsaAQUOgROOABISgyKBQQWETAqODoVagRc2jV+BdBIjgRVbgzY9MQGBAoE/AQEB X-IronPort-AV: E=Sophos;i="5.09,523,1418079600"; d="scan'208";a="120398255" Received: from mail-qc0-f177.google.com ([209.85.216.177]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 05 Feb 2015 13:23:09 +0100 Received: by mail-qc0-f177.google.com with SMTP id p6so6003536qcv.8; Thu, 05 Feb 2015 04:23:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=J76bexpvmlRl4pnq4tIv7n+vgpCPs1L0d/SnFRJH2Bo=; b=q1dkBObymvX/SMWxg+9G6FjMugJ4e5JHB9f0IxGv9KU/9Gi8BR9vtxJFxa1DKQ5JJ7 UV01j8bdVteR8FA/C/zRaBQ2ebvaRZ66Ow4idWdfVs2mrZoqVlhLoOP35y1Nty9wbz2z WOsJbUF3JQ6ih+BmpBf606Ww0FpPyleKdNWZKLf06MhDmJJ/Aku67O2Zpk2XuVhAuUvd FpVOCH7ihq6R/zpl/pJXo7DYZi2ALt0IxDAj8mp2iPNJfjbIHUdLAkWmChQ/ypDSoR6X V9JVqb7RoRfJHCxFQXGXo9Qrd0qOWWlOaDaA1sQ0GY6FrLOYhk34gjJo8Zkb1OlrmL+r VynA== X-Received: by 10.140.93.139 with SMTP id d11mr6873799qge.101.1423138988888; Thu, 05 Feb 2015 04:23:08 -0800 (PST) MIME-Version: 1.0 Sender: fabrissimo@gmail.com Received: by 10.96.86.98 with HTTP; Thu, 5 Feb 2015 04:22:48 -0800 (PST) In-Reply-To: <54D34AF2.1010001@lexifi.com> References: <20150202103256.GA30053@birba.invalid> <54D33020.6010707@lexifi.com> <54D33EC3.2050809@inria.fr> <54D34AF2.1010001@lexifi.com> From: Fabrice Le Fessant Date: Thu, 5 Feb 2015 13:22:48 +0100 X-Google-Sender-Auth: Ui1ZS2VQPgVj_5bnMiuzhnxFSzM Message-ID: To: Alain Frisch Cc: =?UTF-8?Q?Pierre=2DMarie_P=C3=A9drot?= , Ocaml Mailing List Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Caml-list] unmarshaling large data from string on 32 bits Hi, A long time ago, we had a patch on 32 bit systems to transparently extend the size of strings and arrays over the 16 MB limit: http://www.ocamlpro.com/blog/2011/05/06/longval.html At the time, there was little interest in this patch. If I remember well, it was quite small, we could probably include it in the next version of OCPWin32, if you think having it under Windows would help you. --Fabrice On Thu, Feb 5, 2015 at 11:50 AM, Alain Frisch wro= te: > On 02/05/2015 10:58 AM, Pierre-Marie P=C3=A9drot wrote: >> >> On 05/02/2015 09:56, Alain Frisch wrote: >>> >>> One possible work-around is to use an alternative implementation of the >>> demarshaler (there is such a pure OCaml implementation in Frama-C). >>> Another is to avoid the generic marshaling, either by writing a manual >>> version for your specific data type or by generating it from your type >>> definitions (=C3=A0 la bin-prot, I assume). >> >> >> Both workarounds would not work. Indeed, we use closure marshalling in >> Coq, which is not supported by the two proposed implementations. > > > Waow, closure marshaling across processes: you live dangerously :-) I ho= pe > you know precisely which values go into the closures or not (exception > slots, global mutable data structures, etc)... > > >> (Plus, that would be so slow I do not even want to think about it.) > > > I'm not so sure. Serialization/deserialization routines specialized for > specific data types avoid some runtime checks required by the generic > functions (for the block tags and sizes, for detecting possible sharing > everywhere) and can use more specialized (and thus more compact) data > representation. > > We have had that problem at LexiFi, where we used to rely on the generic > marshaling for exchanges messages between processes (hence the ticket I > referred to). We finally decided to write (manually) specialized functio= ns > for our type of messages (no closures, no sharing), and the performance > results where slower but reasonable compared to the generic marshaling > (without putting too much engineering effort into it). Anyway: > > - our previous workaround was to trigger Gc.compact explicitly for big > messages, which was much worse, of course; > > - it's clear that the OCaml implementation of the generic demarshaler wo= uld > be slower; > > - since this only impacts 32-bit systems, nobody seems motivated enough = to > put energy into fixing the core issue. > > > Alain > > > -- > Caml-list mailing list. Subscription management and archives: > https://sympa.inria.fr/sympa/arc/caml-list > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners > Bug reports: http://caml.inria.fr/bin/caml-bugs --=20 Fabrice LE FESSANT Chercheur en Informatique INRIA Paris Rocquencourt -- OCamlPro Programming Languages and Distributed Systems