From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by sympa.inria.fr (Postfix) with ESMTPS id 7A8077F6CC for ; Mon, 2 Feb 2015 18:01:07 +0100 (CET) Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of dhekuir@gmail.com) identity=pra; client-ip=209.85.160.193; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="dhekuir@gmail.com"; x-sender="dhekuir@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail3-smtp-sop.national.inria.fr: domain of dhekuir@gmail.com designates 209.85.160.193 as permitted sender) identity=mailfrom; client-ip=209.85.160.193; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="dhekuir@gmail.com"; x-sender="dhekuir@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail3-smtp-sop.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-yk0-f193.google.com) identity=helo; client-ip=209.85.160.193; receiver=mail3-smtp-sop.national.inria.fr; envelope-from="dhekuir@gmail.com"; x-sender="postmaster@mail-yk0-f193.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CPAQA2rM9Um8GgVdFbhDEEgn2/D4hJAoEXB0MBAQEBAREBAQEBAQYLCwkULoQMAQEBAwESEQQZARsdAQMBCwYFCw0qAgIiAREBBQEcBhMbB4d2AQMBBAQItDg+MYsugWuCd4phChknDRg8hHwBAQEBAQEEAQEBAQEBFgEFDo8CaAeCaIFBBZgugReFR4hBgXMSI4EMCYIkHIFRPTGCQgEBAQ X-IPAS-Result: A0CPAQA2rM9Um8GgVdFbhDEEgn2/D4hJAoEXB0MBAQEBAREBAQEBAQYLCwkULoQMAQEBAwESEQQZARsdAQMBCwYFCw0qAgIiAREBBQEcBhMbB4d2AQMBBAQItDg+MYsugWuCd4phChknDRg8hHwBAQEBAQEEAQEBAQEBFgEFDo8CaAeCaIFBBZgugReFR4hBgXMSI4EMCYIkHIFRPTGCQgEBAQ X-IronPort-AV: E=Sophos;i="5.09,507,1418079600"; d="scan'208";a="98568661" Received: from mail-yk0-f193.google.com ([209.85.160.193]) by mail3-smtp-sop.national.inria.fr with ESMTP/TLS/RC4-SHA; 02 Feb 2015 18:00:59 +0100 Received: by mail-yk0-f193.google.com with SMTP id q9so5682453ykb.0 for ; Mon, 02 Feb 2015 09:00:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=52yufrLj2omdPciXRtluHQ0iGQbGErqyO1LWgap371w=; b=pPJ5EXM2VeufmXEVgpxHuJ9fjLdI/xYslxBO+f9ZAVgwywkUrLX1ifH6ouE9SUKq6a OozlzhiIDjHOBKku1zAA/fV58xcgE1fFO8GXmQl8QVne6yJg6R0jbSmyUdt62abRfosz vrbeGoP9KiVIvhb+rxlOw5OQrhEeqhgha8f8RwVuN5gYXhXhx2r/T0NEYZ50h33o8Mht md/X1AIZqPTsWq0aLW1XTShMu7TY/4oE3eyS8Wpa6eGnULK6UORp0S94K35k+Kcy5llo hjUnbgz4J/SadUy4BAO2p4gAbC3i06YfGDSPlj/LGtDMKToQcxYDbVVdQMmswMb2syqi zwPg== MIME-Version: 1.0 X-Received: by 10.170.115.209 with SMTP id h200mr9429142ykb.87.1422896456229; Mon, 02 Feb 2015 09:00:56 -0800 (PST) Received: by 10.170.217.194 with HTTP; Mon, 2 Feb 2015 09:00:56 -0800 (PST) In-Reply-To: <54CFAA41.20706@zoho.com> References: <54CF9B20.5040300@inria.fr> <54CFAA41.20706@zoho.com> Date: Mon, 2 Feb 2015 18:00:56 +0100 Message-ID: From: Dhek Uir To: Drup Cc: caml-list@inria.fr Content-Type: multipart/alternative; boundary=001a1137bdb4d36818050e1de526 Subject: Re: [Caml-list] Simple compression library --001a1137bdb4d36818050e1de526 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable But since camlzip does not have an input_value function, it seems I must set up a buffer to decompress the data, then use Marshal to obtain input data from the decompressed buffer. But if I use Marshal.from_string to read the decompressed buffer, then I need to keep track of its position (which is not returned by Marshal.from_string), and check whether the buffer is nearly empty, so that I may need to re-read from the compressed in_channel, etc, etc. Did I miss the right function again? It seems that Netgzip would require some similar amount of effort, from what I'm seeing (I see no "channel -> 'a" functions in the API). Thanks for the serialization tips, though I don't care much about the safety, I have strong assumptions about the how the data will be produced and consumed. On Mon, Feb 2, 2015 at 5:48 PM, Drup wrote: > input/output_value are using Marshal under the hood, so it's going to be > the same. > > However, be aware that marshall (and the _value functions) are quite > unsafe. If you have an error in the data (or your change the representati= on > in OCaml), you will have ... surprising errors (including segfaults). > > There are plenty of safer solutions for serialization (deriving, sexp, > yojson, to name a few). > > Le 02/02/2015 17:05, Dhek Uir a =C3=A9crit : > > > > On Mon, Feb 2, 2015 at 4:43 PM, Jacques-Henri Jourdan < > jacques-henri.jourdan@inria.fr> wrote: > >> Le 02/02/2015 16:39, Dhek Uir a =C3=A9crit : >> > I'm looking for a compression library such as camlzip to read/write >> > compressed files (any lossless compression algorithm is fine), but >> > having a function similar to Pervasives.output_value, which does not >> > seem to be available in camlzip (and I do not see how I could easily >> > implement it). >> > >> > Would it be simple to modify camlzip to add this function? I believe >> > that if this were the case, then it would already have been done, but >> > I might be wrong. >> > >> > Otherwise, is there another OCaml library with an API very similar to >> > Pervasives'? I'd prefer a lightweight solution (avoiding Core and >> > Batteries) if possible. >> >> Can't you do Marshal.to_string, and then use Camlzip on the generated >> string ? >> >> > Indeed, I can. I believe the hard part is actually the input_value > operation. > But I haven't looked enough into Marshal, maybe my solution can be found > there. > > > --001a1137bdb4d36818050e1de526 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
But = since camlzip does not have an input_value function, it seems I must set up= a buffer to decompress the data, then use Marshal to obtain input data fro= m the decompressed buffer. But if I use Marshal.from_string to read the dec= ompressed buffer, then I need to keep track of its position (which is not r= eturned by Marshal.from_string), and check whether the buffer is nearly emp= ty, so that I may need to re-read from the compressed in_channel, etc, etc.= Did I miss the right function again?

It seems that Netgzip would require some similar amount of effort, from = what I'm seeing (I see no "channel -> 'a" functions in= the API).

Thanks for the serialization tips, though I don't car= e much about the safety, I have strong assumptions about the how the data w= ill be produced and consumed.


On Mon, Feb 2, 2015 at 5:48 PM, Dr= up <drupyog+caml@zoho.com> wrote:
=20=20 =20=20=20=20 =20=20
input/output_value are using Marshal under the hood, so it's going to be the same.

However, be aware that marshall (and the _value functions) are quite unsafe. If you have an error in the data (or your change the representation in OCaml), you will have ... surprising errors (including segfaults).

There are plenty of safer solutions for serialization (deriving, sexp, yojson, to name a few).

Le 02/02/2015 17:05, Dhek Uir a =C3=A9crit=C2=A0:


On Mon, Feb 2, 2015 at 4:43 PM, Jacques-Henri Jourdan <jacques-henri.jourdan@inri= a.fr> wrote:
Le 02/02/2015 16:39, Dhek Uir a =C3=A9crit :
> I'm looking for a compression library such as camlzip to read/write
> compressed files (any lossless compression algorithm is fine), but
> having a function similar to Pervasives.output_value, which does not
> seem to be available in camlzip (and I do not see how I could easily
> implement it).
>
> Would it be simple to modify camlzip to add this function? I believe
> that if this were the case, then it would already have been done, but
> I might be wrong.
>
> Otherwise, is there another OCaml library with an API very similar to
> Pervasives'? I'd prefer a lightweight soluti= on (avoiding Core and
> Batteries) if possible.

Can't you do Marshal.to_string, and then use Camlzip on the generated
string ?

=C2=A0
Indeed, I can. I believe the hard part is actually the input_value operation.
But I haven't looked enough into Marshal, maybe my solution can be found there.


--001a1137bdb4d36818050e1de526--