Escaped string in sexplib

Escaped string in sexplib

[Hugo Ferreira]

Hello,

I am trying to us the Sexplib library to process data.
However I am having some trouble with strings.
Specifically I need to save and restore strings to a file
so that the escaping done during saving is "undone"
when data is read back in.

How can I do this? Appreciate any pointers.

TIA.
Hugo F.

Re: [Caml-list] Escaped string in sexplib

[Paolo Donadeo]

I'm using Sexplib to store and retrieve the sessions of a web
application into a data base. A "session" has this signature:

module type SESSION_DATA =
  sig
    type t
    val default : t
    val string_of_t : t -> string (* Is this what you want? *)
    val t_of_string : string -> t (* Is this what you want? *)
  end

and the concrete session, which is a blog session, is:

module Blog_session =
  struct
    type user =
      {
        email : string;
        passwd : string;
        first_name : string option;
        last_name : string option;
        displ_name : string option;
        website : string option
      } with sexp

    type t =
      | Anonymous
      | Logged of user
        with sexp

    let default = Anonymous

    let string_of_t = sexp_of_t |- Sexplib.Sexp.to_string_hum |- hex_encode

    let t_of_string s =
      try s |> hex_decode |> Sexplib.Sexp.of_string |> t_of_sexp
      with Sexplib.Conv.Of_sexp_error (_, _) -> default
  end

Functions "hex_encode" and "hex_decode" are trivially implemented
using Cryptokit:

let hex_encode = Cryptokit.transform_string (Cryptokit.Hexa.encode ())
let hex_decode = Cryptokit.transform_string (Cryptokit.Hexa.decode ())

The operators (|-) and (|>) are defined as always:

http://thelema.github.com/batteries-included/hdoc/BatPervasives.html#6_Fundamentalfunctionsandoperators


Hope this helps,


-- 
Paolo

Re: [Caml-list] Escaped string in sexplib

[Markus Mottl]

On Tue, Jun 8, 2010 at 11:44, Hugo Ferreira <hmf@inescporto.pt> wrote:
> I am trying to us the Sexplib library to process data.
> However I am having some trouble with strings.
> Specifically I need to save and restore strings to a file
> so that the escaping done during saving is "undone"
> when data is read back in.
>
> How can I do this? Appreciate any pointers.

The I/O and parsing routines in the S-expression library should take
care of escaped strings just fine.  Btw., the escaping conventions are
absolutely identical to the ones used by OCaml.

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Hello,

Markus Mottl wrote:
> On Tue, Jun 8, 2010 at 11:44, Hugo Ferreira <hmf@inescporto.pt> wrote:
>> I am trying to us the Sexplib library to process data.
>> However I am having some trouble with strings.
>> Specifically I need to save and restore strings to a file
>> so that the escaping done during saving is "undone"
>> when data is read back in.
>>
>> How can I do this? Appreciate any pointers.
> 
> The I/O and parsing routines in the S-expression library should take
> care of escaped strings just fine.  

My apologies but I did not explain myself correctly. The problem here is
that I am trying to automate the conversion to string of the data
structure read from the s-expressions. I have the following:

let string_of_experiment_data data =
   let data = sexp_of_experiment_data data in
   Sexplib.Sexp.to_string_hum data

The problem here is that the "to_string_hum" escapes certain characters,
but I need to have the strings "not escaped". So my question is: is
their any way I can tell sexplib to output a string but not escape
it? Otherwise I will have to hand encode this stuff 8-(

> Btw., the escaping conventions are
> absolutely identical to the ones used by OCaml.
> 

I am aware of this as it is indicated in your documentation and
the ocaml tutorial site. I also searched for a means to "unescape"
the result from "to_string_hum", which would also work. However
I cannot find a String.unescape. This issue has already been pointed
out. A possible solution would be to use the old.

  Token.eval_string

and I cannot seem to use the equivalent of the new camlp4. The
equivalent:

open Camlp4.PreCast
open Syntax
let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s

does not compile (Error: Unbound module Camlp4.PreCast).

Can anyone suggest a solution or correction?

TIA,
Hugo F.







> Regards,
> Markus
> 

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Hi Paolo,

Paolo Donadeo wrote:
> I'm using Sexplib to store and retrieve the sessions of a web
> application into a data base. A "session" has this signature:
> 
> module type SESSION_DATA =
>   sig
>     type t
>     val default : t
>     val string_of_t : t -> string (* Is this what you want? *)
>     val t_of_string : string -> t (* Is this what you want? *)
>   end
> 
> and the concrete session, which is a blog session, is:
> 
> module Blog_session =
>   struct
>     type user =
>       {
>         email : string;
>         passwd : string;
>         first_name : string option;
>         last_name : string option;
>         displ_name : string option;
>         website : string option
>       } with sexp
> 
>     type t =
>       | Anonymous
>       | Logged of user
>         with sexp
> 
>     let default = Anonymous
> 
>     let string_of_t = sexp_of_t |- Sexplib.Sexp.to_string_hum |- hex_encode
> 
>     let t_of_string s =
>       try s |> hex_decode |> Sexplib.Sexp.of_string |> t_of_sexp
>       with Sexplib.Conv.Of_sexp_error (_, _) -> default
>   end
> 

My be a solution. Going to see if I can find an alternate
implementation of hex encoding so as not to require another
library.

Thanks,
Hugo F.


> Functions "hex_encode" and "hex_decode" are trivially implemented
> using Cryptokit:
> 
> let hex_encode = Cryptokit.transform_string (Cryptokit.Hexa.encode ())
> let hex_decode = Cryptokit.transform_string (Cryptokit.Hexa.decode ())
> 
> The operators (|-) and (|>) are defined as always:
> 
> http://thelema.github.com/batteries-included/hdoc/BatPervasives.html#6_Fundamentalfunctionsandoperators
> 
> 
> Hope this helps,
> 
> 

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Hello,

Hugo Ferreira wrote:
> Hello,
> 
snip...
>> Btw., the escaping conventions are
>> absolutely identical to the ones used by OCaml.
>>
> 
> I am aware of this as it is indicated in your documentation and
> the ocaml tutorial site. I also searched for a means to "unescape"
> the result from "to_string_hum", which would also work. However
> I cannot find a String.unescape. This issue has already been pointed
> out. A possible solution would be to use the old.
> 
>  Token.eval_string
> 
> and I cannot seem to use the equivalent of the new camlp4. The
> equivalent:
> 
> open Camlp4.PreCast
> open Syntax
> let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s
> 
> does not compile (Error: Unbound module Camlp4.PreCast).
> 
> Can anyone suggest a solution or correction?
>

After a correction to the ocamlbuild tags file the above works
correctly. So now I have:

open Camlp4.PreCast
open Syntax
let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s


let string_of_experiment_data data =
   let data = sexp_of_experiment_data data in
   let s = Sexplib.Sexp.to_string_hum data in
   eval_string s


which does the job. Don't know if the hex-encoding strategy
is better though.

Regards,
Hugo F.


> TIA,
> Hugo F.
> 
> 
> 
> 
> 
> 
> 
>> Regards,
>> Markus
>>
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 

Re: [Caml-list] Escaped string in sexplib

[Markus Mottl]

On Tue, Jun 8, 2010 at 14:52, Hugo Ferreira <hmf@inescporto.pt> wrote:
> My apologies but I did not explain myself correctly. The problem here is
> that I am trying to automate the conversion to string of the data
> structure read from the s-expressions. I have the following:
>
> let string_of_experiment_data data =
>  let data = sexp_of_experiment_data data in
>  Sexplib.Sexp.to_string_hum data
>
> The problem here is that the "to_string_hum" escapes certain characters,
> but I need to have the strings "not escaped". So my question is: is
> their any way I can tell sexplib to output a string but not escape
> it? Otherwise I will have to hand encode this stuff 8-(

Sexplib does not support any other way of printing S-expressions.
There would not be any reliable way to read back data without
escaping.

If you want to write your own printers, this should still be very
easy.  S-expressions are either a Sexp.Atom or Sexp.List.  It should
only require a handful of lines to convert these to whatever other
string representation you prefer.  Writing a corresponding parser may
be more effort.

Regards,
Markus

-- 
Markus Mottl        http://www.ocaml.info        markus.mottl@gmail.com

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Markus Mottl wrote:
> On Tue, Jun 8, 2010 at 14:52, Hugo Ferreira <hmf@inescporto.pt> wrote:
>> My apologies but I did not explain myself correctly. The problem here is
>> that I am trying to automate the conversion to string of the data
>> structure read from the s-expressions. I have the following:
>>
>> let string_of_experiment_data data =
>>  let data = sexp_of_experiment_data data in
>>  Sexplib.Sexp.to_string_hum data
>>
>> The problem here is that the "to_string_hum" escapes certain characters,
>> but I need to have the strings "not escaped". So my question is: is
>> their any way I can tell sexplib to output a string but not escape
>> it? Otherwise I will have to hand encode this stuff 8-(
> 
> Sexplib does not support any other way of printing S-expressions.
> There would not be any reliable way to read back data without
> escaping.
> 
> If you want to write your own printers, this should still be very
> easy.  S-expressions are either a Sexp.Atom or Sexp.List.  It should
> only require a handful of lines to convert these to whatever other
> string representation you prefer.  Writing a corresponding parser may
> be more effort.
> 

Understood. I will avoid this.
I will stick to the solution I pointed out in my last e-mail

Once gain thanks.
Hugo F.


> Regards,
> Markus
> 

Re: [Caml-list] Escaped string in sexplib

[Török Edwin]

On 06/08/2010 09:52 PM, Hugo Ferreira wrote:
> Hello,
> 
> Markus Mottl wrote:
>> On Tue, Jun 8, 2010 at 11:44, Hugo Ferreira <hmf@inescporto.pt> wrote:
>>> I am trying to us the Sexplib library to process data.
>>> However I am having some trouble with strings.
>>> Specifically I need to save and restore strings to a file
>>> so that the escaping done during saving is "undone"
>>> when data is read back in.
>>>
>>> How can I do this? Appreciate any pointers.
>>
>> The I/O and parsing routines in the S-expression library should take
>> care of escaped strings just fine.  
> 
> My apologies but I did not explain myself correctly. The problem here is
> that I am trying to automate the conversion to string of the data
> structure read from the s-expressions. I have the following:
> 
> let string_of_experiment_data data =
>   let data = sexp_of_experiment_data data in
>   Sexplib.Sexp.to_string_hum data
> 
> The problem here is that the "to_string_hum" escapes certain characters,
> but I need to have the strings "not escaped". So my question is: is
> their any way I can tell sexplib to output a string but not escape
> it? Otherwise I will have to hand encode this stuff 8-(
> 
>> Btw., the escaping conventions are
>> absolutely identical to the ones used by OCaml.
>>
> 
> I am aware of this as it is indicated in your documentation and
> the ocaml tutorial site. I also searched for a means to "unescape"
> the result from "to_string_hum", which would also work. However
> I cannot find a String.unescape. This issue has already been pointed
> out. A possible solution would be to use the old.
> 
>  Token.eval_string
> 
> and I cannot seem to use the equivalent of the new camlp4. The
> equivalent:
> 
> open Camlp4.PreCast
> open Syntax
> let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s
> 
> does not compile (Error: Unbound module Camlp4.PreCast).

It compiles for me if I add -I +camlp4:
ocamlc -I +camlp4 -c x.ml
or:
ocamlfind ocamlc -package camlp4.lib -c x.ml

It also links if you add -linkpkg.

Best regards,
--Edwin

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Török Edwin wrote:
> On 06/08/2010 09:52 PM, Hugo Ferreira wrote:
>> Hello,
>>
>> Markus Mottl wrote:
>>> On Tue, Jun 8, 2010 at 11:44, Hugo Ferreira <hmf@inescporto.pt> wrote:
>>>> I am trying to us the Sexplib library to process data.
>>>> However I am having some trouble with strings.
>>>> Specifically I need to save and restore strings to a file
>>>> so that the escaping done during saving is "undone"
>>>> when data is read back in.
>>>>
>>>> How can I do this? Appreciate any pointers.
>>> The I/O and parsing routines in the S-expression library should take
>>> care of escaped strings just fine.  
>> My apologies but I did not explain myself correctly. The problem here is
>> that I am trying to automate the conversion to string of the data
>> structure read from the s-expressions. I have the following:
>>
>> let string_of_experiment_data data =
>>   let data = sexp_of_experiment_data data in
>>   Sexplib.Sexp.to_string_hum data
>>
>> The problem here is that the "to_string_hum" escapes certain characters,
>> but I need to have the strings "not escaped". So my question is: is
>> their any way I can tell sexplib to output a string but not escape
>> it? Otherwise I will have to hand encode this stuff 8-(
>>
>>> Btw., the escaping conventions are
>>> absolutely identical to the ones used by OCaml.
>>>
>> I am aware of this as it is indicated in your documentation and
>> the ocaml tutorial site. I also searched for a means to "unescape"
>> the result from "to_string_hum", which would also work. However
>> I cannot find a String.unescape. This issue has already been pointed
>> out. A possible solution would be to use the old.
>>
>>  Token.eval_string
>>
>> and I cannot seem to use the equivalent of the new camlp4. The
>> equivalent:
>>
>> open Camlp4.PreCast
>> open Syntax
>> let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s
>>
>> does not compile (Error: Unbound module Camlp4.PreCast).
> 
> It compiles for me if I add -I +camlp4:
> ocamlc -I +camlp4 -c x.ml
> or:
> ocamlfind ocamlc -package camlp4.lib -c x.ml
> 
> It also links if you add -linkpkg.
> 

Correct. I had forgotten to add the relevant
tags to the ocamlbuild file. The following works
for me:

<src/ilp/experiment.*>: use_camlp4, use_dynlink

Thanks,
Hugo F.


> Best regards,
> --Edwin
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
> 

Re: [Caml-list] Escaped string in sexplib

[Paolo Donadeo]

How your function eval_string:

let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s

actually modifies a string?


-- 
Paolo

Re: [Caml-list] Escaped string in sexplib

[Hugo Ferreira]

Paolo Donadeo wrote:
> How your function eval_string:
> 
> let eval_string s = Camlp4.Struct.Token.Eval.string ~strict:() s
> 
> actually modifies a string?
> 
> 

It does the opposite of String.escaped [1]


[1] http://caml.inria.fr/pub/docs/old-311/libref/String.html