[Caml-list] A camlp4 task for reading in 'structs'

[Caml-list] A camlp4 task for reading in 'structs'

[Jonathan Roewen]

Hi All,

The idea: a type-safe way to parse an IO.input stream as structured
data (think C structs).

I'm wondering if anyone is interested in trying to create a camlp4
extension for reading structured data from an IO.input stream. I'm not
sure if it's at all possible, but I'd like to return a tuple, and have
it type-checked properly.

Something like:

type field = Byte | Word | DWord | Bytes of int | Char | String of int | All;;
(* corresponding output types: int | int | int32 | int list | char |
string | string; *)

What I want is code that I can pattern match easily (lists are okay,
but I get non-exhaustive match warnings all the time, and it doesn't
guarantee that I'm getting the right values. I also need to use a
second type if I want to have values other than ints.

something like: let (a,b,c) = read_struct i [Byte; Char; Word];;
(using IO module, would return IO.No_more_input if it can't read
enough, which should be okay).

where i = IO.input, such as let i = IO.input_string "hello" in...

I figure something should be possible, as the size of the tuple would
match the size of the list. I'm just not sure if camlp4 is capable of
this; I myself would have zero idea where to begin if I have to write
it... so any help would be greatly appreciated, as I would use this a
heck of a lot in my operating system project.

The operations that would use I think would be:
Byte: IO.read_byte
Word: IO.read_ui16
DWord: IO.read_real_i32
Bytes of int: n * IO.read_byte : int list
Char: IO.read
String of int: IO.really_nread n
All: read until IO.No_more_input is raised, returning a string

Actually, I just had a thought: instead of a list, you could use a
tuple as well .. if that'd make it more possible. Ohh, I forgot about
endianness .. not sure how to handle that...

Kindest Regards,

Jonathan

Re: [Caml-list] A camlp4 task for reading in 'structs'

[Gerd Stolpmann]

Jonathan Roewen said:
> Hi All,
>
> The idea: a type-safe way to parse an IO.input stream as structured
> data (think C structs).

My SunRPC implementation contains that. It can read and write all data
types specified for SunRPC, including all types you mention. Usage is
quite simple:

- Describe your data structure in an IDL file (syntax close to C), e.g.

  struct sample {
    int x;
    string y<10>;     /* String with max length 10 */
  };

- Compile that idl file with ocamlrpcgen:

  ocamlrpcgen -aux sample.x

  That generates sample_aux.{ml,mli}. Among other things, you can find
  here:

  * For every IDL type a corresponding O'Caml type. In this example:

    type sample = { x : int; y : string }

  * Functions converting values of type sample to/from the generic
    Xdr.xdr_value:

    val _to_sample : xdr_value -> sample
    val _of_sample : sample -> xdr_value

  * The dynamic representation of the IDL type:

    val xdrt_sample : xdr_type_term

- You can now easily convert values s of type sample to strings:

  let t = Xdr.validate_xdr_type xdrt_sample in  (* do this only once *)
  let v = _of_sample s in
  Xdr.pack_xdr_value_as_string v t []

- And back:

  let v = Xdr.unpack_xdr_value ~fast:true str t [] in
  _to_sample v

Although there are currently no stream functions, this would be easy to add.

Note that XDR, the marshalling technique used by SunRPC, is an Internet
standard. The representation is portable (e.g. the endianess question is
solved). You can, for example, read such values easily in from a C
program.

Link:
http://www.ocaml-programming.de/programming/rpc.html

Gerd

> I'm wondering if anyone is interested in trying to create a camlp4
> extension for reading structured data from an IO.input stream. I'm not
> sure if it's at all possible, but I'd like to return a tuple, and have
> it type-checked properly.
>
> Something like:
>
> type field = Byte | Word | DWord | Bytes of int | Char | String of int |
> All;;
> (* corresponding output types: int | int | int32 | int list | char |
> string | string; *)
>
> What I want is code that I can pattern match easily (lists are okay,
> but I get non-exhaustive match warnings all the time, and it doesn't
> guarantee that I'm getting the right values. I also need to use a
> second type if I want to have values other than ints.
>
> something like: let (a,b,c) = read_struct i [Byte; Char; Word];;
> (using IO module, would return IO.No_more_input if it can't read
> enough, which should be okay).
>
> where i = IO.input, such as let i = IO.input_string "hello" in...
>
> I figure something should be possible, as the size of the tuple would
> match the size of the list. I'm just not sure if camlp4 is capable of
> this; I myself would have zero idea where to begin if I have to write
> it... so any help would be greatly appreciated, as I would use this a
> heck of a lot in my operating system project.
>
> The operations that would use I think would be:
> Byte: IO.read_byte
> Word: IO.read_ui16
> DWord: IO.read_real_i32
> Bytes of int: n * IO.read_byte : int list
> Char: IO.read
> String of int: IO.really_nread n
> All: read until IO.No_more_input is raised, returning a string
>
> Actually, I just had a thought: instead of a list, you could use a
> tuple as well .. if that'd make it more possible. Ohh, I forgot about
> endianness .. not sure how to handle that...
>
> Kindest Regards,
>
> Jonathan
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>


------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
------------------------------------------------------------