[Caml-list] Marshalling unknown types

[Caml-list] Marshalling unknown types

[Alex Baretta]

I would like Caml breeders and Caml riders to comment on the following 
issue: the need to type-safe unmarshalling of datastructures and the way 
to achieve it.

The Marshal module appropriately provides polymorphic functions as
 > val from_string : string -> 'a (* Offset parameter omitted *)

Such polymorphism is unable to provide static type safety. I would be 
happy with runtime type safety: in other words, I'd be happy with an 
exception being thrown at runtime if a datastructure is unmarshalled to 
the wrong runtime type. The obvious way to do this is the following.

module Type_safe_marshal = struct
   let to_string (ex:exn) = Marshal.to_string ex []
   let from_string s = raise (Marshal.from_string s 0)
end

module Foobar_marshal = struct
   open Type_safe_marshal
   type foobar = Foo of int | Bar of string
   exception Foobar of foobar

   let to_string (x:foobar) = to_string (Foobar x)
   let from_string s = try raise (from_string s) with
     | Foobar x -> x
end

open Foobar_marshal
let foo1 = foo 1
let what = from_string (to_string foo1)

Obviously this does not work. For some reason, as someone already 
pointed out, pattern matching does not work on unmarshalled exceptions. 
When the above code is sent to the toplevel interpreter I get the 
following output.

module Type_safe_marshal :
   sig val to_string : exn -> string val from_string : string -> 'a end

module Foobar_marshal :
   sig
     type foobar = Foo of int | Bar of string
     exception Foobar of foobar
     val to_string : foobar -> string
     val from_string : string -> foobar
   end
val foo1 : Foobar_marshal.foobar = Foo 1
Exception: Foobar_marshal.Foobar _.

I'm sure that there are excellent reasons why this should occur. So, I 
tried a less obvious means of achieving runtime type safety for 
unmarshalled objects.

module Type_safe_marshal_2 = struct
   let to_string (f:unit -> 'a) = Marshal.to_string f [Marshal.Closures]
   (* f is a function which builds and throws an appropriate exception *)
   let from_string s () = Marshal.from_string s 0 ()
end

module Foobar_marshal_2 = struct
   open Type_safe_marshal_2
   type foobar = Foo of int | Bar of string
   exception Foobar of foobar

   let raise_foobar x () = raise (Foobar x)
   let to_string x = to_string (raise_foobar x)
   let from_string s = try from_string s () with
     | Foobar x -> x
end

open Foobar_marshal_2
let foo1 = Foo 1
let what = from_string (to_string foo1)
let _ = Printf.eprintf "Hey, I managed to build \"what\" alright!\n"

Here I try to build and raise the exceptions at unmarshalling time by 
actually marshalling closures of functions know how to throw an 
exception containing the datastructure I wish to marshal. However, this 
code cannot be run in the toplevel. This is what it says:

Exception: Invalid_argument "output_value: abstract value".

Apparently, closures built by the toplevel compiler cannot be 
marshalled. So let's try with ocamlc. I placed the above code in 
marshal_foobar.ml and compiled it with ocamlc.

$ ocamlc foobar_marshal.ml -o foobar_marshal
$ ./foobar_marshal
Fatal error: exception Foobar_marshal.Foobar_marshal_2.Foobar(1)

I can find no reasonable explanation for this behavior. I realize that 
exceptions are nasty beasts and that the marshalling functions were not 
meant for them, but in this case, the exception is only built and raised 
*after* unmarshalling occurs, so I don't see why pattern matching should 
fail.

***

All this said, O great Camlers, what is your verdict? If my approach is 
sound, how can I fix my code to make it work? If my approach is 
misguided, how should type-safe unmarshalling be achieved?


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Marshalling unknown types

[Richard Jones]

On Mon, Jan 26, 2004 at 09:22:13AM +0100, Alex Baretta wrote:
> I can find no reasonable explanation for this behavior. I realize that 
> exceptions are nasty beasts and that the marshalling functions were not 
> meant for them, but in this case, the exception is only built and raised 
> *after* unmarshalling occurs, so I don't see why pattern matching should 
> fail.

IIRC you cannot marshal/unmarshal functions safely.  It only works in
the limited case where you are unmarshaling in the exact same compiled
program.

What's needed is for someone to rewrite the marshal code to work
safely.  However there's not enough type information available at
runtime for this to work.  eg. [1; 2; 3] looks the same as (1, 2, 3)
at runtime.

Rich.

-- 
Richard Jones. http://www.annexia.org/ http://freshmeat.net/users/rwmj
Merjis Ltd. http://www.merjis.com/ - improving website return on investment
http://www.YouUnlimited.co.uk/ - management courses

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Marshalling unknown types

[James Leifer]

We (Gilles Peskine, Peter Sewell, Keith Wansbrough, and I) have been
working on type safety and related questions concerned with
marshalling:

  http://pauillac.inria.fr/~leifer/articles/leifer-globas-icfp2003.ps.gz
  http://pauillac.inria.fr/~leifer/articles/leifer-marabreb.ps.gz

However, we're far from a patch to the actual Ocaml compiler right
now.  We've rather concentrated on building a prototype ML interpreter
which we can use for exploring the design space (type safety,
abstraction safety, versioning, dynamic rebinding to local resources,
etc.).

If you don't need abstraction safety, then Jun Furuse's work on safe
marshalling might interest you.  Michel Mauny might be reimplementing
this for 3.07?

Richard Jones said:

> What's needed is for someone to rewrite the marshal code to work
> safely.  However there's not enough type information available at
> runtime for this to work.  eg. [1; 2; 3] looks the same as (1, 2, 3)
> at runtime.

Well... Peter, Xavier, and I were scratching our heads at POPL two
weeks ago and said: For a really polymorphic marshal then you need to
pass some type information around at run-time or introduce user type
representations.

Of course, if you know *at compile time* that you're marshalling or
unmarshalling to an int list or an int triple, there's no problem
distinguishing between those cases (and including the right data for
doing the dynamic type check).

-James

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners