Marshal and Polymorphism

Marshal and Polymorphism

[Jonathan Bryant]

I know the subjects of the questions I ask on this list must seem all
over the map, but I'm working on a large project with many different
parts.  Anyway, on to my question :).

I'm trying to write a disk bound hashtable.  I'm doing this because I
have to store somewhere between 1,000,000,000 - 2,000,000,000 (no, that
is NOT a typo) words and their corresponding index numbers, as well as
the documents they appeared in (20,000,000,000+) and frequency
information.  My question is this:

If I make the module have

type = ('a, 'b) t

in the module definition, can I use

(Marshal.from_string ... : 'a)

and get whatever 'a was defined as in the definition.  For example, if
it was a (string * int) t, would the function return a string?  Or would
this be redundant (since Marshal.from_string already returns 'a)?

-- 
*=========================*
|Jonathan Bryant          |
|Valdosta State University|
|Information Technology   |
|System Operations        |
|-------------------------|
|jtbryant@valdosta.edu    |
|x6358                    |
*=========================*

Re: [Caml-list] Marshal and Polymorphism

[Stephane Glondu]

Hi,

Jonathan Bryant wrote:
> If I make the module have
> 
> type = ('a, 'b) t
> 
> in the module definition, can I use
> 
> (Marshal.from_string ... : 'a)
> 
> and get whatever 'a was defined as in the definition.

In the way I understood your message, you cannot. However, if you do
something like :

let get (table : ('a, 'b) t) index =
  (Marshal.from_string ... : 'a)

The 'a type variable must be bound /inside/ a surrounding function
(and (Marshal.from_string val : 'a) should be forbidden outside a
function).

> For example, if
> it was a (string * int) t, would the function return a string? 

Now, yes.

> Or would
> this be redundant (since Marshal.from_string already returns 'a)?

No! Actually, the return type of Marshal.from_string is [forall 'a.
'a], so it is different from any other 'a which could be already bound
at this moment. Therefore, you must explicitly tell the return type of
Marshall.from_string, even if it is polymorphic.

I hope this is clear.

Best regards,

-- 
Stephane Glondu

Re: [Caml-list] Marshal and Polymorphism

[Jonathan Bryant]

Yeah, I'm sorry I wasn't real clear.  Let me try again:

This code works:

module Test :
    sig
        type ('a, 'b) t
        val create : 'a -> 'b -> ('a, 'b) t
        val serialize : ('a, 'b) t -> string
        val deserialize : string -> ('a, 'b) t
        val get_word : ('a, 'b) t -> 'a
        val get_index : ('a, 'b) t -> 'b
    end
    =
    struct
        type ('a, 'b) t = { word : 'a; index : 'b };;
        let create x y = { word = x; index = y };;
        let serialize x = Marshal.to_string x [];;
        let deserialize x = (Marshal.from_string x 0 : ('a, 'b) t);;
        let get_word x = x.word;;
        let get_index x = x.index;;
    end
;;

let _ =
    let x = Test.create "Hello" 1 in
    let ser_data = Test.serialize x in
    let deser_data = Test.deserialize ser_data in
    Printf.printf "%s: %d\n" (Test.get_word deser_data) (Test.get_index
deser_data);
;;

My question is:  Will this /always/ work (given that the type of data I
read out of the file is the same)?  Is it really that simple?

On Thu, 2005-08-04 at 13:06 -0700, Stephane Glondu wrote:
> Hi,
> 
> Jonathan Bryant wrote:
> > If I make the module have
> > 
> > type = ('a, 'b) t
> > 
> > in the module definition, can I use
> > 
> > (Marshal.from_string ... : 'a)
> > 
> > and get whatever 'a was defined as in the definition.
> 
> In the way I understood your message, you cannot. However, if you do
> something like :
> 
> let get (table : ('a, 'b) t) index =
>   (Marshal.from_string ... : 'a)
> 
> The 'a type variable must be bound /inside/ a surrounding function
> (and (Marshal.from_string val : 'a) should be forbidden outside a
> function).
> 
> > For example, if
> > it was a (string * int) t, would the function return a string? 
> 
> Now, yes.
> 
> > Or would
> > this be redundant (since Marshal.from_string already returns 'a)?
> 
> No! Actually, the return type of Marshal.from_string is [forall 'a.
> 'a], so it is different from any other 'a which could be already bound
> at this moment. Therefore, you must explicitly tell the return type of
> Marshall.from_string, even if it is polymorphic.
> 
> I hope this is clear.
> 
> Best regards,
> 
-- 
*=========================*
|Jonathan Bryant          |
|Valdosta State University|
|Information Technology   |
|System Operations        |
|-------------------------|
|jtbryant@valdosta.edu    |
|x6358                    |
*=========================*

Re: [Caml-list] Marshal and Polymorphism

[Stephane Glondu]

Jonathan Bryant wrote:
> Yeah, I'm sorry I wasn't real clear.  Let me try again:
> 
> This code works:
> 
> module Test : [...]
> 
> let _ =
>     let x = Test.create "Hello" 1 in
>     let ser_data = Test.serialize x in
>     let deser_data = Test.deserialize ser_data in
>     Printf.printf "%s: %d\n" (Test.get_word deser_data) (Test.get_index
> deser_data);
> ;;
> 
> My question is:  Will this /always/ work (given that the type of data I
> read out of the file is the same)?

It depends on what you mean by "work". Types are not marshaled.

Hence, your deserialize will return ('a, 'b) t (or something like
('_a, '_b) t) in the same way Marshal.from_string returns 'a, which is
meaningless. So you should *always* explicitly give the actual type
(without type variable) when you call deserialize, i.e. your test code
should be:

let _ =
    let x = Test.create "Hello" 1 in
    let ser_data = Test.serialize x in
    let deser_data = (Test.deserialize ser_data :
                        (string, int) Test.t) in
    ... ;;

Otherwise, you could write something like:

let _ =
    let x = Test.create "Hello" 1 in
    let ser_data = Test.serialize x in
    let deser_data = Test.deserialize ser_data in
    (Test.get_index deser_data) ();;

and get a runtime crash (you wouldn't be able to call "deser_data ()"
directly, though).

> Is it really that simple?

As simple as using Marshal.* functions: if you are aware of the traps...


BTW, here, giving the return type in
(Marshal.from_string x 0 : ('a, 'b) t)
is useless, since it is enforced by the signature.


I hope this will help,

-- 
Stephane Glondu

Re: [Caml-list] Marshal and Polymorphism

[skaller]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

On Thu, 2005-08-04 at 16:39 -0400, Jonathan Bryant wrote:

> My question is:  Will this /always/ work (given that the type of data I
> read out of the file is the same)?  Is it really that simple?

No, it won't *always* work in the sense that some data
types cannot be serialised. See also 18.9.1 of the manual.

-- 
John Skaller <skaller at users dot sourceforge dot net>


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]