Obj.magic

Obj.magic

[David Chemouil]

Hi again,



I'd like to get information about Obj.magic. I know it is some kind of
identity function `a -> `b, but I'd like to know how it can be used, why
it can be interesting and why, also, it is sometimes dangerous
(considering some older posts on this mailing-list). For example, I
found myself trying some different things (with polymorphic variants and
coercions, too):

# type arg = [`A | `B];;
type arg = [ `A | `B]
# let id : arg -> arg = function
  | `A -> `A
  | `B -> `B;;
val id : arg -> arg = <fun>
# let x = `A;;
val x : [> `A] = `A
# id x;;
- : arg = `A
# x;;
- : [> `A] = `A
# let y = ref `A;;
val y : _[> `A] ref = {contents=`A}
# id !y;;
- : arg = `A
# y;;
- : arg ref = {contents=`A}
# let z = Obj.magic `A;;
val z : '_a = <poly>
# id z;;
- : arg = `A
# z;;
- : arg = `A
# let t = (x :> arg);;
val t : arg = `A

-- 
David Chemouil [mailto:chemouil@enseeiht.fr] [mobile: 06 84 16 26 65]

Laboratoire d'informatique et de mathématiques appliquées (IRIT-INPT)

"Je vous ai fait trop faibles pour sortir du gouffre, parce que 
 je vous ai fait assez forts pour n'y point tomber" -- Rousseau

polymorphic variants (2) + magic

[David Chemouil]

Considering my two previous posts ("polymorphic variants" and
"Obj.magic"), I'd like to show how I had to merge them to have my
application work.

Indeed, I don't know yet completely the type 'argument', but I already
know some of its constructors. Then I use polymorphic constructors when
I need them and, when I'm able to do it, I will generate the 'argument'
type. 

The problem is that there are already functions which manipulate the
'argument' type. So, if I try to use them on an already known
polymorphic constructor, it doesn't work, because the constructor is not
known to belong to 'argument', as 'argument' isn't defined. 

My problem is then to be able to apply a function, of which the domain
is a yet undefined type, to a polymorphic constructor. My idea was to
have my already known polymorphic constructors considered as weak type
variables. Here is a simple example of the situation (I consider `A as
an already known constructor):

# type argument;;
type argument
# let id : argument -> argument = function x -> x;;
val id : argument -> argument = <fun>
# let x = `A;;
val x : [> `A] = `A
# let y = ! (ref `A);;
val y : _[> `A] = `A
# let z = Obj.magic `A;;
val z : '_a = <poly>
# id x;;
Characters 3-4:
This expression has type [> `A] but is here used with type argument
# id y;;
Characters 3-4:
This expression has type [> `A] but is here used with type argument
# id z;;
- : argument = <abstr>
# z;;
- : argument = <abstr>

As the example shows, the only solution for me is to use Obj.magic. Then
I'm sure everything will be well-typed. But is it completely correct to
do like this? 


-- 
David Chemouil [mailto:chemouil@enseeiht.fr] [mobile: 06 84 16 26 65]

Laboratoire d'informatique et de mathématiques appliquées (IRIT-INPT)

"Je vous ai fait trop faibles pour sortir du gouffre, parce que 
 je vous ai fait assez forts pour n'y point tomber" -- Rousseau

Re: polymorphic variants (2) + magic

[Ken Wakita]

The Obj module allows nearly direct access to the memory image of the
Caml runtime system.  My small program explains usefulness and danger
of this module.  The "show_val" function takes an arbitrary Caml value
and prints out their internal representation.

See: http://www.is.titech.ac.jp/~wakita/caml/magic.tgz

Ken

In message (<3940B178.280A76CC@enseeiht.fr>)
from David Chemouil <David.Chemouil@enseeiht.fr>,
talking about "Obj.magic",
on Fri, 09 Jun 2000 10:57:28 +0200

> I'd like to get information about Obj.magic. I know it is some kind of
> identity function `a -> `b, but I'd like to know how it can be used, why
> it can be interesting and why, also, it is sometimes dangerous
> (considering some older posts on this mailing-list). For example, I
> found myself trying some different things (with polymorphic variants and
> coercions, too):

Re: Obj.magic

[Daniel de Rauglaudre]

Hello,

On Fri, Jun 09, 2000 at 10:57:28AM +0200, David Chemouil wrote:

> I'd like to get information about Obj.magic. I know it is some kind of
> identity function `a -> `b, but I'd like to know how it can be used, why
> it can be interesting and why, also, it is sometimes dangerous
> (considering some older posts on this mailing-list).

I use Obj.magic:

1/ In Camlp4 in extensible grammars to type parsing trees
2/ In GeneWeb in my data base system to optimize disk access

  ---

1/ Camlp4 provides a system of extensible grammars.

When I write a code like this (Camlp4 syntax):

   EXTEND
     expr:
       [ [ x = expr; "+"; y = expr -> x + y
         | z = int -> z ] ];
   END

It is converted into something like this:

   extend expr
     [ ([Nterm expr; Term "+"; Nterm expr], fun x y -> x + y);
       ([Nterm int], fun z -> z) ]

But it is bad typed for several reasons: one of them (that you can
see) is that the functions in the right part of these two couples have
different types. Another one is that the Camlp4 constructor "Nterm"
accepts any kind of 'a entry and it is perfectly possible to have a
"int entry" and a "string entry" in the same list.

To resolve that, I use Obj.magic and type constraints to type this
expression. The code is actually something like this:

   extend (Obj.magic expr : 'a entry)
     [ ([Nterm (Obj.magic (expr : 'a1 entry)); Term "+";
         Nterm (Obj.magic (expr : 'a2 entry))],
              (Obj.magic (fun (x : 'a1) (y : 'a2) -> (x + y : 'a))));
       ([Nterm (Obj.magic (int : 'a3 entry))],
              (Obj.magic (fun (z : 'a3) -> (z : 'a)))) ]

In this example, the normal type inferrence of Ocaml gives the type
"int" to all of these above type variables 'a, 'a1, 'a2, and 'a3.
However, in the general case, any types could be inferred.

I absolutely need to give to the function "extend" the parameters with
this form because I want to be able to extend the entry again: internally,
I record a parsing tree (for left factorization) and I need the details of
this tree to apply associativity rules.

They are trees with these parsing patterns at each node and functions
at each leaf. All patterns can have different types. A function in a
given leaf has n parameters where n is the length of the path from the
root to the leaf. The type of its parameters is respectively the same
than the type of all patterns of the path (in the same order). The
result type of all functions is the same.

It is not dangerous if this type extension is correct. (Not proved but
I have used it since several years...)

  ---

2/ In GeneWeb (genealogy software).

The normal "input_value" of Ocaml is written in C. I use it to very quickly
input large array of records (input_value is very fast). But I sometimes need
to read just one record in such arrays. For that, I wrote an input_value
function for an array item (the offset have been previously computed). This
function needs Obj.magic.

It is dangerous if the data on disk is not of the good type. Actually it
is not more dangerous than input_value.

-- 
Daniel de Rauglaudre
daniel.de_rauglaudre@inria.fr
http://cristal.inria.fr/~ddr/

Re: polymorphic variants (2) + magic

[Jacques Garrigue]

From: David Chemouil <David.Chemouil@enseeiht.fr>
Subject: polymorphic variants (2) + magic
Date: Fri, 09 Jun 2000 11:15:58 +0200

> Considering my two previous posts ("polymorphic variants" and
> "Obj.magic"), I'd like to show how I had to merge them to have my
> application work.
> 
> Indeed, I don't know yet completely the type 'argument', but I already
> know some of its constructors. Then I use polymorphic constructors when
> I need them and, when I'm able to do it, I will generate the 'argument'
> type. 

You don't necessarily have to use polymorphic variants for that.
In fact the goal of polymorphic variant is somewhat othogonal to your
problem.

With standard sums, you can write

type 'a message =
    Server_message1
  | Server_message2
  | Server_message3
  | Application_message of 'a

Then you can write all your server functions handling only server
messages, and returning application messages to the application.

Moreover, if your server is going to run as an indepedent program,
communicating with applications using input_value/output_value,
then you probably want to make application messages abstract.

types app_msg
type server_message = app_msg message

This way you are sure your server will not fiddle with application
messages. It can receive or send them, but not read their contents.

Polymorphic variants will only be useful if you have a more complex
hierarchy of messages, and want for instance to share some messages
(but not all) between different applications; but it will do nothing
for solving your first problem.

> The problem is that there are already functions which manipulate the
> 'argument' type. So, if I try to use them on an already known
> polymorphic constructor, it doesn't work, because the constructor is not
> known to belong to 'argument', as 'argument' isn't defined. 

With the structure I suggest above, your functions will be polymorphic
in the type of application messages, so this problem does not occur.

> As the example shows, the only solution for me is to use Obj.magic. Then
> I'm sure everything will be well-typed. But is it completely correct to
> do like this? 

Certainly not correct. You should _never_ use Obj.magic.
When you use it things are not well-typed, they are just untyped,
and you can say hello to segmentation faults.
This is only intended to do on the Caml side things that otherwise you
would have to do anyway on the C-side. So, basically the only case in
which you can use Obj.magic is when you have _yourself_ implemented a C
interface, and want to do some casts on C-values. Never use it on ML
values.

        Jacques
---------------------------------------------------------------------------
Jacques Garrigue      Kyoto University     garrigue at kurims.kyoto-u.ac.jp
		<A HREF=http://wwwfun.kurims.kyoto-u.ac.jp/~garrigue/>JG</A>

Re: Obj.magic

[Jean-Christophe Filliatre]

> The Obj module allows nearly direct access to the memory image of the
> Caml runtime system.  My small program explains usefulness and danger
> of this module.  The "show_val" function takes an arbitrary Caml value
> and prints out their internal representation.

I should  mention that  I also  wrote such a  function to  display the
internal representation of ocaml values some time ago, which of course
makes a great use of Obj.magic. It is available here:

      http://www.lri.fr/~filliatr/ftp/ocaml/display/

For a direct access to the documented code with an example, see:

      http://www.lri.fr/~filliatr/ftp/ocaml/display/display.ps.gz

-- 
Jean-Christophe Filliatre    
  Computer Science Laboratory   Phone (650) 859-5173
  SRI International             FAX   (650) 859-2844
  333 Ravenswood Ave.           email  filliatr@csl.sri.com
  Menlo Park, CA 94025, USA     web    http://www.csl.sri.com/~filliatr