Where are the AST specs?

Where are the AST specs?

[Loup Vaillant]

Hello everyone,

I am currently working on giving Ocaml a Lisp syntax. On of my goals
is to give macros to Ocaml. (I find Camlp4 quite awkward to use as a
macro system).

So I wonder if someone can tell me where I can find the specs of the
Abstract Syntax Tree, as the compiler can receive it as an input. I
can't find it in the sources of Ocaml.

Regards,
Loup Vaillant

Re: [Caml-list] Where are the AST specs?

[Tom]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

There is the AST defined in /parsing/parsetree.mli, which you have
undoubtedly discovered, and if you look at it carefully and search for the
occurences of different constructors in /parsing/parser.mly, you should be
able to figure where and how they are used.

Furthermore, you should try compiling sample programs with "ocamlc
-dparsetree" which will print the AST of the program.

Unfortunately, I am busy in these months, but you can send me some questions
and I can try to answer them (may take long time), as I have spent quite
some time inspecting the OCaml sources.
- Tom

[-- Attachment #2: Type: text/html, Size: 627 bytes --]

Re: [Caml-list] Where are the AST specs?

[Olivier Andrieu]

On 3/18/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> Hello everyone,
>
> I am currently working on giving Ocaml a Lisp syntax.

In case you haven't noticed it, there used to be a lispish syntax for
ocaml written with camlp4.
It's in the camlp4 tree :
http://camlcvs.inria.fr/cgi-bin/cvsweb/ocaml/camlp4/unmaintained/scheme/pa_scheme.sc?rev=1.3

-- 
  Olivier

Re: [Caml-list] Where are the AST specs?

[Loup Vaillant]

2007/3/19, Olivier Andrieu <oandrieu@nerim.net>:
> On 3/18/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> > Hello everyone,
> >
> > I am currently working on giving Ocaml a Lisp syntax.
>
> In case you haven't noticed it, there used to be a lispish syntax for
> ocaml written with camlp4.
> It's in the camlp4 tree :
> http://camlcvs.inria.fr/cgi-bin/cvsweb/ocaml/camlp4/unmaintained/scheme

Actually, I haven't noticed it. At first glance this syntax is close
to my standards. However, Do you know if it has macros? Clean macros
remain my main goal. My holy grail is the power (and ease of use) of
Common Lisp macros.

Loup

Re: [Caml-list] Where are the AST specs?

[Nicolas Pouillard]

On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> 2007/3/19, Olivier Andrieu <oandrieu@nerim.net>:
> > On 3/18/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> > > Hello everyone,
> > >
> > > I am currently working on giving Ocaml a Lisp syntax.
> >
> > In case you haven't noticed it, there used to be a lispish syntax for
> > ocaml written with camlp4.
> > It's in the camlp4 tree :
> > http://camlcvs.inria.fr/cgi-bin/cvsweb/ocaml/camlp4/unmaintained/scheme
>
> Actually, I haven't noticed it. At first glance this syntax is close
> to my standards. However, Do you know if it has macros? Clean macros
> remain my main goal. My holy grail is the power (and ease of use) of
> Common Lisp macros.

I think, it don't has macros, but Daniel de Rauglaudre would give a
more accurate answer.

BTW camlp4 provides a simple macro system through a dynamic syntax
extension. It features DEFINE, IFDEF, INDEF, and INCLUDE.

However camlp4 is itself a powerful way to manage the OCaml syntax as
AST through it's quotation system.

Have you a simple specification of what you want for OCaml (or some examples).

-- 
Nicolas Pouillard

Re: [Caml-list] Where are the AST specs?

[Loup Vaillant]

> BTW camlp4 provides a simple macro system through a dynamic syntax
> extension. It features DEFINE, IFDEF, INDEF, and INCLUDE.

Actually, I didn't meant C macros.

> However camlp4 is itself a powerful way to manage the OCaml syntax as
> AST through it's quotation system.

That is what I meant. However, I want it as simple as possible.

> Have you a simple specification of what you want for OCaml (or some examples).

For instance, I woud have wanted
(defun f x y z (+ x (* y z)))

expanded into:
(value f (fun (x y z) (+ x (* y z)))

Through a simple macro.
another example:
(+ x y z)

(plus (plus x y) z)

I havent specified the macro system yet. I think I shoud wait untill i
have written a reader. (And therefore specified an AST)

Re: [Caml-list] Where are the AST specs?

[Nicolas Pouillard]

On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> > BTW camlp4 provides a simple macro system through a dynamic syntax
> > extension. It features DEFINE, IFDEF, INDEF, and INCLUDE.
>
> Actually, I didn't meant C macros.

Yes, but these are somewhat better than C ones.

> > However camlp4 is itself a powerful way to manage the OCaml syntax as
> > AST through it's quotation system.
>
> That is what I meant. However, I want it as simple as possible.
>
> > Have you a simple specification of what you want for OCaml (or some examples).
>
> For instance, I woud have wanted
> (defun f x y z (+ x (* y z)))

DEFINE F(x, y, z) = x + y * z;;

F(F(1, 2, 3), 4, 5);;

[...]

> another example:
> (+ x y z)
>
> (plus (plus x y) z)

Yes the macro extension is to weak for that.

Here is a camlp4 snippet todo it:

$ cat macros.ml
open Camlp4.PreCast;;
AstFilters.register_str_item_filter begin
  Ast.map_expr begin function
  | <:expr@loc< \!+ ($tup:e$) >> ->
      begin match Ast.list_of_expr e [] with
      | [] | [_] -> assert false
      | x :: xs -> List.fold_left (fun acc x -> <:expr@loc< $acc$ + $x$ >>) x xs
      end
  | <:expr< \!+ $e$ >> -> e
  | e -> e
  end
end#str_item;;

Can be used like that:

$ cat test_macros.ml
!+ (1, 2, 3, 4);;
$ camlp4o ./macros.cmo test_macros.ml
let _ = ((1 + 2) + 3) + 4

> I havent specified the macro system yet. I think I shoud wait untill i
> have written a reader. (And therefore specified an AST)

IMHO starting from the macro extension of camlp4 and write a new one
more powerful is a good way to go.

-- 
Nicolas Pouillard

Re: [Caml-list] Where are the AST specs?

[Nicolas Pouillard]

On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> 2007/3/19, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> > On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> > > another example:
> > > (+ x y z)
> > >
> > > (plus (plus x y) z)
> >
> > Yes the macro extension is to weak for that.
> > Here is a camlp4 snippet todo it:
> > [...]
>
> Looks promising. Now, I have more:
> (* x y z)
> (:: x y z [])
> etc. You see the pattern? I need a "fold" macro that writes those macros.

open Camlp4.PreCast;;
AstFilters.register_str_item_filter begin
  Ast.map_expr begin function
  | <:expr@loc< foldl($lid:op$, $e$) >> ->
      let rec foldl =
        function
        | [] -> assert false
        | [x] -> x
        | x :: xs -> <:expr@loc< $lid:op$ $foldl xs$ $x$ >>
      in foldl (List.rev (Ast.list_of_expr e []))
  | <:expr@loc< foldr($lid:op$, $e$) >> ->
      let rec foldr =
        function
        | [] -> assert false
        | [x] -> x
        | x :: xs -> <:expr@loc< $lid:op$ $x$ $foldr xs$ >>
      in foldr (Ast.list_of_expr e [])
  | e -> e
  end
end#str_item;;

$ cat test_macros.ml
let cons x xs = x :: xs;;

foldl(( + ), 1, 2, 3, 4);;
foldr(cons, 1, 2, 3, []);;

$ camlp4o ./macros.cmo test_macros.ml
let cons x xs = x :: xs
let _ = ((1 + 2) + 3) + 4
let _ = cons 1 (cons 2 (cons 3 []))

> Besides, I think my biggest challenge will probably be the automatic
> handling of "loc", and a beautiful anti quotation system.

You're right loc is something hard.
What's wrong with the current anti quotation system?

-- 
Nicolas Pouillard

Re: [Caml-list] Where are the AST specs?

[Loup Vaillant]

2007/3/19, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> [...camlp4code...]
> $ cat test_macros.ml
> let cons x xs = x :: xs;;
>
> foldl(( + ), 1, 2, 3, 4);;
> foldr(cons, 1, 2, 3, []);;
>
> $ camlp4o ./macros.cmo test_macros.ml
> let cons x xs = x :: xs
> let _ = ((1 + 2) + 3) + 4
> let _ = cons 1 (cons 2 (cons 3 []))

Impressive.
If it is possible, It would be best would be to be able to write something like:
(def_foldl + plus)

and then just
(+ x y z)
(+ x y z w)
...
The same for cons:
(def_foldr :: cons)

(:: x y z L)
(:: x y z t [])

Such macros are possible, and I think not very hard in Common Lisp. I
will try tonight. As for Camlp4, I don't know. (I don't know camlp4
enough --yet). The point is to define macros that write macro
definitions.

> What's wrong with the current anti quotation system?
Err, actually, nothing... I just found the '$' ugly. But it is one
character, and I may as well use it "as is".

Loup Vaillant

Re: [Caml-list] Where are the AST specs?

[Nicolas Pouillard]

On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> 2007/3/19, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> > [...camlp4code...]
> > $ cat test_macros.ml
> > let cons x xs = x :: xs;;
> >
> > foldl(( + ), 1, 2, 3, 4);;
> > foldr(cons, 1, 2, 3, []);;
> >
> > $ camlp4o ./macros.cmo test_macros.ml
> > let cons x xs = x :: xs
> > let _ = ((1 + 2) + 3) + 4
> > let _ = cons 1 (cons 2 (cons 3 []))
>
> Impressive.
> If it is possible, It would be best would be to be able to write something like:
> (def_foldl + plus)
>
> and then just
> (+ x y z)
> (+ x y z w)
> ...
> The same for cons:
> (def_foldr :: cons)
>
> (:: x y z L)
> (:: x y z t [])

$ cat macros.ml
open Camlp4.PreCast;;
let foldr_funs = ref [];;
let foldl_funs = ref [];;
AstFilters.register_str_item_filter begin
  Ast.map_expr begin function
  | <:expr@loc< def_foldr $lid:name$ $e$ >> ->
      foldr_funs := (name, e) :: !foldr_funs; <:expr@loc<()>>
  | <:expr@loc< def_foldl $lid:name$ $e$ >> ->
      foldl_funs := (name, e) :: !foldl_funs; <:expr@loc<()>>
  | e -> e
  end
end#str_item;;
AstFilters.register_str_item_filter begin
  Ast.map_expr begin function
  | <:expr@loc< $lid:name$($tup:e$) >> when List.mem_assoc name !foldl_funs ->
      let op = List.assoc name !foldl_funs in
      let rec foldl =
        function
        | [] -> assert false
        | [x] -> x
        | x :: xs -> <:expr@loc< $op$ $foldl xs$ $x$ >>
      in foldl (List.rev (Ast.list_of_expr e []))
  | <:expr@loc< $lid:name$($tup:e$) >> when List.mem_assoc name !foldr_funs ->
      let op = List.assoc name !foldr_funs in
      let rec foldr =
        function
        | [] -> assert false
        | [x] -> x
        | x :: xs -> <:expr@loc< $op$ $x$ $foldr xs$ >>
      in foldr (Ast.list_of_expr e [])
  | e -> e
  end
end#str_item;;

$ cat test_macros.ml
let cons x xs = x :: xs;;

def_foldl ( !+ ) ( + );;
def_foldr ( !:: ) cons;;

!+ (1, 2, 3, 4);;
!:: (1, 2, 3, []);;

$ camlp4of ./macros.cmo test_macros.ml
let cons x xs = x :: xs
let _ = ()
let _ = ()
let _ = ((1 + 2) + 3) + 4
let _ = cons 1 (cons 2 (cons 3 []))

> > What's wrong with the current anti quotation system?
> Err, actually, nothing... I just found the '$' ugly. But it is one
> character, and I may as well use it "as is".

I don't like the sacrifice of $ for that purpose but that's just a
matter of taste.

-- 
Nicolas Pouillard

Re: [Caml-list] Where are the AST specs?

[Loup Vaillant]

Here is the promised snipet of code. this is emacs lisp. One can
evaluate these from the *scratch* buffer. (C-xC-e at the end of the
expression). Of course, I assume here that '+' accept only two
arguments, or else these macros are useless. (Actually, they are.)

; This macro is right associative
(defmacro plus (first &rest tail)
  (if (null tail)
    first
    `(+ ,first
	(plus ,@tail))))

; This one is left associative
(defmacro sulp (first &rest tail)
  (if (null tail)
    first
    `(sulp (+ ,first ,(car tail))
	   ,@(cdr tail))))

; some examples
(plus 1 2 3 4 5)
(sulp 1 2 3 4 5)

I have not defined the macro definfing macros (I am no expert), yet I
belive they do not take more than six lines each. Note that I will
need read macros as well as regulars ones.

That is the kind of power I want from Ocaml. If I succeed, I will
consider solved (at least for my personnal usage) many syntax problems
I read about in this list.


2007/3/19, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> On 3/19/07, Loup Vaillant <loup.vaillant@gmail.com> wrote:
> > 2007/3/19, Nicolas Pouillard <nicolas.pouillard@gmail.com>:
> > > [...camlp4code...]
> > > $ cat test_macros.ml
> > > let cons x xs = x :: xs;;
> > >
> > > foldl(( + ), 1, 2, 3, 4);;
> > > foldr(cons, 1, 2, 3, []);;
> > >
> > > $ camlp4o ./macros.cmo test_macros.ml
> > > let cons x xs = x :: xs
> > > let _ = ((1 + 2) + 3) + 4
> > > let _ = cons 1 (cons 2 (cons 3 []))
> >
> > Impressive.
> > If it is possible, It would be best would be to be able to write something like:
> > (def_foldl + plus)
> >
> > and then just
> > (+ x y z)
> > (+ x y z w)
> > ...
> > The same for cons:
> > (def_foldr :: cons)
> >
> > (:: x y z L)
> > (:: x y z t [])
>
> $ cat macros.ml
> open Camlp4.PreCast;;
> let foldr_funs = ref [];;
> let foldl_funs = ref [];;
> AstFilters.register_str_item_filter begin
>   Ast.map_expr begin function
>   | <:expr@loc< def_foldr $lid:name$ $e$ >> ->
>       foldr_funs := (name, e) :: !foldr_funs; <:expr@loc<()>>
>   | <:expr@loc< def_foldl $lid:name$ $e$ >> ->
>       foldl_funs := (name, e) :: !foldl_funs; <:expr@loc<()>>
>   | e -> e
>   end
> end#str_item;;
> AstFilters.register_str_item_filter begin
>   Ast.map_expr begin function
>   | <:expr@loc< $lid:name$($tup:e$) >> when List.mem_assoc name !foldl_funs ->
>       let op = List.assoc name !foldl_funs in
>       let rec foldl =
>         function
>         | [] -> assert false
>         | [x] -> x
>         | x :: xs -> <:expr@loc< $op$ $foldl xs$ $x$ >>
>       in foldl (List.rev (Ast.list_of_expr e []))
>   | <:expr@loc< $lid:name$($tup:e$) >> when List.mem_assoc name !foldr_funs ->
>       let op = List.assoc name !foldr_funs in
>       let rec foldr =
>         function
>         | [] -> assert false
>         | [x] -> x
>         | x :: xs -> <:expr@loc< $op$ $x$ $foldr xs$ >>
>       in foldr (Ast.list_of_expr e [])
>   | e -> e
>   end
> end#str_item;;
>
> $ cat test_macros.ml
> let cons x xs = x :: xs;;
>
> def_foldl ( !+ ) ( + );;
> def_foldr ( !:: ) cons;;
>
> !+ (1, 2, 3, 4);;
> !:: (1, 2, 3, []);;
>
> $ camlp4of ./macros.cmo test_macros.ml
> let cons x xs = x :: xs
> let _ = ()
> let _ = ()
> let _ = ((1 + 2) + 3) + 4
> let _ = cons 1 (cons 2 (cons 3 []))
>
> > > What's wrong with the current anti quotation system?
> > Err, actually, nothing... I just found the '$' ugly. But it is one
> > character, and I may as well use it "as is".
>
> I don't like the sacrifice of $ for that purpose but that's just a
> matter of taste.
>
> --
> Nicolas Pouillard
>