Camlp4 help

Camlp4 help

[Andre Nathan]

Hello

I'm adding support for property testing in OSpec. Currently you can
write a specification like

  forall (list_of int) (fun l -> (List.rev (List.rev l)) should = l)

and it's also possible to add a constraint as in

  forall (list_of int) ~given:(fun l -> List.length l > 0)
         (fun l -> l should match x::xs)

This automatically generates lists of random sizes containing random
elements, and runs the specified property for each of them. I've been
trying to turn this into a syntax extension that would look like

  forall (list_of int) l . (List.rev (List.rev l)) should = l

or

  forall (list_of int) l . List.length l > 0 => l should match x::xs

The best I could to to make this work was forcing parenthesis around the
expression that comes after the dot, with the following rule:

  "forall"; "("; gen = expr; ")"; var = ipatt; "."; OPT "(";
  e1 = expr; OPT ")"; impl = OPT "=>"; e2 = OPT expr ->

With that I can write the two specifications above as

  forall (list_of int) l . ((List.rev (List.rev l)) should = l)

and

  forall (list_of int) l . (List.length l > 0) => l should match x::xs

which is not that bad, but not exactly what I wanted... 

If I simplify the rule above to

  "forall"; "("; gen = expr; ")"; var = ipatt; ".";
  e1 = expr; impl = OPT "=>"; e2 = OPT expr ->

then everything after the dot is bound to e1, even when there's a "=>". 

Is there some other matcher in camlp4 other than "expr" that I could use
for that? If not, is there another way to parse this correctly without
the extra parenthesis?

Thanks in advance,
Andre

Re: [Caml-list] Camlp4 help

[Andre Nathan]

On Sun, 2009-04-12 at 21:05 -0300, Andre Nathan wrote:
> If I simplify the rule above to
> 
>   "forall"; "("; gen = expr; ")"; var = ipatt; ".";
>   e1 = expr; impl = OPT "=>"; e2 = OPT expr ->
> 
> then everything after the dot is bound to e1, even when there's a "=>". 

For the archives, this happens because "=>" is a valid infix operator
name, and thus is matched by "e1 = expr". Changing it to "->" did the
trick. Thanks to bluestorm at #ocaml for that :)

Andre

Re: [Caml-list] Camlp4 help

[blue storm]

Oops, missing code : http://bluestorm.info/camlp4/pa_holes.ml.html

Re: [Caml-list] Camlp4 help

[blue storm]

On 3/21/09, Andre Nathan <andre@sneakymustard.com> wrote:
> I think I understand, although I thought the "x = expr" rule in the sum
> form definition meant that before "plus" any expression would be
> allowed.

What you wanted is   let a = b in (a plus b). The expression is not
before plus, plus is inside the expression. Your rule means that
inside plus, the left member can be an expression (eg. (let a = b in
a) plus b).

> I want to allow any expression inside a "sum" block, which I think I
> could do by defining it as a new rule in "expr", but I'd like "plus"
> expressions to only be allowed inside a "sum" block, which I'm not sure
> how to do.

I see (but there may be a better choice) two solutions :
- create a new structure expression_inside_sum wich is a complete copy
of "expr" with sums added. This is ugly and redundant.
- use a "marker" trick in two pass :
   - in your grammar you "mark" plus nodes by translating them into a
specific AST node (wich cannot be produced by any usual camlp4
construct), for example "a plus b" -> <:expr< $id:"camlp4.plus"$ a b
>>
   - then you use an Ast.map or a Camlp4Filter to explore the
resulting AST, translating the marked nodes (the ones with
"camlp4.plus" as identifiers) into different things depending on
wether you're inside a sum-block or not (or possibly raising an error
outside a sum-block).

If you want an example, I used a similar trick in my "pa_holes"
extension : "\1"-like identifiers are allowed only inside a (\ ... )
block; at first they're allowed everywhere (the "ident" rule is
changed), then they're transformed into a valid camlp4 construction
inside the (\ ... ) blocks ("expr" rule), and finally the AST is
explored and any remaining \n raise an error (Remove_holes filter).

This solution is also ugly, but has little redudancy.

Re: [Caml-list] Camlp4 help

[Andre Nathan]

Hi

On Sat, 2009-03-21 at 16:26 +0100, blue storm wrote:
> This is not a camlp4-specific problem : the grammar you described
> apparently do not conform to what you have in mind.

I think I understand, although I thought the "x = expr" rule in the sum
form definition meant that before "plus" any expression would be
allowed.

I want to allow any expression inside a "sum" block, which I think I
could do by defining it as a new rule in "expr", but I'd like "plus"
expressions to only be allowed inside a "sum" block, which I'm not sure
how to do.

Thanks,
Andre

> 
> According to your definition, the "sum do ... done" can only contain
> "sum" forms, not an arbitrary expression. The only valid way to parse
> "sum do let a = b in a plus c done" is thus "sum do (let a = b in a)
> plus b done", as "sum do let a = b in (a plus b) done" has an
> expression (instead of a sum) between do ... done (and a sum in expr
> position, wich is also incorrect, but it can also be parsed as a
> correct expression).
> 
> What is the syntax you want to be supported ?
> 
> On 3/21/09, Andre Nathan <andre@sneakymustard.com> wrote:
> > Hello
> >
> > I'm just beginning with camlp4 here, and I'm stuck with what I think is
> > a precedence issue. I have the following syntax extension:
> >
> > open Camlp4.PreCast
> > open Syntax
> >
> > let sum = Gram.Entry.mk "sum"
> >
> > EXTEND Gram
> >   expr: LEVEL "top"
> >     [ [ "sum"; "do"; seq = LIST1 sum; "done" ->
> >         <:expr< do { $list:seq$ } >> ] ]
> >     ;
> >   sum:
> >     [ [ x = expr; "plus"; y = expr ->
> >         <:expr< $x$ + $y$ >> ] ]
> >     ;
> > END
> >
> > This works fine for something like this:
> >
> > sum do
> >   1 plus 2
> > done
> >
> > which becomes (1 + 2).
> >
> > However, it breaks on
> >
> > sum do
> >   let a = 1 in
> >   let b = 2 in
> >   a plus b
> > done
> >
> > because it becomes ((let a = 1 in let b = 2 in a) + b).
> >
> > How can fix that (allowing "b" to be in scope for the second argument of
> > "plus")?
> >
> > Also, sequences of operations don't parse:
> >
> > sum do
> >   1 + 2;
> >   3 + 4
> > done
> >
> > gives "Parse error: [sum] or "done" expected (in [expr])"
> >
> > What am I missing here?
> >
> > Thanks in advance,
> > Andre
> >
> > _______________________________________________
> > Caml-list mailing list. Subscription management:
> > http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> > Archives: http://caml.inria.fr
> > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> > Bug reports: http://caml.inria.fr/bin/caml-bugs
> >
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs

Re: [Caml-list] Camlp4 help

[blue storm]

This is not a camlp4-specific problem : the grammar you described
apparently do not conform to what you have in mind.

According to your definition, the "sum do ... done" can only contain
"sum" forms, not an arbitrary expression. The only valid way to parse
"sum do let a = b in a plus c done" is thus "sum do (let a = b in a)
plus b done", as "sum do let a = b in (a plus b) done" has an
expression (instead of a sum) between do ... done (and a sum in expr
position, wich is also incorrect, but it can also be parsed as a
correct expression).

What is the syntax you want to be supported ?

On 3/21/09, Andre Nathan <andre@sneakymustard.com> wrote:
> Hello
>
> I'm just beginning with camlp4 here, and I'm stuck with what I think is
> a precedence issue. I have the following syntax extension:
>
> open Camlp4.PreCast
> open Syntax
>
> let sum = Gram.Entry.mk "sum"
>
> EXTEND Gram
>   expr: LEVEL "top"
>     [ [ "sum"; "do"; seq = LIST1 sum; "done" ->
>         <:expr< do { $list:seq$ } >> ] ]
>     ;
>   sum:
>     [ [ x = expr; "plus"; y = expr ->
>         <:expr< $x$ + $y$ >> ] ]
>     ;
> END
>
> This works fine for something like this:
>
> sum do
>   1 plus 2
> done
>
> which becomes (1 + 2).
>
> However, it breaks on
>
> sum do
>   let a = 1 in
>   let b = 2 in
>   a plus b
> done
>
> because it becomes ((let a = 1 in let b = 2 in a) + b).
>
> How can fix that (allowing "b" to be in scope for the second argument of
> "plus")?
>
> Also, sequences of operations don't parse:
>
> sum do
>   1 + 2;
>   3 + 4
> done
>
> gives "Parse error: [sum] or "done" expected (in [expr])"
>
> What am I missing here?
>
> Thanks in advance,
> Andre
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

Re: [Caml-list] camlp4 help

[Gerd Stolpmann]

Am Mittwoch, den 20.07.2005, 09:37 +0200 schrieb Hendrik Tews:
>    #ocamlfind ocamlc -package str -c -pp "camlp4o -I . pa_extend.cmo q_MLast.cmo " -I /usr/lib/ocaml/3.08.3/camlp4 te.ml
>    
> I've never used ocamlfind, but why not
> 
> ocamlfind ocamlc -package str -c -pp "camlp4o pa_extend.cmo q_MLast.cmo " \
>         -I +camlp4 te.ml
> 
> (deleted "-I ." and use "-I +camlp4") ?

ocamlfind has even integrated support for camlp4:

ocamlfind ocamlc -package "str, camlp4.extend, camlp4.quotations" \
                 -syntax camlp4o -c te.ml

Gerd
-- 
------------------------------------------------------------
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany 
gerd@gerd-stolpmann.de          http://www.gerd-stolpmann.de
Telefon: 06151/153855                  Telefax: 06151/997714
------------------------------------------------------------

Re: [Caml-list] camlp4 help

[Hendrik Tews]

Pietro Abate <Pietro.Abate@anu.edu.au> writes:

   However, for some reasons, it doesn't work as expected. The function test
   correctly recognizes the string "-----" but then I get a Stream.Error from
   nowhere ( guess from Grammar.Entry.of_parser ) and the parsin fails...
   
Your test function is _not_ an oracle that guards an camlp4
grammar rule. Instead test is kind of a parsing function itself.
Consequently, if parsing succeeds you have to discard some
tokens:

let test rex strm =
  match Stream.peek strm with
    | Some(_,s) when s =~ rex -> 
	prerr_endline "!!!!" ; 
	Stream.junk strm
    | _ -> raise Stream.Failure
;;


   #ocamlfind ocamlc -package str -c -pp "camlp4o -I . pa_extend.cmo q_MLast.cmo " -I /usr/lib/ocaml/3.08.3/camlp4 te.ml
   
I've never used ocamlfind, but why not

ocamlfind ocamlc -package str -c -pp "camlp4o pa_extend.cmo q_MLast.cmo " \
        -I +camlp4 te.ml

(deleted "-I ." and use "-I +camlp4") ?

Bye,

Hendrik

Re: [Caml-list] camlp4 help

[Pietro Abate]

On Mon, Jul 18, 2005 at 12:12:27PM +0200, Hendrik Tews wrote:
> See "More Power for Camlp4 recursive descent parsing" in 
> http://wwwtcs.inf.tu-dresden.de/~tews/ocamlp4/camlp4-undoc.html

Inspired from your page I'm trying to mix regular expression
and streams... below there's a bit of code to parsa entry of the form

TEST ------ END

where you can have an arbitray ( > 1) number of dashes.

However, for some reasons, it doesn't work as expected. The function test
correctly recognizes the string "-----" but then I get a Stream.Error from
nowhere ( guess from Grammar.Entry.of_parser ) and the parsin fails...

what am I doing wrong ?

:)
p

------------
(*pp camlp4o -I . pa_extend.cmo q_MLast.cmo *)

let (=~) s re = Str.string_match (Str.regexp re) s 0;;

let test rex strm =
    match Stream.peek strm with
    | Some(_,s) when s =~ rex -> print_endline "!!!!" ; ()
    | _ -> raise Stream.Failure
;;

let test_sep_dash =
  Grammar.Entry.of_parser Pcaml.gram "test_sep_dash" ( test "-+" )
;;

EXTEND
GLOBAL: Pcaml.str_item ;
    Pcaml.str_item: [[
        "TEST"; test_sep_dash; "END" -> <:str_item< value ii = "!!!!" >>
    ]];
END
-------------

#ocamlfind ocamlc -package str -c -pp "camlp4o -I . pa_extend.cmo q_MLast.cmo " -I /usr/lib/ocaml/3.08.3/camlp4 te.ml

#camlp4o /usr/lib/ocaml/3.08.3/str.cma ./te.cmo pr_o.cmo oo.ml
!!!!
File "oo.ml", line 2, characters 5-13:
Parse error: [test_sep_dash] expected after 'TEST' (in [str_item])
Uncaught exception: Stream.Error("[test_sep_dash] expected after 'TEST' (in [str_item])")

-- 
++ Blog: http://blog.rsise.anu.edu.au/?q=pietro
++ 
++ "All great truths begin as blasphemies." -George Bernard Shaw
++ Please avoid sending me Word or PowerPoint attachments.
   See http://www.fsf.org/philosophy/no-word-attachments.html

Re: [Caml-list] camlp4 help

[Hendrik Tews]

Pietro Abate <Pietro.Abate@anu.edu.au> writes:

   declared. Is there a way to tell campl4 : pass everything in between 
   RULE and END to a function my_parser ?
   
In principle yes: You can convert any function (of the right
type) into a camlp4 grammar entry. Then you can do something like

EXTEND
  ....
  | "RULE"; result = enty_of_your_function -> ...

See "More Power for Camlp4 recursive descent parsing" in 
http://wwwtcs.inf.tu-dresden.de/~tews/ocamlp4/camlp4-undoc.html
(if this leaves anything unclear just ask).

Bye,

Hendrik

Re: [Caml-list] camlp4 help

[Martin Jambon]

On Fri, 15 Jul 2005, Pietro Abate wrote:

> Hi all,
> I'm having problems (once again) with camlp4...
>
> I'd like to parse and produce a bit of ocaml code out of the
> following description :
>
> ---------------------
> CONNECTIVES
>     "_&_",AssocLeft,And ;
>     "_v_",AssocLeft,Or
> END
>
> TABLEAU
>
>     RULE "And"
>     { A & B }
>     ----------
>       A ; B
>     END
>
> END
> ---------------
>
> The first time around I used quotes around the "{A & B}" and parsed the
> expression with a adhoc parser outside camlp4 (and it's all good and
> working).  Now I want to remove the quotes tut-cur and do everything
> using the pre-processor (quotes make my code quote un-readable).  I've
> started writing this code below, but I don't know how to proceed...
>
> The main problem is of course that I've to parse an expression (in rule),
> with a generic parser built on the top of the connectives that I've just
> declared. Is there a way to tell campl4 : pass everything in between
> RULE and END to a function my_parser ?

The normal way to do this would be to use a quotation, which is a lexical
element (like a string constant).
Your example would be written as:

[...]
TABLEAU

    <:rule< "And"
    { A & B }
    ----------
      A ; B
    >>

END


So if you create a quotation expander named "rule", it will be allowed to
appear where an expr or a patt is expected. You can probably find some
simple trick to check that it is not used out of a TABLEAU block if
necessary.
Then TABLEAU will be parsed as a list of patts or a list of exprs (LIST1
expr instead of LIST1 rule in your code) and you might have to disable
some of them
using a secondary check, i.e. expand the quotation to some reserved expr
or patt such as <:expr< Rule_quotation "text to parse" >>
and check that the list you get has only items of this form.
And now you have the freedom to convert the "TABLEAU" list of exprs
(which for you is just a list of strings to parse) into a
str_item containing whatever you like.

This strategy is a bit weird, but it should work.
An extreme solution would be to patch plexer.ml so that something of
the form RULE ... END becomes only one token which would be equivalent to
<:rule< ... >>, but modifying the camlp4 sources is probably not what you
want.


Martin


> I was looking for inspiration in M. Jambon excellent tutorial, but
> without much luck...
>
> thanks,
> p
>
> -----------------------(not tested)
> let rule = Grammar.Entry.create Pcaml.gram "rule";;
> let connective = Grammar.Entry.create Pcaml.gram "connective";;
>
> type assoc = |AssocLeft |AssocRight |AssocNone ;;
>
> let conntable = Hashtbl.create 15;;
>
> EXTEND
>   Pcaml.str_item: [
>     "CONNECTIVES"; clist = LIST1 connective SEP ";"; "END" ->
>       List.iter Hashtbl.add conntable clist;
>       <:str_item< value version = "connectives declared" >>
>     |"TABLEAU"; LIST1 rule; "END" ->
>         <:str_item< value version = "tableau declared" >>
>   ];
>
>   connective: [
>     s = STRING; ","; a = UIDENT; ","; r = UIDENT -> (s,a,r)
>   ];
>
>   (* how can I write a quotation outside the EXTEND syntax ? *)
>   (* how does this quotation looks like ? *)
>   (* Quotation.add my_parser ??? *)
>   rule: [
>     "RULE"; r = my_parser ??? "END" -> do_something r
>   ];
>
> END
> ------------------------
>
> --
> ++ Blog: http://blog.rsise.anu.edu.au/?q=pietro
> ++
> ++ "All great truths begin as blasphemies." -George Bernard Shaw
> ++ Please avoid sending me Word or PowerPoint attachments.
>    See http://www.fsf.org/philosophy/no-word-attachments.html
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

--
Martin Jambon, PhD
http://martin.jambon.free.fr