[Caml-list] Re: Syntax extensions without Camlp4

[Hongbo Zhang <bobzhang1988@gmail.com>]

On 5/28/12 5:59 AM, Gabriel Scherer wrote:
Hi,
    It would be cool to combine camlp4ast and typedtree to generate more 
meaningful code. one typical examples is tracer. did any one give a try?
> As this seems to be the kind of "christmas shopping list" discussion
> where everyone throws something in, here is my opinion on camlp4 evolution:
> - we should forget about the general problem of extending syntax rules;
> it's too complex, doesn't compose well, and is only of arguable benefit
> - we should focus on blessing specific *fixed* syntactical extensibility
> points and making them enter the standard syntax for the language; there
> are two elephants in the rooms: quotations and type-conv-like annotations.
>
> Quotations are useful, used in practice (not always under the
> camlp4-blessed form), simple to define, understand and use, and compose
> extremely well. One problem with current camlp4 quotations is that they
> are a bit lexically heavy (<:name< ... >>) and inflexible (you basically
> can't use >> inside the quotation); users have tried to workaround this
> by re-coding quotations, eg. for smart string literals u"blah" or regexp
> syntactic sugar s/foo/bar/. I think we could consider having
> lexing-level extensibility (have a tool to modify lexing rule instead of
> grammar rules like Camlp4; hopefully lexing rules are simpler and
> compose better).
>
> Type-conv like annotations are a bit less well-defined, and would need
> some design work: what exactly is their intended scope/expressivity? Are
> they a general annotation mechanism, a rigid way to just add new phrases
> after each annotation phrase, or something in between?
>
> Other point of extensibility could be considered (eg. Jeremy Yallop's
> work on pattern-matching extensibility), but only added if they are
> simple enough, useful enough, and compose well.
>
> I would still welcome having central, well-supported libraries to
> manipulate Ocaml syntactic AST and, why not, typed AST (typedtree).
> Among its many features, Camlp4 currently provides the former (in a
> relatively convenient way thanks to its AST quotations), and this is
> quite useful.
>
> On Mon, May 28, 2012 at 11:35 AM, Daniel Bünzli
> <daniel.buenzli@erratique.ch <mailto:daniel.buenzli@erratique.ch>> wrote:
>
>
>
>     Le lundi, 28 mai 2012 à 00:43, Wojciech Meyer a écrit :
>
>      > Runtime meta-programming is a generalisation of static meta
>      > programming. MetaOCaml has a nice set of abstraction to generate
>      > typechecking code - yes - either at runtime or during compile time.
>
>
>     You meant 'typechecked' (?). It's obviously a generalization but I
>     didn't know there was support to use it at compile time.
>
>      > The
>      > problem is that it's purely for partial evaluation and not
>     extending the
>      > syntax.
>
>
>     Then it's perfect ! I think it's wrong to try to extend the language
>     per se. Most of the time, except for very particular things (e.g.
>     introducing a monad notation), the dsl approach is perfectly
>     sufficient. Don't think you absolutely need to extend the OCaml
>     grammar, embed your dsl directly into OCaml, using OCaml language
>     binders if you need variables.
>
>     Make libraries, not pet syntactic constructs.
>
>     Many things that can be done with camlp4, can be done with that
>     approach. Not only is it very elegant, it's much easier to maintain
>     w.r.t. the evolution of the OCaml language itself. The techniques in
>     these papers [1] should be more known and used.
>
>      > - It should not be external tool - like previously observed - it's
>      > difficult to support for code highlighters or refactoring (tools in
>      > general) - if it depends on a build step or command line options.
>
>
>
>
>     If you extend the grammar itself, code highlighters or any tool
>     expecting an OCaml expression is going to break whether the tool is
>     external or not. But for the rest of your comments I agree
>     wholeheartedly (even though I'm not sure all that power is needed,
>     but at least it would make the tool non-ugly to me).
>
>     Best,
>
>     Daniel
>
>     [1]
>     http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.31.9782&rep=rep1&type=pdf
>     <http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.31.9782&rep=rep1&type=pdf>
>     http://research.microsoft.com/apps/pubs/default.aspx?id=67334
>
>
>
>
>
>
>      > Therefore the main purpose of syntactical abstraction is missing
>      > - but that's not a problem - MetaOCaml wasn't designed for it.
>      >
>      > Things that I would like to see in future "incarnation" or
>     integrating of
>      > meta programming facilities to the language would be (beware
>     that's my
>      > blue dreams!):
>      >
>      > - first of all non destructive updates to the grammar e.g: "let open
>      > lang Sexp in ..." should open the Sexp syntax extension, install the
>      > grammar, but when it goes out of scope it should vanish. Currently
>      > Camlp4 can install, delete the rules after the functor is
>     applied, and
>      > no way of saying OK - let's go back.
>      >
>      > - Composable - in particular one language should behave like a
>     module,
>      > or functor, should have an interface consisting of grammar rules,
>     AST,
>      > AST transforms etc. So one could parametrise one syntax extension
>      > over another, and possibly reuse the language grammar or AST in
>      > other. Currently Camlp4 syntax extension is just a single separate
>      > module which when loaded possibly expects some existing grammar rules
>      > to be in place and mutates them as it's needed.
>      >
>      > - should be type safe and as mentioned before obey scoping rules. We
>      > should be able to propagate type information even when the syntax
>      > changes. This is difficult part - but I've seen it can be done
>     with some
>      > extra annotations - not talking about Camlp4
>      >
>      > - Recursive - it should be able to apply the grammar rules not only
>      > once but expand until it reached the fixpoint.
>      >
>      > - Reflective - it should be possible after each successful
>     expansion have
>      > the type information available for the next expansion.
>      >
>      > - Grammar itself should be lexer-less - memoizing PEG with left
>      > recursion - it's hurdle to define new grammar in terms of old lexer,
>      > or having a stateful lexer that depends on context.
>      >
>      > - It should not be external tool - like previously observed - it's
>      > difficult to support for code highlighters or refactoring (tools in
>      > general) - if it depends on a build step or command line options.
>      >
>      > That's said I find Camlp4 extremely useful for code generation
>     purposes
>      > - when I need to generate some ML code through quotations. Also, some
>      > very important projects depend on Camlp4 (or Camlp5) like Coq. I
>     don't
>      > see that ML can live without some meta programming facilities out
>     of the
>      > box.
>      >
>      > --
>      > Wojciech Meyer
>      > http://danmey.org
>      >
>      > --
>      > Caml-list mailing list. Subscription management and archives:
>      > https://sympa-roc.inria.fr/wws/info/caml-list
>      > Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>      > Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>
>
>
>     --
>     Caml-list mailing list.  Subscription management and archives:
>     https://sympa-roc.inria.fr/wws/info/caml-list
>     Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>     Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>