From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Original-To: caml-list@sympa.inria.fr Delivered-To: caml-list@sympa.inria.fr Received: from mail1-relais-roc.national.inria.fr (mail1-relais-roc.national.inria.fr [192.134.164.82]) by sympa.inria.fr (Postfix) with ESMTPS id DF7367ED7A for ; Wed, 19 Sep 2012 23:21:07 +0200 (CEST) Received-SPF: None (mail1-smtp-roc.national.inria.fr: no sender authenticity information available from domain of bobzhang1988@gmail.com) identity=pra; client-ip=209.85.220.182; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="bobzhang1988@gmail.com"; x-conformance=sidf_compatible Received-SPF: Pass (mail1-smtp-roc.national.inria.fr: domain of bobzhang1988@gmail.com designates 209.85.220.182 as permitted sender) identity=mailfrom; client-ip=209.85.220.182; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="bobzhang1988@gmail.com"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail1-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@mail-vc0-f182.google.com) identity=helo; client-ip=209.85.220.182; receiver=mail1-smtp-roc.national.inria.fr; envelope-from="bobzhang1988@gmail.com"; x-sender="postmaster@mail-vc0-f182.google.com"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AjIBAOA2WlDRVdy2mGdsb2JhbABFhgq2MggjAQEBAQEICQ0HFCeCIAEBAQMBAQEBDwIJBhUIARsdAQMBCwYFCw0CAgUhAgIPAhIRAQUBHAYNAQcBAR6HTgEDCQYLm14JA4tWT4JzhQ8KGScNWYh0AQEEDIEVjyqBEgOSM4MxjkI/hCI X-IronPort-AV: E=Sophos;i="4.80,450,1344204000"; d="scan'208";a="173872933" Received: from mail-vc0-f182.google.com ([209.85.220.182]) by mail1-smtp-roc.national.inria.fr with ESMTP/TLS/RC4-SHA; 19 Sep 2012 23:21:07 +0200 Received: by vcbfw7 with SMTP id fw7so3028427vcb.27 for ; Wed, 19 Sep 2012 14:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:newsgroups:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=XrOlWwJlw2QaBw8bcOb57SXkOl0Zv8ZHg4kRUDc4UA0=; b=CMP7PpSd/wXFPsVsOvblzTeBZtJnABM/9Sm1lUAWAvJdq0Gx/W1acZOkEBraGtTpak QHhTMTef15g0hLcqAt51a3Ihdu2A/H6o9c0GVpOIW7AfMlqMoj4kq5Nc0aWczuSPfeN5 B7le/r3JFWd47lhOWX/j+01ENqeHr1N2ZT8xuQTbi+/SZSE4aVPNkauymC83yZ1/ntO6 RC7ORRL9v6QKzbb5ux7BbQLMJVIklTCoT7nMa4/hMu0ftHLCMfEyDToaMiqaDbM4IomU MwEalF3HDP2qhmHRYfp60rG17UO87ASzxLbMgoEuXfsM40Y6VkuxlBFpjoYzDkplm5p8 CXEg== Received: by 10.52.72.3 with SMTP id z3mr2236849vdu.12.1348089666044; Wed, 19 Sep 2012 14:21:06 -0700 (PDT) Received: from seas1228.wireless-pennnet.upenn.edu (seas1228.wireless-pennnet.upenn.edu. [158.130.108.206]) by mx.google.com with ESMTPS id zz18sm855472veb.4.2012.09.19.14.21.04 (version=SSLv3 cipher=OTHER); Wed, 19 Sep 2012 14:21:05 -0700 (PDT) Message-ID: <505A373F.40801@gmail.com> Date: Wed, 19 Sep 2012 17:21:03 -0400 From: Hongbo Zhang User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120907 Thunderbird/15.0.1 MIME-Version: 1.0 Newsgroups: gmane.comp.lang.caml.inria To: Wojciech Meyer CC: Caml List , Steve Zdancewic References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [Caml-list] Re: Call for collaboration on the future of camlp4 On 9/19/12 3:55 PM, Wojciech Meyer wrote: > Hi Hongboz, > Greetings, > Thanks for the slides, it's good. Indeed it makes sense now to focus on > extending Camlp4, however as usually there are some gotchas, the process > of extending syntax with Camlp4 is a bit not well known. Camlp5 has an > excellent documentation, and great support for some of the things. Yet, > Camlp4 is more modular, and is just easier to write application on top > of Camlp4. > Yes, camlp5 documentation is much more comprehensive for the users. But the underlying mechanism is not explained either, I did not see any difference for low-level users, another problem is that camlp5 is not heavily tested. > On OUD we had open ended discussions, and many people share (like me) > the same impression - -ppx that exist on trunk is powerful and that's > the way to implement most of the meta programming facilities (deriving > the code from type definitions, or using quotations to embed DSLs in > OCaml), but I believe that the direct syntax extensions (opening the > OCaml syntax) certainly has some benefits. How to do this in a clean way > myself - I don't know. I posted just some time ago what would be a macro > system of my dreams: -ppx is orthogonal to what camlp4 did, it's pretty easy to integrate 'ast rewriter' in Fan or camlp4. There are 3 problems with Ast Rewriter 1. No Quasiquotaion support It's ok to construct the Ast by hand for the first order macros, as I pointed out before, you can not write macros which write macros without quasi-quotation support. Remember that macros are itself programs, and there are dupliated code in macros as well, so you still need to scrap those dupliated code. So the expressivity to Ast Rewriter is like higher-order programming to first-order programming. It's a shame that so far there are not too many camlp4 plugins which generate macros to remove the duplicated code for camlp4 plugins. The camlp4 source tree itself is also written in a verbose way. Adding Quasiquotation support directly to current parsetree is not an easy way, for macro programming, you really want to quote, and antiquote everywhere, and create some illegal Ast. I have programmed a lot in Template Haskell, and I really appreciate that camlp4 have a much better quote-antiquot support. 2. No Delimited Syntax Extension support The Ast Rewriter can only override existing syntax(which is a bit fragile, IMO). To make things worse, if you want to link the compiler to your pre-processor, you really want to mark which piece of code should be eval during pre-processing or delayed at run-time. So, even the minimal delimited syntax extension will increase the expressivety a lot 3. General Syntax Extension General Syntax Extension is definitely very useful when I finished the functional parser part. A problem with delimited syntax extension support is that sometimes you really want to mix the existing grammars with your new grammars, currently there's no easy way to do that in delimited syntax extension. When we finished functional parser, we can restrict the syntax extension scope to the minimal expression level Btw, all the problems that camlp4 or Fan came across will not disappear in Ast Rewriter, you don't get any benefit from them, hygenic, no, the same problem. The beauty of camlp4 is that it is bootstrapped itself, when you evolve camlp4, you are making a more and more expressive system, part of my work is to re-write camlp4 to make it much more succinct. :-) > > https://sympa.inria.fr/sympa/arc/caml-list/2012-05/msg00184.html About your proposals: Runtime meta-programming is a generalisation of static meta programming. MetaOCaml has a nice set of abstraction to generate typechecking code - yes - either at runtime or during compile time. Meta-ocaml is orthogonal to camlp4 as well, but I did not see any hope that it will be pushed into ocaml compiler, since the main purpose of meta-ocaml is to do some run-time optimization, without a native jit compiler support, It does not make sense. " - first of all non destructive updates to the grammar e.g: "let open lang Sexp in ..." should open the Sexp syntax extension, install the grammar, but when it goes out of scope it should vanish. Currently Camlp4 can install, delete the rules after the functor is applied, and no way of saying OK - let's go back." Yes! - Composable - in particular one language should behave like a module, or functor, should have an interface consisting of grammar rules, AST, AST transforms etc. So one could parametrise one syntax extension over another, and possibly reuse the language grammar or AST in other. Currently Camlp4 syntax extension is just a single separate module which when loaded possibly expects some existing grammar rules to be in place and mutates them as it's needed. Yes! - should be type safe and as mentioned before obey scoping rules. We should be able to propagate type information even when the syntax changes. This is difficult part - but I've seen it can be done with some extra annotations - not talking about Camlp4 Personally, I prefer common lisp's macros to scheme. It's painful to play with racket's macros as well - Recursive - it should be able to apply the grammar rules not only once but expand until it reached the fixpoint. Yes! - Reflective - it should be possible after each successful expansion have the type information available for the next expansion. Possibly with the compiler library help - Grammar itself should be lexer-less - memoizing PEG with left recursion - it's hurdle to define new grammar in terms of old lexer, or having a stateful lexer that depends on context. Yes! - It should not be external tool - like previously observed - it's difficult to support for code highlighters or refactoring (tools in general) - if it depends on a build step or command line options. Yes!