Re: [Caml-list] proposal for finding, loading and composing ppx preprocessors

[Alain Frisch <alain@frisch.fr>]

On 07/15/2014 03:59 PM, Benjamin Canou wrote:
> My belief is that there is room for a less ambitious, plug-in based
> (statically or dynamically linked) annotation mechanism, on top of PPX.
> For instance, if we consider the current use of camlp4, we can assume
> that most ppxs will probably just define some annotation on some
> specific kind of AST node in order to rewrite them and / or insert
> auxiliary code, without carrying so much state. Having a common
> mechanism for registering such common simple tasks and assigning to
> names to them could be useful, without breaking the model.

The registration mechanism exists:  a ppx processor is supposed to call 
Ast_mapper.register.  By default, this runs directly the main program to 
support for ppx protocol for this single processor, but you can very 
well override the behavior of Ast_mapper.register by calling 
Ast_mapper.register_function first.

You can look at https://github.com/alainfrisch/ppx_drivers for an 
experiment of creating such a driver.

But this notion of registration is quite independent on how multiple 
processors interact.  I'm yet to be convinced that a composition mode 
more fine-grained that function composition is worth the extra mental 
overhead.  It would also be necessary to decide between top-down and 
bottom-up rewriting, none of them being the best pick for all 
processors.  Typically, a macro expander would want other processors to 
apply to the result of macro expansions (i.e. top-down rewriting style), 
while a type-conv-like processor might prefer a bottom-up rewriting (so 
that the macro expander can run first on type declarations before it 
takes control).

Concerning the notion of state in ppx processors, I've written the 
following ppx processors:

  - sedlex (a lexer generator): it needs to remember about all lexer 
definitions in the current unit to be able to generate some global 
function definitions (shared by various lexers)  (for instance, we don't 
want to generate definitions used only by code that would be excluded by 
a conditional compilation processor)

  - ppx_metaquot: some extension nodes with very short names %e, %t, %p 
are recognized only in the context of an enclosing extension node such 
as %expr ( 
https://github.com/alainfrisch/ppx_tools/blob/master/ppx_metaquot.ml )

I expect most other ppx processors to be slightly more complex than 
purely local transformations, which makes it even more difficult to 
reason on local composition (and "purely local" term rewriting systems 
are already hard to reason about their composition).

> Well, one of the use cases of extension nodes is to integrate external
> notations into literals such as [%json{| ... |}]. I believe IDEs could
> still use a little help to know how to format these, instead of showing
> plain OCaml strings.

Indeed, although I think this is going to be a rather rare use of 
extension points / quoted strings.


-- Alain