Re: [Caml-list] PPX is harmful to our community in the long term

[Gerd Stolpmann <info@gerd-stolpmann.de>]

So far I understand, with "syntax" you really mean syntax-as-string.

I think most often the AST changes by adding more fields or more AST 
nodes. Wouldn't the situation already improve a lot if we switched to 
record variants for the AST types? (Besides that the compiler would get 
way more readable.) New fields wouldn't be a big problem anymore because 
they are looked up by name and not by position.

Gerd


On 21.04.17 18:04, Yotam Barnoy wrote:
> My 2 cents: I personally think we did PPX wrong. PPX should have used
> the syntax as its starting point for full backwards compatibility.
> Currently, the PPX cycle is
>
> OCaml: { syntax -> AST -> serialized AST } -> PPX: {serialized AST ->
> AST -> PPX modification -> serialized AST} -> OCaml: {serialized AST
> -> AST -> ... }
>
> Exposing the AST is the cause of the problem you mention. If instead,
> every PPX extension had a particular OCaml compiler's syntax parser
> and syntax printer integrated into it, and the PPX cycle was:
>
> PPX: { syntax -> AST -> PPX modification -> syntax } -> compiler: {
> syntax -> AST -> ...}
>
> We would have far fewer issues with PPX plugins, since they would be
> as backwards-compatible as the syntax.
>
> I think this mistake happened because everyone was trying to move away
> from camlp4's complexity, and we didn't notice that the problem with
> camlp4 wasn't the approach, but rather the fact that it wasn't making
> use of the same code as the compiler and therefore required its own
> heavy learning-curve and parallel maintenance to keep up with the
> compiler. To be fair though, camlp4 came before the age of OCaml on
> github, opam, and the associated easy code/library sharing.
>
> The only issue I can see with this idea of improving PPX is that the
> PPX-generated syntax wouldn't have the original source locations. We'd
> need an annotation that indicated the original source locations
> (perhaps row and column offsets) when a PPX translated to modified
> syntax and printed it out, so that the compiler's errors would have
> proper locations relative to the original source.
>
> -Yotam
>
>
> On Fri, Apr 21, 2017 at 11:41 AM, Hongbo Zhang (BLOOMBERG/ 731 LEX)
> <hzhang295@bloomberg.net> wrote:
>> Dear OCaml developers:
>> Given that bitten by PPX from time to time, finally, I think it is a time to
>> spend two hours sharing my experience with PPX and why you(the OCaml library
>> developer) should avoid PPX as much as you can.
>>
>> Here is a story I just experienced this morning, I tried to install a
>> package from opam, and it complained my compiler is too old - 4.02.3, to be
>> honest, 4.02.3 is still a pretty modern OCaml compiler, even debian stable
>> still stays on 4.01. Anyway, I switched to 4.04.1, after half an hour, it
>> failed to compile again, complaning about some ppx error message. This is
>> not my first time experience, and finally it made me to write an essay about
>> why PPX is harmful.
>> PPX is a compiler plugin, it imposes a very large compiler surface API to
>> your library, and we don't have any backward compatibility guarantee from
>> the compiler, which means your library will only work against a specific
>> compiler. Even worse, OCaml is an elegant but small community, we don't have
>> too many maintainers for a library, if you have a library which relies on
>> PPX (the dependency chain could be really really huge, for example,
>> ppx_metaquot depends on typing environment, you can find lots of stories
>> about node_modules in Node community), it will probably not work against
>> next version of OCaml compiler, and it will be a huge maintenance overhead
>> for other people to pick it up.
>>
>> OCaml is already a very expressive language, probably more expressive than
>> any other mainstream language, (Go, Java, C/C++, etc), it is fine to write
>> some boilerplate code, or we can cut PPX as a dev dependency, after your
>> PPXed your code, check in the generated source code(via -dsource), so it
>> will not bring dependency to end users.
>> There are some valid use cases of PPX, for example, in BuckleScript or
>> JS_of_OCaml, we want to customize OCaml language a bit for external FFI, or
>> if you have a very large team, and committed effort to maintain your PPX.
>> Happy hacking in OCaml without PPX, Thanks -- Hongbo
>>
>>