Re: [Caml-list] PPX is harmful to our community in the long term

[Yotam Barnoy <yotambarnoy@gmail.com>]

My 2 cents: I personally think we did PPX wrong. PPX should have used
the syntax as its starting point for full backwards compatibility.
Currently, the PPX cycle is

OCaml: { syntax -> AST -> serialized AST } -> PPX: {serialized AST ->
AST -> PPX modification -> serialized AST} -> OCaml: {serialized AST
-> AST -> ... }

Exposing the AST is the cause of the problem you mention. If instead,
every PPX extension had a particular OCaml compiler's syntax parser
and syntax printer integrated into it, and the PPX cycle was:

PPX: { syntax -> AST -> PPX modification -> syntax } -> compiler: {
syntax -> AST -> ...}

We would have far fewer issues with PPX plugins, since they would be
as backwards-compatible as the syntax.

I think this mistake happened because everyone was trying to move away
from camlp4's complexity, and we didn't notice that the problem with
camlp4 wasn't the approach, but rather the fact that it wasn't making
use of the same code as the compiler and therefore required its own
heavy learning-curve and parallel maintenance to keep up with the
compiler. To be fair though, camlp4 came before the age of OCaml on
github, opam, and the associated easy code/library sharing.

The only issue I can see with this idea of improving PPX is that the
PPX-generated syntax wouldn't have the original source locations. We'd
need an annotation that indicated the original source locations
(perhaps row and column offsets) when a PPX translated to modified
syntax and printed it out, so that the compiler's errors would have
proper locations relative to the original source.

-Yotam


On Fri, Apr 21, 2017 at 11:41 AM, Hongbo Zhang (BLOOMBERG/ 731 LEX)
<hzhang295@bloomberg.net> wrote:
> Dear OCaml developers:
> Given that bitten by PPX from time to time, finally, I think it is a time to
> spend two hours sharing my experience with PPX and why you(the OCaml library
> developer) should avoid PPX as much as you can.
>
> Here is a story I just experienced this morning, I tried to install a
> package from opam, and it complained my compiler is too old - 4.02.3, to be
> honest, 4.02.3 is still a pretty modern OCaml compiler, even debian stable
> still stays on 4.01. Anyway, I switched to 4.04.1, after half an hour, it
> failed to compile again, complaning about some ppx error message. This is
> not my first time experience, and finally it made me to write an essay about
> why PPX is harmful.
> PPX is a compiler plugin, it imposes a very large compiler surface API to
> your library, and we don't have any backward compatibility guarantee from
> the compiler, which means your library will only work against a specific
> compiler. Even worse, OCaml is an elegant but small community, we don't have
> too many maintainers for a library, if you have a library which relies on
> PPX (the dependency chain could be really really huge, for example,
> ppx_metaquot depends on typing environment, you can find lots of stories
> about node_modules in Node community), it will probably not work against
> next version of OCaml compiler, and it will be a huge maintenance overhead
> for other people to pick it up.
>
> OCaml is already a very expressive language, probably more expressive than
> any other mainstream language, (Go, Java, C/C++, etc), it is fine to write
> some boilerplate code, or we can cut PPX as a dev dependency, after your
> PPXed your code, check in the generated source code(via -dsource), so it
> will not bring dependency to end users.
> There are some valid use cases of PPX, for example, in BuckleScript or
> JS_of_OCaml, we want to customize OCaml language a bit for external FFI, or
> if you have a very large team, and committed effort to maintain your PPX.
> Happy hacking in OCaml without PPX, Thanks -- Hongbo
>
>