OCaml Weekly News

As soon as the new version of ppxlib is released, we will work towards our next milestone. As a reminder, our current goal is to setup a ppx ecosystem that is continously compatible with the trunk of OCaml. To achieve that goal, we plan to add a stable API called "Astlib" on top of the compiler libraries. To keep things sustainable on the compiler side and increase flexibility, Astlib will be minimal and will be targeted at ppxlib only rather than be a general API aimed at ppx authors.

The difficulty of this API is that it must expose a stable interface to the OCaml AST, which is composed of a large collection of data types. To make it work, we plan to use the technology developed in ocaml-migrate-parsetree; i.e. whole AST migration functions.

While we eventually want Astlib to live in the compiler repository, we will initially develop it inside the ppxlib repository. Once it is ready, we will submit it for inclusion in the compiler. Although, we will keep a copy inside ppxlib for older versions of the compiler.

We also plan to setup a smooth workflow for compiler developers to update Astlib when they change the development AST.

Once this is all done, we will be in a situation where the ppx ecosystem is compatible with the trunk of OCaml at all time. And as a result, new releases of the compiler will no longer break ppx packages as long as they limit themselves to the ppxlib API.

While this work will make the ecosystem compatible with the trunk of OCaml at all times, it will essentially move the backward compatibility issue from the compiler to ppxlib.[1] This will already give us a lot more flexibility as for instance a single version of ppxlib can be compatible with a wide range of OCaml versions. However, we recognise that it is not usual to ask a community to rely on an unstable API.

We made this choice as a trade-off between sustainability and complexity. Indeed, we want to maintain Astlib and Ppxlib over the long term and the best way to make things sustainable is to use simple and clear designs. While we do have solutions in our sleeves that would provide a fully stable ppx API, these are much more complicated to maintain and work with.

To mitigate this, we are setting up a Dune based workflow to upgrade all ppx rewriters at once. So once the system is rolling and if your ppx rewriters are up to date and using Dune, you should expect to receive pull requests as we update ppxlib. This last part will take some time to be fully rolling, so please bear with us :)

In any case, about a year after this new world is fully setup, we will review the situation and decide whether it is sustainable or whether we need to go all the way and mint a fully stable ppx API.

• today: ocaml-migrate-parsetree 2.0.0 is being released
• next week: a ppxlib compatible version is released
• December 2020: astlib is ready inside the ppxlib repository
• next OCaml release after that: astlib lives in the compiler
• September 2021: we review the situation and decide what to do next

[1]: At any given time the API of ppxlib refer to a single version of the OCaml AST. In order to allow OCaml users to enjoy both ppx rewriters and new language features, the version of the AST selected by ppxlib needs to be bumped after each release of the compiler, which is a breaking change that has the potential to break several ppx packages. As a result, ppx packages will still need to be regularly updated in order to stay compatible with the latest version of ppxlib.