[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

[Alan Schmitt <alan.schmitt@polytechnique.org>]

[-- Attachment #1: Type: text/plain, Size: 19204 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of July 20 to 27,
2021.

Table of Contents
─────────────────

pyre-ast: full-fidelity Python parser in OCaml
OCaml+Opam Images for Docker for Windows
Borns a stream talking about OCaml/Reason & ReScript language
An Update on the State of the PPX Ecosystem and `ppxlib''s Transition
How to send email from Dream
Old CWN


pyre-ast: full-fidelity Python parser in OCaml
══════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-pyre-ast-full-fidelity-python-parser-in-ocaml/8177/1>


Jia Chen announced
──────────────────

  I am happy to announce the initial opam release of [`pyre-ast'], a
  Python parsing library.

  The library features its full-fidelity to the official Python
  spec. Apart from a few technical edge cases, as long as a given file
  can be parsed/rejected by the CPython interpreter, `pyre-ast' will be
  able to parse/reject it as well. Furthermore, abstract syntax trees
  obtained from `pyre-ast' is guaranteed to 100% match the results
  obtained by Python's own `ast.parse' API, down to every AST node and
  every line/column number.

  Another notable feature of this library is that it represents the
  Python syntax using the *tagless-final style*. This style typically
  offers more flexibility and extensibility for the downstream consumers
  of the syntax, and allow them to build up their analysis without
  explicitly constructing a syntax tree. That said, for developers who
  are less familiar with the tagless-final approach, we also offer
  alternative interfaces that operates on traditional syntax tree
  represented as algebraic data types.

  Documentation of the library can be found [here].

  The reason why we can can claim full-conformance with CPython is
  really simple: the library is, under the hood, merely an OCaml wrapper
  around the parsing logic in CPython source code.  The project was
  initially motivated to replace the custom `menhir'-based parser
  currently used in the Pyre type checker (hence the name), but I
  figured that it would be useful to release this as a standalone `opam'
  package to the community so other static Python analyzers or other
  DSLs with Python-based syntax can leverage it as well.

  The library has yet to be put into production for Pyre (I'm working on
  it though) so please do expect bugs/jankiness at times. Feedback and
  bug reports are very welcomed.

  Happy parsing!


[`pyre-ast'] <https://github.com/grievejia/pyre-ast>

[here] <https://grievejia.github.io/pyre-ast/doc/pyre-ast/>


OCaml+Opam Images for Docker for Windows
════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-ocaml-opam-images-for-docker-for-windows/8179/1>


Antonin Décimo announced
────────────────────────

  I'm glad to announce the availability of OCaml and opam [native
  Windows Container][windows-containers] images for Docker for
  Windows. This is the result of my hard work at Tarides, with precious
  help from @dra27, @talex5, @avsm, and the rest of the team.

  They can be found under the [ocaml/opam][hub] repository in the Docker
  Hub. Try them with [Docker for Windows][docker-for-windows]! Be sure
  to [switch Docker to Native Windows Containers][enable-native].

  ┌────
  │ docker run -it ocaml/opam:windows-mingw
  │ docker run -it ocaml/opam:windows-msvc
  └────

  We provide images for the mingw-w64 (from OCaml 4.02 to 4.12) and the
  MSVC (from OCaml 4.06 to 4.12) ports. They are based on each release
  of Windows 10 amd64 currently supported by [Microsoft on the Docker
  Hub][mcr]. The images use opam 2.0, and we plan to update to opam 2.1
  when it's released. The images also ship a [Cygwin][cygwin]
  installation, [Git for Windows][git-for-windows], and the [winget
  package manager][winget].

  We use @fdopen's [OCaml for Windows][ocaml-for-windows] distribution
  and opam-repository fork. As it is getting deprecated at the end of
  August 2021, we'll transition to opam 2.1 and the standard
  opam-repository when that happens.

  In order to get the correct environment for any `RUN' command
  involving OCaml or opam, prefix the command with

  • `ocaml-env exec --64 --' if based on mingw-w64; or
  • `ocaml-env exec --64 --ms=vs2019 --' if based on MSVC.

  The images are built at <https://base-images.ocamllabs.io/>, using an
  [OCurrent][ocurrent] pipeline that [builds Docker
  images][docker-base-images]. You can rebuild them yourself using the
  [OCluster][ocluster] set of tools that I have ported to Windows.

  We provide a comprehensive set of tags (replace _port_ with either
  _mingw_ or _msvc_):
  • `windows-port': the latest version of OCaml for each Windows
    version;
  • `windows-port-winver': the latest version of OCaml for Windows 10
    _winver_;
  • `windows-port-ocaml-mlver': OCaml version _mlver_ for each Windows
    version;
  • `windows-port-winver-ocaml-mlver': OCaml version _mlver_ for Window
    10 _winver_.

  When the Windows version is not specified in the tag, the image is a
  multiarch image that will work on every supported version of Windows
  10. Docker automatically selects the appropriate one based on the host
  version.

  We will be using these images in the upcoming `ocaml-ci' and
  `opam-repo-ci' for Windows.

  Further work on these include the transition to opam 2.1, and we'll
  provide the Cygwin port of OCaml when it's fixed upstream and
  available in the Cygwin package repository.

  Happy hacking!


Borns a stream talking about OCaml/Reason & ReScript language
═════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-borns-a-stream-talking-about-ocaml-reason-rescript-language/8185/1>


David Sancho announced
──────────────────────

  I'm very excited to announce starting a new show in Twitch to bring
  OCaml, Reason and ReScript community best brains to casually
  talk. It's called emelleTV

  It's made by [@fakenickels] and myself [@davesnx], and we will try to
  do our best!

  Our first guest is [@___zth___]

  <https://aws1.discourse-cdn.com/standard11/uploads/ocaml/optimized/2X/e/e9f08607687aeb843968a430e4e9082541cf87c2_2_1380x690.jpeg>

  We go live on [http://twitch.tv/emelletv] next Wednesday.  Subscribe
  to not miss it!

  Thanks for reading, hope to see you there!


[@fakenickels] <https://twitter.com/fakenickels>

[@davesnx] <https://twitter.com/davesnx>

[@___zth___] <https://twitter.com/___zth___>

[http://twitch.tv/emelletv] <http://twitch.tv/emelletv>


An Update on the State of the PPX Ecosystem and `ppxlib''s Transition
═════════════════════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/an-update-on-the-state-of-the-ppx-ecosystem-and-ppxlib-s-transition/8200/1>


Sonja Heinze announced
──────────────────────

  I hope you're all having a nice summer (or a nice whichever season
  you're in, of course)!  We've set up a new [wiki page on the ppxlib
  repository] containing a status overview of the current `ppxlib'
  transition, which aims at keeping the PPX ecosystem always
  up-to-date. We'll keep that wiki page up-to-date, as well.

  @jeremiedimino and @NathanReb have already explained our three-part
  plan for this transition in different posts here on discuss. Nothing
  has changed in that plan, but it has been a while since we [last
  posted about the overall transition] and even longer since we [last
  posted about the `Astlib' transition in detail]. So if you want, you
  can refresh your memory about that transition and get updated about
  its current state (in more detail than the new wiki page) by reading
  this post.


[wiki page on the ppxlib repository]
<https://github.com/ocaml-ppx/ppxlib/wiki/The-State-of-the-PPX-Transition>

[last posted about the overall transition]
<https://discuss.ocaml.org/t/ppxlib-0-22-an-update-on-the-state-of-ppx/7296>

[last posted about the `Astlib' transition in detail]
<https://discuss.ocaml.org/t/ppx-omp-2-0-0-and-next-steps/6231>

Which Issues `ppxlib' was Facing
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

  With `ocaml-migrate-parsetree' (`OMP'), the PPX ecosystem became
  cross-compiler-compatible.  With `ppxlib', the latest compiler
  features were supported more easily and broadly within the PPX
  ecosystem, while `ppxlib' also brought along other improvements such
  as the one in performance and the clear composition semantics when
  using several PPXs. With that, both `OMP' and `ppxlib' have taken away
  several maintenance burdens from the PPX maintainers and have created
  a more homogeneous and up-to-date PPX ecosystem. However, we were
  facing the following issues:
  1. To keep the PPX ecosystem cross-compiler compatible
     1. `ppxlib' was handling parts of the unstable `compiler-libs' API
        to abstracting them away;
     2. the `OMP~/~ppxlib' maintainers needed to keep the AST migration
        information up-to-date by coordination with the compiler devs.
  2. To guarantee new feature support, `ppxlib' needed to bump the
     `ppxlib' AST to the newest version.
  3. Bumping the AST implies a breaking change. That was an issue for a
     homogeneous and up-to-date PPX ecosystem.
  4. Not all PPXs migrated from `OMP' to `ppxlib'. That was also an
     issue for a homogeneous and up-to-date PPX ecosystem.

  Some time ago, there was the very ambitious plan of tackling Issues 1,
  2, and 3 all at once by writing a stable AST abstraction and
  upstreaming it to the compiler. That plan has been put on ice for
  now. Instead we're currently on track with a more down-to-earth plan,
  outlined below.


Tackling the Issues in Three Parts
╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌

  The plan we're currently following contains three simultaneous
  parts. It approaches three of the four issues I've pointed out
  above. However, it leaves the need to bump the AST (Issue 2)
  untouched.


Part One: `Astlib' as an Interface between `ppxlib' and the Compiler
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

  The first part works towards continuous cross-compiler compatibility
  (Issue 1 above) while making the situation of still having PPXs based
  on `OMP' (Issue 4 above) even more of a problem. It consists of
  implementing an interface module called `Astlib' between `ppxlib' and
  the compiler, then upstreaming it to the compiler. As long as `Astlib'
  is stable and up-to-date, the rest of `ppxlib' won't be affected by
  any compiler changes—neither by new AST versions nor by compiler
  library changes.

  The first step of this part of the plan was moving the `OMP' driver
  and other `OMP' features from `OMP' to `ppxlib'. That was done in
  August 2020, and it introduced `OMP2'. Since the PPX driver has to be
  unique, this was the start of having the PPX ecosystem split into the
  two incompatible worlds of `OMP1' PPXs on one hand and `ppxlib' PPXs
  on the other hand.

  By now, we have written [`Astlib' as an internal `ppxlib' library] and
  have reduced `ppxlib''s compiler library usage as much as possible to
  keep `Astlib' minimal. As you can see, it contains a minimal compiler
  library sub-API in addition to the former `OMP' modules of our
  supported ASTs and the migration information between them. We will
  upstream `Astlib' to the compiler asking for it to be kept stable and
  up-to-date, while also keeping our local copy for old compiler
  support.


[`Astlib' as an internal `ppxlib' library]
<https://github.com/ocaml-ppx/ppxlib/tree/master/astlib>


Part Two: Sending Patch PRs when Bumping the AST
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

  So, thanks to Part One of the plan, `ppxlib' will always be compatible
  with the development compiler _OCaml trunk_ and the newest compiler
  version. However, to also support the newest compiler features, we
  need to bump the internal `ppxlib' AST to the newest version. That
  modifies some of the AST nodes and so it breaks any PPX that rewrites
  one of those nodes (Issue 3 above). Usually just a handful of PPXs are
  affected, but we still want them to be up-to-date.

  Our current plan doesn't provide a solution for that problem, but it
  does make handling the problem more efficient and, once again, it
  takes away the burden from the PPX maintainers.  Since the AST bump to
  `4.10', whenever we bump the AST, we send patch PRs to the PPXs we
  break. Not much has changed since February, when @NathanReb last
  [explained our workflow of sending patch PRs] in detail.  To some it
  up: we create a workspace with all `ppxlib' reverse dependencies on
  opam fulfilling a certain standard, which we call the
  _ppx-universe_. We then fix the PPXs that break all at once and open
  the PRs.

  Lately, the _ppx-universe_ has also proven very useful to make
  well-founded decisions regarding our API by having an easy look at our
  reverse dependencies. You can find a [_ppx-universe_ snapshot],
  currently from March, on our repo.

  In our experience, once the _ppx-universe_ is created and "builds up
  to the expected breakages," writing a couple of patches takes very
  little time, so we plan to make the tooling that creates and interacts
  with the workspace more sophisticated.


[explained our workflow of sending patch PRs]
<https://discuss.ocaml.org/t/ppxlib-0-22-an-update-on-the-state-of-ppx/7296>

[_ppx-universe_ snapshot] <https://github.com/ocaml-ppx/ppx_universe>


Part Three: Porting PPXs to Put an End to the "Split-World Situation"
┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄

  As explained above, Part One split the PPXs into the two incompatible
  worlds of `OMP1' PPXs on one hand and `ppxlib' PPXs on the other
  hand. That made the fact that some PPXs were still based on `OMP'
  (Issue 4 above) even more of a problem. For some PPX maintainers, the
  reason to avoid porting their PPXs to `ppxlib' was that `ppxlib'
  depended on `base' and `stdio', so we decided to tackle this situation
  by three means:

  • Dropping the `base' and the `stdio' dependencies, which was done in
    August last year. Now, all dependencies are the very basic `ocaml',
    `dune', `ocaml-compiler-libs', `stdlib-shims', `sexplib0' and
    `ppx_derivers'.
  • Porting and reviewing some of the most important PPXs ourselves. So
    far we've ported `js_of_ocaml', `bisect_ppx', and `tyxml' with the
    help of the respective maintainers, and we've also reviewed several
    ports.
  • Spreading the word about the need to port PPXs and asking for help.

  About a year ago, we made a non-exhaustive [list of PPXs that needed
  to be ported].  Since then, this community has proven to be awesome
  and there has been an amazing porting effort by a lot of people. So by
  now, all packages on that list have been ported with the exception of
  one(*). So hopefully the "split world" situation can soon be
  considered past.  :tada:

  By the way, thanks to all involved in porting PPXs to `ppxlib'! It has
  been a great joint effort so far. :heart: And if anyone still has or
  comes across a project somewhere that needs porting and wants to port
  it, that's awesome!

  You can find the full list of opam packages that are still stuck in
  the `OMP1' world by [filtering for them in opam's health check
  pipeline].  However, notice that that's a generated list, so it also
  contains libraries that intrinsically form part of the `OMP1'
  ecosystem (such as `ppx_tools_versioned'), PPXs that have already been
  ported but haven't relesed their port on opam yet (such as
  `graphql_ppx'), deprecated PPXs that aren't marked as deprecated yet
  (such as `mirage-dns'), and several PPXs that only transitively depend
  on `OMP1'.

  (*) `ppx_import' has a PR for a port to `ppxlib', but it's not quite
  ready to be merged just yet.


[list of PPXs that needed to be ported]
<https://github.com/ocaml-ppx/ppxlib/issues?q=is%3Aissue+label%3Aport-to-ppxlib+>

[filtering for them in opam's health check pipeline]
<http://check.ocamllabs.io:8080/?comp=4.12&available=4.12&show-latest-only=true&sort-by-revdeps=true&maintainers=&logsearch=ocaml-migrate-parsetree%5C.1%5C.8%5C.0&logsearch_comp=4.12>


How to send email from Dream
════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/how-to-send-email-from-dream/8201/1>


Joe Thomas announced
────────────────────

  I’ve written a short [blog post ] about what I learned building simple
  email features for a web server written in the Dream framework. The
  accompanying source code is available here:

  <https://github.com/jsthomas/dream-email-example>

  I’m interested in adding more examples and tutorials to the OCaml
  ecosystem and would be happy to get your feedback, positive or
  negative, on this write-up (here or via email/github/discord).


[blog post ] <https://jsthomas.github.io/ocaml-email.html>


Old CWN
═══════

  If you happen to miss a CWN, you can [send me a message] and I'll mail
  it to you, or go take a look at [the archive] or the [RSS feed of the
  archives].

  If you also wish to receive it every week by mail, you may subscribe
  [online].

  [Alan Schmitt]


[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <https://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <https://alan.petitepomme.net/>


[-- Attachment #2: Type: text/html, Size: 30858 bytes --]