Hello Here is the latest OCaml Weekly News, for the week of January 05 to 12, 2021. Table of Contents ───────────────── Marshal determinism and stability Sedlex + Menhir parser for both tty and file parsing First release of awa-ssh Introducing Feather: A lightweight shell-scripting library for OCaml postdoc researcher and research engineer positions for CHERI and Arm verification First ocb (OCaml Badgen) release Release of OCaml-Git v3.0 and co Other OCaml News Old CWN Marshal determinism and stability ═════════════════════════════════ Archive: Continuing this thread, David Allsopp said ────────────────────────────────────────── A couple of notes on `Marshal', which I don't think have been covered • Although the guarantee is only between identical versions of OCaml, the implementation actually goes to considerable lengths to maintain backwards compatibility (so a value _written_ by older OCaml remains _readable_ in newer OCaml). Our own testsuite, for example, indirectly [includes a test which unmarshals a 3.12.1 value]. I don't know exactly how far back the support goes. • As it happens, the change which affected Unison in 4.08 was the first breaking change to Marshal since either 4.00 or 4.01. The fact that it doesn't break often (and that the two code paths - at least at present - are small) meant I have suggested a few months back that we could in future add an additional flag in the style of `Compat_32' to allow values to be written in a way which should be readable on older versions of OCaml. Indeed, it's small enough that flags could be added for the changes in 4.08 ([PR#1683]) and in 4.11 ([PR#8791]). • Neither point undermines using alternative formats either for network serialisation or persistent storage, for the many reasons discussed above! [includes a test which unmarshals a 3.12.1 value] [PR#1683] [PR#8791] Sedlex + Menhir parser for both tty and file parsing ════════════════════════════════════════════════════ Archive: Bernard Sufrin announced ──────────────────────── I am a great fan of Menhir, and have used it in several private language projects, using the ulexing scanner generator to provide Unicode-capable scanners. Alarmed by the obsolescence of ulexing, and needing a utf8-capable scanner in a hurry I decided to (teach myself to) use Sedlex. On the whole the experience was very satisfactory, and I found it straightforward to produce a variant of the sedlexing library which supports buffers with variable chunk sizes, thereby enabling efficient lexing on channels connected to files as well as immediately responsive lexing on channels connected to terminals. I also wanted to teach myself how to use the error-reporting, incremental, interfaces to Menhir-generated parsers. In the hope that it might be useful to others facing the same learning task, or the problem of adapting Sedlex for efficient interactive use, I have placed the example mock-S-Expression parser that resulted from this excursion in: [Git Repository: github.com/sufrin/InteractiveSedlexMenhirExample] [Git Repository: github.com/sufrin/InteractiveSedlexMenhirExample] First release of awa-ssh ════════════════════════ Archive: Hannes Mehnert announced ──────────────────────── I'm happy to announce that `awa-ssh' () has just been merged into opam-repository. It is a pure OCaml implementation of the ssh (secure shell, ) protocol. This is the initial release, please report issues you encounter. It was initially developed by Christiano Haesbaert in 2016, and revived mid-2019 by myself and in 2020 it was migrated to the MirageOS organization on GitHub for further development and maintenance. Both client and server code are present in the library (pure code in the main awa package), though the awa-lwt package implements only a server, and the awa-mirage package implements only a client. Tests and examples are in the test subdirectory. The MirageOS client has been successfully used to clone git repositories (on private servers, on GitHub, etc.). It supports apart from RSA keys also ED25519 keys (and key exchanges). Introducing Feather: A lightweight shell-scripting library for OCaml ════════════════════════════════════════════════════════════════════ Archive: Charles announced ───────────────── I wrote a shell scripting library called [Feather]. I like idea of writing bash-like code quickly, later being able to intersperse OCaml to add more typeful components as needed. It's kind of like [Shexp] but without the monadic interface and with Async support. ([Feather_async]) There's a tutorial and some examples in the link above but here's a quick taste: ┌──── │ open Feather │ │ let lines = find "." ~name:"*.ml" │ |. tr "/" "\n" │ |. map_lines ~f:String.capitalize │ |. sort │ |. process "uniq" [ "-c" ] │ |. process "sort" [ "-n" ] │ |. tail 4 │ |> collect_lines │ in │ String.concat ~sep:", " lines |> print_endline └──── Let me know if you have any feedback! And feel free to file bug reports [here]. Hope it ends up being useful, entertaining, or both! [Feather] [Shexp] [Feather_async] [here] postdoc researcher and research engineer positions for CHERI and Arm verification ═════════════════════════════════════════════════════════════════════════════════ Archive: Peter Sewell announced ────────────────────── We are looking for postdoctoral researchers and postdoctoral or postgraduate research engineers to help develop semantics and verification to improve the foundations and security of mainstream computer systems, for CHERI and Arm system software verification, at the University of Cambridge. OCaml expertise to help develop verification tools will be especially welcome. Closing date 13 January 2021 - see the advert . First ocb (OCaml Badgen) release ════════════════════════════════ Archive: zapashcanon announced ───────────────────── A few days ago, I released [ocb]. It's a library and a command-line tool to generate SVG badges. To get started quickly: ┌──── │ ocb --label Hello --color green --style flat --labelcolor white --status Goodbye └──── Will gives this result: [SVG example]. My first use case was [To.ml] where I'm using [bisect_ppx] to generate and deploy a [coverage report]. I wanted to display the coverage percentage in the README and tried existing tools but wasn't fully satisfied as they didn't work or were failing randomly. Now, [I'm generating the badge directly in a GitHub action]. The project was inspired by [badgen]. I still have to add support for icons and to improve the documentation but it's usable. [ocb] [SVG example] [To.ml] [bisect_ppx] [coverage report] [I'm generating the badge directly in a GitHub action] [badgen] Release of OCaml-Git v3.0 and co ════════════════════════════════ Archive: Ulugbek Abdullaev announced ─────────────────────────── We, the [`ocaml-git'] team, are happy to announce a new major release of `ocaml-git v3.0' and related libraries. [`ocaml-git'] Release Notes ╌╌╌╌╌╌╌╌╌╌╌╌╌ OCaml-Git v3.0 ┄┄┄┄┄┄┄┄┄┄┄┄┄┄ [*OCaml-Git*] is a library that implements `git' format and protocol implementation in pure OCaml. The library is used by libraries such as [`irmin'], a git-like distributed database, or [`pasteur'], a MirageOS unikernel-based snippet storage service. [*OCaml-Git*] [`irmin'] [`pasteur'] Changes ┈┈┈┈┈┈┈ The main goal behind this major release was to get better compatibility with various platforms, including [~MirageOS~](mirage.io), 32-bit platforms, and `js_of_ocaml'. In order to achieve that, we broke down `ocaml-git' into several components, which are represented as sub-libraries. We will describe some of those components later in this post. Along with better support for various platforms, `ocaml-git 3.0' also comes with SSH support for `fetch/push' and various bug fixes. The rest of the changes are mostly internal and pave a way for interesting features such as a full-blown `git' [garbage collector] and wire protocol v2 ([announcment] and [spec]). *References:* • Full [changes list] • [PR] that introduced the major rewrite of `ocaml-git' — In the new version of `ocaml-git', we try to have better separation of concerns by breaking some of the `ocaml-git' components into sub-libraries, which do not contain `git'-specific logic and can be reused for other purposes. [garbage collector] [announcment] [spec] [changes list] [PR] Carton ┄┄┄┄┄┄ Git uses [PACK files] to store old git objects such as commits and transfer objects over wire using git's wire protocols (`git-nss' library mentioned below implements [v1] of the protocol; [v2] implementation is in progress). [*Carton*] is a library to work with PACK files. The library does not contain git-specific code, so one can easily reuse the library and PACK format for non-git objects. One can see how `ocaml-git' uses `carton' for its purposes [here]. *References:* • [PR] that introduces `carton' [PACK files] [v1] [v2] [*Carton*] [here] [PR] Git-NSS (Not So Smart) ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ When one wants to synchronize with a remote repository using git, they need to use `git fetch/push'. Communication and synchronization/negotiation is defined by git *wire protocol*, which has two versions: older version 1 and newer leaner version 2. The protocols are defined for four wire transports: HTTP(S), SSH, and `git://' (TCP). [`Not-So-Smart'] library is a library that allows for such synchronization based on the git wire protocols but without git-specific code, meaning that files being fetched do not need to be git objects or that there is no assumptions on the "repository" that one is synchronizing with. So, as well as `carton', the library aims to be reusable for other purposes. This release features support for SSH using [awa-ssh] by @hannesm (see [the release]), support for [partial-clone] (of various `depth'), and memory consumption fixes for unikernels. *Note 1:* The library's name "Not so smart" is a play on the git's "smart" protocol, a part of wire protocol v1 over HTTP(S) transport. *Note 2:* only client side logic is implemented for wire protocols. The server-side is planned but not yet implemented. One can use `git' as the server for now. [`Not-So-Smart'] [awa-ssh] [the release] [partial-clone] Mimic ┄┄┄┄┄ [*Mimic*] is a small reimplementation of [`conduit'], a library that helps to abstract over a transport protocol such as HTTP(S) or SSH. In other words, the code using `mimic' can deal not with different types that represent an HTTP or SSH connection, but just deal, e.g., read from or write to, with a `flow' value, which hides protocol-specific details under its hood. — There are several independent libraries that were upgraded along with `ocaml-git 3.0'. [*Mimic*] [`conduit'] Duff v0.3 ┄┄┄┄┄┄┄┄┄ [*Duff*] is a library that implements git's [`libXdiff'] (`xdiff' algorithm) in OCaml. PACK files use a binary diff algorithm, `xdiff', to compress binary data. More on the project [page] and release [notes] for `ocaml-git 2.0'. [*Duff*] [`libXdiff'] [page] [notes] Changes ┈┈┈┈┈┈┈ This release fixes the support for 32-bit architecture platforms. Encore v0.7 ┄┄┄┄┄┄┄┄┄┄┄ [*Encore*] is a library that can create an encoder/decoder based on the format given. It also ensures isomorphism by construction. [*Encore*] Changes ┈┈┈┈┈┈┈ Extensive changes to the API. See the project page. Decompress v1.2.0 ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ [*Decompress*] is an OCaml implementation of certain decompression algorithms such as `Zlib', `Gzip', etc. [*Decompress*] Changes ┈┈┈┈┈┈┈ `ocaml-git 3.0' uses new version of `decompress' with extensive performance improvements documented in *Tarides's* blog [API changes] and [performance improvements]. We'd be happy to get your feedback or questions! :-) [API changes] [performance improvements] Other OCaml News ════════════════ From the ocamlcore planet blog ────────────────────────────── Here are links from many OCaml blogs aggregated at [OCaml Planet]. • [How We Lost at The Delphi Oracle Challenge] • [Tarides sponsors the Oxbridge Women in Computer Science Conference 2020] • [Coq 8.12.2 is out] • [First release of MetAcsl plugin] • [Announcing MirageOS 3.10] • [ ReScript 8.4] • [Coq 8.13+beta1 is out] [OCaml Planet] [How We Lost at The Delphi Oracle Challenge] [Tarides sponsors the Oxbridge Women in Computer Science Conference 2020] [Coq 8.12.2 is out] [First release of MetAcsl plugin] [Announcing MirageOS 3.10] [ ReScript 8.4] [Coq 8.13+beta1 is out] Old CWN ═══════ If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] [the archive] [RSS feed of the archives] [online] [Alan Schmitt]