Hello Here is the latest OCaml Weekly News, for the week of April 21 to 28, 2020. Table of Contents ───────────────── opam 2.0.7 and 2.1.0 alpha OCaml 4.11, release plan ocamlformat pre-commit hook New release of naboris 0.1.2 ANN: Releases of ringo resto 0.2 released Retrofitting Parallelism onto OCaml (research paper) Multicore Update: April 2020, with a preprint paper Why did Core remove polymorphic comparison operators in OCaml 4.10.0? New release of js_of_ocaml 3.6.0 Other OCaml News Old CWN opam 2.0.7 and 2.1.0 alpha ══════════════════════════ Archive: R. Boujbel announced ──────────────────── We are pleased to announce the release of [opam 2.0.7] and an [2.1.0 alpha]. This 2.0.7 version contains backported fixes, you can find more information in this [blog post]. The 2.1.0~alpha contains many new features (cf. [blog post] or [release note]). If you want to take a look, a few glitches or regressions are expected, please report them to [the bug-tracker]. *opam is a source-based package manager for OCaml. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.* [opam 2.0.7] [2.1.0 alpha] [blog post] [blog post] [release note] [the bug-tracker] Anil Madhavapeddy then added ──────────────────────────── Thanks for all the hard work that's gone into this release, @rjbou @AltGr and @dra27! To set expectations, this alpha release is for our users to smoke test the new features and let us know if they work for your usecases. In particular, the opam external dependency management and support for recursive pins are both commonly requested features. Please do take this alpha for a spin, and report feedback here on this thread. After this alpha is cut, there will be a sequence of beta releases (the number of which depend on the reported bug tail), and then the final opam 2.1.0 release. Your testing _now_ will greatly help us put a quality release out of the door. OCaml 4.11, release plan ════════════════════════ Archive: octachron announced ─────────────────── The new version of OCaml, OCaml 4.11.0, has started its bugfix period: the set of new features is now mostly frozen, and in the three upcoming months, we will focus mostly on fixing bugs. For this release cycle, we will experiment releasing an alpha version of the compiler. This new alpha version is expected to work as a synchronization point for people working on updating the opam ecosystem for the new release. Once the opam ecosystem is in shape for some wider audience testings, we will publish a beta version as usual. This should be happen around June. One of the most notable change in this release is `Statmemprof', a new statistical memory profiler directly integrated into the GC. The provisional Changes list is [here]. At this point of time, it is better to take this list with a grain of salt: there are a handful of new features that are still under integration, problematic features might be removed, and of course the list of bug fixes is incomplete. But one of the most notable feature in this change log, `Statmemprof' which a new statistical memory profiler API, is most probably here to stay. [here] Guillaume Munch-Maccagnoni then added ───────────────────────────────────── It should be mentioned that Memprof is documented as “~EXPERIMENTAL~”, and at least one breaking change is being considered in 4.12. This also mean that suggestion for improvement will be welcome (AFAIU). ocamlformat pre-commit hook ═══════════════════════════ Archive: Brendan Long announced ────────────────────── This is kind of trivial but I figured it might be useful for other people. We created a hook config for using [ocamlformat] with [pre-commit]: pre-commit is a tool that makes it easier to run checks on changed files before commiting them, and this makes it so you can auto-run ocamlformat and ensure no unformatted code gets into your repo. 1. [Install pre-commit] like `pip install pre-commit' 2. In your repo, add a .pre-commit-config.yaml like: ┌──── │ --- │ repos: │ - repo: https://github.com/arenadotio/pre-commit-ocamlformat │ rev: master # or pick a commit sha I guess │ hooks: │ - id: ocamlformat └──── 1. Run `pre-commit install' 2. Now every time you run `git commit' (or `pre-commit run'), it will run every staged OCaml file through ocamlformat and complain if there are any changes: ┌──── │ $ pre-commit run ocamlformat │ ocamlformat.....Failed │ - hook id: ocamlformat │ - files were modified by this hook │ $ git add . │ $ pre-commit run ocamlformat │ ocamlformat.....Passed └──── [ocamlformat] [pre-commit] [Install pre-commit] New release of naboris 0.1.2 ════════════════════════════ Archive: Shawn McGinty announced ─────────────────────── Simple http server for OCaml/ReasonML. [naboris] has been updated to 0.1.2 This release comes with a few improvements to the API but most notably it has much better documentation at [naboris.dev] [naboris] [naboris.dev] ANN: Releases of ringo ══════════════════════ Archive: Raphaël Proust announced ──────────────────────── On behalf of Nomadic Labs, I am please to announce the first few releases of Ringo: a library for caches. Ringo offers two kinds of caches: Maps for caches of key-value pairs and Sets for caches of simple elements. In addition, each kind of cache can be tweaked to handle their bounds differently. Ringo versions 0.1, 0.2 and 0.3 are available on `opam'. As the version number and the bundled announce suggests, this library is still in early phases of release: additional replacement policies will be added, the interface will probably change somewhat, etc. Suggestions welcome! Even though the interface is still in early phases of release, the implementation is covered by a lot of tests and is already in use in the Tezos project. The code is available at resto 0.2 released ══════════════════ Archive: Raphaël Proust announced ──────────────────────── Release of `resto 0.5' ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ On behalf of Nomadic Labs, I'm happy to announce the release of version 0.5 of `resto'. The main change brought in this release are: • relaxing of dependency bounds, • documentation! Retrofitting Parallelism onto OCaml (research paper) ════════════════════════════════════════════════════ Archive: Guillaume Munch-Maccagnoni announced ──────────────────────────────────── The following paper on the multicore GC design by @kayceesrk and his coauthors has been posted on arXiv today and might interest the community: Multicore Update: April 2020, with a preprint paper ═══════════════════════════════════════════════════ Archive: Anil Madhavapeddy announced ─────────────────────────── Welcome to the April 2020 update from the Multicore OCaml team, across the UK, India, France and Switzerland! Although most of us are in lockdown, we continue to march forward. As with [previous updates], thanks to @shakthimaan and @kayceesrk for help assembling it all. [previous updates] ◊ Preprint: Retrofitting Parallelism onto OCaml We've put up a preprint of a paper titled ["Retrofitting Parallelism onto OCaml" ] for which we would be grateful to receive feedback. The paper lays out the problem space for the multicore extension of OCaml and presents the design choices, implementation and evaluation of the concurrent garbage collector (GC). Note that this is *not a final paper* as it is currently under peer review, so any feedback given now can still be incorporated. Please use the e-mail contact details in the [pdf paper] for @kayceesrk and myself so we can aggregate (and acknowledge!) any such comments. ["Retrofitting Parallelism onto OCaml" ] [pdf paper] ◊ Rebasing Progress The Multicore OCaml rebase from 4.06.1 has gained momentum. We have successfully rebased the parallel-minor-GC all the way onto the [4.09 OCaml trees]. We will publish updated opam packages when we get to the recently branched 4.11 in the next couple of weeks. Rebasing complex features like this is a "slow and steady" process due to the number of intermediate conflicts and bootstrapping, so we will not be publishing opam packages for every intermediate version – instead, the 4.11 trees will form the new "stable base" for any PRs. [4.09 OCaml trees] ◊ Higher-level Domainslib API A thread from [last month's update] on building a parallel raytracer led to some useful advancements in the [domainslib] library to provide async/await-style task support. See the updates below for more details. There is also an interesting discussion on [ocaml-multicore/ocaml-multicore#324] about how to go about profiling and optimising your own small programs. More experiments with parallel algorithms with different scheduling properties would be most useful at this time. [last month's update] [domainslib] [ocaml-multicore/ocaml-multicore#324] ◊ Upstreamed features in 4.11 The [4.11 release has recently branched] and has the following multicore-relevant changes in it: • A concurrency-safe marshalling implementation (originally in [ocaml#9293], then implemented again in [ocaml#9353]). This will have a slight speed hit to marshalling-heavy programs, so feedback on trying this in your projects with 4.11 will be appreciated to the upstream OCaml issue tracker. • A runtime eventlog tracing system using the CTF format is on the verge of being merged in 4.11 over in [ocaml#9082]. This will also be of interest to those who need sequential program profiling, and is a generalisation of the infrastructure that was essential to our development of the multicore GC. If anyone is interested in helping with hacking on the OCaml side of CTF support to build clients, please get in touch with me or @kayceesrk. In addition to the above highlights, we have also been making continuous improvements and additions to the Sandmark benchmarking test infrastructure. The various ongoing and completed tasks are provided below for your reference. [4.11 release has recently branched] [ocaml#9293] [ocaml#9353] [ocaml#9082] Multicore OCaml ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ◊ Ongoing • [ocaml-multicore/ocaml-multicore] Promote Multicore OCaml to trunk The rebasing of Multicore OCaml from 4.06 to 4.10 is being worked, and we are now at 4.09! In a few weeks, we expect to complete the rebase to the latest trunk release. • [ocaml-multicore/eventlog-tools]: OCaml Eventlog Tools A project that provides a set of tools for runtime tracing for OCaml 4.11.0 and higher has been created. This includes a simple OCaml decoder for eventlog's trace and a built-in chrome converter tool. • [ocaml-multicore/domainslib#5] Add parallel_scan to domainslib A [parallel_scan] implementation that uses the Task API with prefix_sum and summed_area_table has now been added to the Domain-level Parallel Programming library for Multicore OCaml (domainslib) library. [ocaml-multicore/ocaml-multicore] [ocaml-multicore/eventlog-tools] [ocaml-multicore/domainslib#5] [parallel_scan] ◊ Completed The following PRs have been merged into Multicore OCaml and its ecosystem projects: • [ocaml-multicore/ocaml-multicore#328] Multicore compiler with Flambda Support for Flambda has been merged into the Multicore OCaml project repository. The translation is now performed at cmmgen instead of lambda for clambda conversion. • [ocaml-multicore/ocaml-multicore#324] Optimizing a Multicore program The following [documentation] provides a detailed example on how to do performance debugging for a Multicore program to improve the runtime performance. • [ocaml-multicore/ocaml-multicore#325] Added eventlog_to_latencies.py script A script to generate a latency report from an eventlog has now been included in the ocaml-multicore repository. • [ocaml-multicore/domainslib#4] Add support for task_pools The domainslib library now has support for work-stealing task pools with async/await parallelism. You are encouraged to try the [examples]. [ocaml-multicore/ocaml-multicore#328] [ocaml-multicore/ocaml-multicore#324] [documentation] [ocaml-multicore/ocaml-multicore#325] [ocaml-multicore/domainslib#4] [examples] Benchmarking ╌╌╌╌╌╌╌╌╌╌╌╌ A number of new benchmarks are being ported to the [Sandmark] performance benchmarking test suite. • [ocaml-bench/sandmark#104] Added python pip3 dependency A check_dependency function has now been defined in the Makefile along with a list of dependencies and pip packages for Ubuntu. You can now run `make depend' prior to building the benchmark suite to ensure that you have the required software. The `python3-pip' package has been added to the list of dependencies. • [ocaml-bench/sandmark#96] Sandmark Analyze notebooks The setup, builds and execution scripts for developer branches on bench2.ocamllabs.io have been migrated to winter.ocamllabs.io. A UI and automated script driven notebooks for analyzing sequential bench results is being worked upon. • [ocaml-bench/sandmark#108] Porting mergesort and matrix multiplication using Task Pool API library This is an on-going PR to implement merge sort and matrix_multiplication using `parallel_for'. • [cubicle] `Cubicle' is a model checker and an automatic SMT theorem prover. At present, it is being ported to Multicore OCaml, and this is a work in progress. • [raytracers] Raytracers is a repository that contains ray tracer implementation for different parallel functional programming languages. The OCaml implementation has now been updated to use the new `Domainslib.Task' API. Also, a few [experiments] were performed on flambda parameters for the Multicore raytracer which gives around 25% speedup, but it does not yet remove the boxing of floats. The experiments are to be repeated with a merge against the wip flambda2 trees on 4.11, that removes float boxing. [Sandmark] [ocaml-bench/sandmark#104] [ocaml-bench/sandmark#96] [ocaml-bench/sandmark#108] [cubicle] [raytracers] [experiments] OCaml ╌╌╌╌╌ ◊ Ongoing • [ocaml/ocaml#9082] Eventlog tracing system A substantial number of commits have gone into this PR based on reviews and feedback. These include updates to the configure script, handling warnings and exceptions, adding build support for Windows, removing unused code and coding style changes. This patch will be cherry-picked for the 4.11 release. [ocaml/ocaml#9082] ◊ Completed • [ocaml/ocaml#9353] Reimplement `output_value' using a hash table to detect sharing This PR which implements a hash table and bit vector as required for Multicore OCaml has been merged to 4.11. Our thanks as always go to all the OCaml developers and users in the community for their continued support, and contribution to the project! [ocaml/ocaml#9353] Acronyms ╌╌╌╌╌╌╌╌ • API: Application Programming Interface • GC: Garbage Collector • PIP: Pip Installs Python • PR: Pull Request • SMT: Satisfiability Modulo Theories • UI: User Interface Why did Core remove polymorphic comparison operators in OCaml 4.10.0? ═════════════════════════════════════════════════════════════════════ Archive: Trung Ta asked ────────────── I'm using the Core library in a project, and recently when I upgraded my OCaml from 4.08.1 to 4.10.0, plenty of compilation errors suddenly appears for comparison expressions like: `if (xs = []) then ...' or `if (x = true) then ...' I saw that this change was discussed in this [thread] about monomorphic comparison operators in Base, but did not expect that Core would make it a default behavior. So I'd like to ask since which version that Core removed such polymorphic comparison operators? (I couldn't find it in release notes of Core) Also, if I defined a sum type like `type ternary = True | False | Unkn', what will be a correct way to write `if (x = True) then ...' (which is allowed in the new Core)? I can temporarily fix by writing `if (x == True) then ...', but using `==' doesn't seem correct, since `==' is about comparing physical objects… Thanks for spending your time to check my question. [thread] Aaron L. Zeng replied ───────────────────── The change was announced in , although unfortunately it doesn't look like the CHANGES.md file was updated in the repo. I would consider the thread to be the canonical announcement. Also, if I defined a sum type like `type ternary = True | False | Unkn' , what will be a correct way to write `if (x = True) then ...' (which is allowed in the new Core)? Here's a few suggestions: 1. Define equality/compare functions using [`ppx_compare'] ┌──── │ type ternary = True | False | Unkn [@@deriving equal] │ │ let f x = if (equal_ternary x True) then ... └──── 2. Define equality/compare functions manually ┌──── │ let equal_ternary t1 t2 = │ match t1, t2 with │ | True, True | False, False | Unkn, Unkn -> true │ | _ -> false └──── 3. Explicitly request polymorphic comparison operators using the `Poly' module: ┌──── │ let f x = if (Poly.(=) x True) then ... └──── [`ppx_compare'] Trung said and Aaron L. Zeng replied ──────────────────────────────────── btw, ┌──── │ type ternary = True | False | Unkn [@@deriving equal] └──── should be: `[@@deriving eq]' That depends on which preprocessor you are using. `[@@deriving equal]' comes from ppx_compare, whereas `[@@deriving eq]' comes from [ppx_deriving]. Base/Core and the like have better support for the former, which is a Jane Street project, although you can feel free to use the latter—the naming conventions are different, so it may not be as convenient. [ppx_deriving] New release of js_of_ocaml 3.6.0 ════════════════════════════════ Archive: Hhugo announced ─────────────── I'm pleased to announce the release [Js_of_ocaml] 3.6.0. Js_of_ocaml is a compiler from OCaml bytecode to JavaScript. It makes it possible to run pure OCaml programs in JavaScript environment like browsers and Node.js. Try it [online]. Notable changes: • The `js_of_ocaml' compiler now accepts sub-commands (link, build-runtime, build-fs, ..). The plan for future versions is to remove other binary (e.g. jsoo_link) and consolidate everything inside the `js_of_ocaml' binary itself. • The standard JavaScript runtime is now embedded in the compiler (findlib is no longer needed to locate it) • Add support for the Str library (Regular expressions and high-level string processing) shipped with the OCaml compiler • Change memory representation of `Int64.t' (you might need to update your JavaScript stubs) • Many bug fixes (thanks to many more tests) [Js_of_ocaml] [online] Kirill Alexander Khalitov asked and Hhugo replied ───────────────────────────────────────────────── 1 Does the project have roadmap? There is no official roadmap, the project evolves based on issues, requests and contributions. You can take a look at some of the [Github issues] 2 Is the project generally exists only for Ocsigen needs? js_of_ocaml is used by various projects, not only Ocsigen. See [Bonsai], [sketch-sh] or [jscoq] for instance. 3 Will it be adopted for modern front-end development (commonjs/esmodules compatibility for working with existing building tools ex. webpack, etc). Being more friendly with the JavaScript ecosystem as been discussed here and there in the past but little has been done, possibly by lack of interest or use cases. 4 Does the project competing with bucklescript? I don't think so. The two projects have different goals and different audience. One of Js_of_ocaml main goal is to stay as close as possible to the official OCaml semantic, allowing to leverage existing OCaml libraries without any modification. 5 Why not to do ocaml to js compiler tools (based on js_of_ocaml and bucklescript experience) that combine possibility of using native ocaml and js libraries across back-end and front-end like implemented in Scala.js/Fable F#? I don't understand this question. I would expect both js_of_ocaml and bucklescript to be like Scala.js/Fable F# in their own way. [Github issues] [Bonsai] [sketch-sh] [jscoq] Kirill Alexander Khalitov then said ─────────────────────────────────── I mean what Scala.js/Fable F# allows to use the most native libraries (not all) and JS ones (from npm registry or from custom JS module) in one project (ex. front-end). But in case of js_of_ocaml we limited to native OCaml libs and "HTML scripts" (not JS compatible modules). For bucklescript case we have whole JS ecosystem but have no access to useful native libs from opam registry. Xavier Van de Woestyne replied ────────────────────────────── In Js_of_OCaml, you can deal with JavaScript's module (and npm/yarn), using for example: ┌──── │ (* val require : string -> 'a *) │ let require module_name = │ let open Js.Unsafe in │ fun_call │ (js_expr "require") │ [|inject (Js.string module_name)|] └──── Other OCaml News ════════════════ >From the ocamlcore planet blog ────────────────────────────── Here are links from many OCaml blogs aggregated at [OCaml Planet]. • [Every proof assistant] • [opam 2.0.7 release] • [opam 2.1.0 alpha is here!] [OCaml Planet] [Every proof assistant] [opam 2.0.7 release] [opam 2.1.0 alpha is here!] Old CWN ═══════ If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] [the archive] [RSS feed of the archives] [online] [Alan Schmitt]