Hello Here is the latest OCaml Weekly News, for the week of May 25 to June 01, 2021. Table of Contents ───────────────── Dream — a simple, yet feature-complete Web framework Ocaml developer at Routine, Paris, Remote OK Feather 0.2.0 BAP 2.3.0 Release Building Ahrefs codebase with Melange Lwt 5.4.1 Other OCaml News Old CWN Dream — a simple, yet feature-complete Web framework ════════════════════════════════════════════════════ Archive: Anton Bachin announced ────────────────────── I am pleased to announce [*Dream*], a very easy-to-use Web framework with high performance, secure defaults, and thorough documentation! It is available now from opam, with `opam install dream'. Dream offers: • [WebSockets] and [GraphQL]. • A [template syntax], which you can see in the image above. • Trivial [HTTPS and HTTP/2 support], allowing simple deployments without a proxy. • [Sessions] with pluggable [back ends]. • Easy [secure cookies] and [CSRF-safe forms]. …and more, yet Dream sticks to a simple programming model: • Web apps are just [bare functions] from requests to responses. • [Middlewares] are just higher-order wrapper functions. • [Routes] tell the [router] which of these functions to call. Indeed, for those who like algebra, there is a certain [structure] to Dream. However, that's not the point of this post! Dream is meant to be very easy to understand. It sticks to base types, introducing only a few types of its own, and uses existing languages, such as HTML for templates, and URLs for routes. Dream itself is one module in one opam package, which lives in a monorepo. The [docs] are on one page. Dream is loosely coupled. Even though Dream offers many defaults, it is unopinionated, and you can quickly configure or replace anything. For example, it is easy to [use TyXML] for templates, and Dream happily supports such usage with examples. Security-sensitive features, such as cookies, are arranged so that simple and obvious usage is automatically secure. Wherever security still depends on the Dream app, the docs [highlight] it. Dream has selected a modern [cipher] as a default, supports [key rotation], and offers suggestions for other purposes, such as password hashing. It implements and abstracts away all of the [OWASP] security guidelines that are relevant to its level. Dream is designed for full internationalization. It has a centralized [error handler] that intercepts even lower-level HTTP errors, so that you can decorate them with your app's own error template, and leak no hardcoded strings. Dream's URL encoders [favor] internationalized (UTF-8) URIs, and the router accepts them. Finally, Dream is designed for a wide range of applications, including with or without a proxy, standalone or embedded in larger binaries, and with external static assets or [assets compiled in]. [*Dream*] [WebSockets] [GraphQL] [template syntax] [HTTPS and HTTP/2 support] [Sessions] [back ends] [secure cookies] [CSRF-safe forms] [bare functions] [Middlewares] [Routes] [router] [structure] [docs] [use TyXML] [highlight] [cipher] [key rotation] [OWASP] [error handler] [favor] [assets compiled in] Documentation ╌╌╌╌╌╌╌╌╌╌╌╌╌ Dream is very extensively documented. See… • [*Examples*], the first several of which make up a tutorial. Each example is a complete project. • The online [*playground*], which features many of the examples, and is itself a [Dream app]! • The [*API docs*]. In particular, see • Deployment examples for [Heroku], Digital Ocean [with Docker], and Digital Ocean [with systemd], all of which include GitHub Actions scripts and instructions. • Full-stack examples with [js_of_ocaml], [ReScript], and [Melange]. • Examples in [Reason syntax]. • Development [watching] and [live reloading]. [*Examples*] [*playground*] [Dream app] [*API docs*] [Heroku] [with Docker] [with systemd] [js_of_ocaml] [ReScript] [Melange] [Reason syntax] [watching] [live reloading] Contributing ╌╌╌╌╌╌╌╌╌╌╌╌ Dream has already received several very helpful [contributions], and more are very welcome! See [`CONTRIBUTING.md']. I must also acknowledge all the people working on Dream's [dependecies] and [prior art]. In particular, Dream relies heavily on the HTTP and WebSocket [servers] primarily by Spiros Eliopoulos (@seliopou) and Antonio Nuno Monteiro (@anmonteiro). Apart from accepting code, docs, and examples, Dream will happily link to: • Blogs and articles, as different people learn best from different presentations. • "Downstream" libraries to use with Dream. For example, Thibaut Mattio (@tmattio) is working on [dream-livereload], a live-reloading middleware for Dream, similar to the [example], which he also contributed! Once dream-livereload is slightly more mature, Dream will link to it from its README. There is also [dream-serve], a live-reloading static site server based on Dream and libuv, which was used to develop the docs. [contributions] [`CONTRIBUTING.md'] [dependecies] [prior art] [servers] [dream-livereload] [example] [dream-serve] Roadmap ╌╌╌╌╌╌╌ Dream is currently in an alpha state. It is thought (by me) to be internally quite stable. However, there will probably be various API tweaks before release 1.0.0. My current, rough plan is to release several alphas of Dream over six months or so. The releases will address: 1. Flow control for very large responses, and getting the "advanced" part of the I/O API to be as close to zero-copy and non-allocating as possible (or reasonable). 2. Remaining (optional) [security enhancements], such as a [default content security policy]. 3. Remaining [session improvements], such as re-keying. 4. Friction in handling of JSON, database access, etc. This is not properly part of or due to Dream, but it should be addressed for a better Web development experience. 5. Multicore and effects support. That's all. Let's bring OCaml to the Web! Happy Web programming! [security enhancements] [default content security policy] [session improvements] Anton Bachin then added ─────────────────────── For readers who saw the repo during the earlier ["leak,"] the main updates are: • A large number of new examples, including [deployment]. • The [playground], which runs the examples, and itself served as a test. • An esy-based [quick start] script. There have also been very many smaller changes to the code, API, and the rest of the docs, but the above changes are the biggest "chunks." The rest is too much to detail :) ["leak,"] [deployment] [playground] [quick start] Ivan Gotovchits asked and Anton Bachin replied ────────────────────────────────────────────── I was always wondering how does the source code that uses [templates] work with OCaml tooling, in particular with merlin, ocp-indent, ocaml-format, tuareg and other editor modes? It doesn't work well in practice with anything other than syntax highlighting. Note that you control the syntax mode with the extension. If your template is mostly HTML, you can name it `foo.eml.html'. The intent is that the templates should contain mostly HTML in a large project, and most of them would be in their own `template/' subdirectory. OCaml tooling wouldn't be needed for these mostly-HTML files. For a still-small, but real example of this, see the Playground's [`client.eml.html']. The one-file `.ml' projects with templates, where tooling is a problem, are mostly good for the very first steps of getting started, and examples. There is also an issue about this in the repo, [#55 " how to apply ocamlformat"]. Note that, as in the announcement text, you can use Dream with other templaters, including [TyXML], which has an [HTML PPX]. In addition, if you are using Reason, you can use [TyXML JSX]. Either of these options interacts well with tooling, as far as I know. I didn't make TyXML the default because it considerably increases the Dream learning curve for getting basic tasks done. However, Dream still supports the choice of using TyXML with examples and links. [templates] [`client.eml.html'] [#55 " how to apply ocamlformat"] [TyXML] [HTML PPX] [TyXML JSX] Ocaml developer at Routine, Paris, Remote OK ════════════════════════════════════════════ Archive: mefyl announced ─────────────── Routine () is looking for an OCaml developer. Routine is a personal productivity assistant. The technological revolves heavily around OCaml which represents 90% of the codebase, the remaining 10% being the UI in Typescript and Vue.js. We target both the browser and desktop through electron, using Js_of_ocaml. While the product is "just" a web app, our technological and academic background leads us to use designs that, I think, can pique the interest of seasoned Ocaml developer. Amongst other things : • Type-driven programming based on ppx derivers that produces typescript declaration for frontend bindings, JSON schema to expose and consume external REST APIs (Google, Notion, …), automatic SQL bindings, etc. • Angstrom based parsing for the interactive console with highlighting and completion. • Incremental based state updates to refresh minimal subsets of the app. • Highly concurrent implementation through Lwt, exception-free design. We use state of the art CI/CD and development processes. We plan on distributing open sources packages of these utilities (type-driven system, Google API bindings, Notion API bindings, …). Future exciting subjects could be extending Angstrom with manual rollback to implement generic completions or binding Vue in OCaml directly using melange or rescript to achieve rock solid typing down to the very frontend code (highly prospective teases, don't quote me on this yet :). The company is very much a startup, having just completed YC batch W21 and closed its first round of investment. Salary is up to market standard depending on the profile, plus usual options package, to be discussed. While we expect great OCaml and general computer science proficiency, we're open to most levels of experience. Thoroughness and a love for well rounded, robust and beautiful software design is a must have - but that comes bundled with OCaml love, right ? Do not hesitate to reach out for any question here, at quentin.hocquet@routine.co or refer this to someone who may be interested. Thanks for your time and happy camel riding ! Feather 0.2.0 ═════════════ Archive: Charles announced ───────────────── I'm happy to announce feather version 0.2.0! Feather is a minimal library for bash-like scripting and process execution. ([github], [opam]) This release fixes some bugs and adds three new functions • `val and_ : cmd -> cmd -> cmd' — chain two commands, short circuiting if the first fails, akin to bash's `&&' operator. • `val or_ : cmd -> cmd -> cmd' — chain two commands, short circuiting if the first succeeds, akin to bash's `||' operator. • `val sequence : cmd -> cmd -> cmd' — chain two commands regardless of exit status. We include two new operators `&&.' and `||.' which correspond to `and_' and `or_' respectively. They'll be found in the `Feather.Infix' module, which has been renamed from `Feather.File_redirection_infix'. Many thanks to new contributors @Firobe @juxd and @tmarti2 for making this release possible! [github] [opam] BAP 2.3.0 Release ═════════════════ Archive: Ivan Gotovchits announced ───────────────────────── We're proud to release the next stable version of Carnegie Mellon University Binary Analysis Platform ([BAP]). The full list of changes can be found on the [release page] but the most interesting new features are highlighted below. [BAP] [release page] The Primus Lisp Frontend ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Now BAP is able to understand not only binary programs but sources written in Primus Lisp. In case if you don't know, [Primus Lisp] is our DSL for writing analysis and library stubs (e.g., to specify semantics of missing library functions). Now, it is possible to reify Primus Lisp programs into static representation. For example, we can translate the following Lisp program ┌──── │ ;; file demo.lisp │ │ (defun example1 (x) │ (set R0 1) │ (set R1 2) │ (set R3 (+ R1 R2 (* R1 R2 3))) │ (memory-write R4 (+ R3 R1)) │ (if (> R0 (* R0 R0)) │ (exec-addr 0xDEADBEEF) │ (set R0 (* R0 R2 R3)))) └──── into the BIL (BAP Instruction Language) AST and then pretty print it, ┌──── │ $ bap show --primus-lisp-load=demo --target=armv7+le -obap:bil example1 │ example1: │ "{ │ R0 := 1 │ R1 := 2 │ R3 := R1 + R2 + R1 * R2 * 3 │ mem := mem with [R4] <- low:8[R3 + R1] │ #1 := R0 * R0 < R0 │ if (#1) { │ jmp 0xDEADBEEF │ } │ else { │ R0 := R0 * R2 * R3 │ } │ }" └──── This new feature not only allows us to reify our Lisp stubs into static form but also enables the main killer feature. It is now possible to specify the semantics of machine instructions in Primus Lisp. This feature enables rapid development and experimentation with CPU semantics. And this brings us to the next new feature. [Primus Lisp] New Target: RISC-V ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ The first application of the Primus Lisp Frontend was writing the RISC-V semantics. It took me only one day to write the semantic of the [minimal subset] of RISC-V instruction. Well, partially it is because RISCV-V is truly RISC, like the `add' instruction just adds, ┌──── │ (defun ADDI (dst rm rn) │ (set$ dst (+ rm rn))) └──── [minimal subset] New Target: ARMv8 (Aarch64) ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ The next target that we tried was Aarch64, the 64-bit ARM architecture. It was a little bit [harder] but still definitely more readable than the official ARM semantics. [harder] Adds namespaces (packages) to Primus Lisp ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Since now we have much more code in Primus Lisp we found ourselves struggling with name clashes. The Primus Lisp program model is a set of mututally recursive overloaded definitions, so naming things is crucial for us. Therefore we implemented namespaces (which are, following Common Lisp trandition, named packages). We ended up in a very Common Lisp look and fill but without inheriting CL problems, like the dependency on the order of inclusion and package redefinitions, and so on. Given our model, and that Primus Lisp features type inference and Haskell-style type classes for overloading, it wasn't that easy to implement :) Adds the `bap dependencies' command ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ The [command] outputs program dependencies such as libraries and symbols. The information is collected recursively with various output options, including dependency graph, YAML, JSON, and SEXP. Much like `nm~+~ldd' on steroids and cross-platform (works on PE/ELF/COFF, and on binaries that are not native to the host). So it could be quite useful even if you're not doing program analysis, but just want to solve a nasty missing library feature or figure our what programs use what libraries, e.g., ┌──── │ $ bap dependencies `which ping` --recursive --ldconfig -ograph | graph-easy --as boxart │ ┌────────────────┐ │ │ libresolv.so.2 │ ──────────────────────────────────┐ │ └────────────────┘ │ │ ▲ │ │ │ │ │ │ │ │ ┌──────────────┐ ┌──────────────────────────┐ ┌────────────────┐ │ │ │ libidn.so.11 │ ◀── │ ping │ ──▶ │ libnettle.so.6 │ │ │ └──────────────┘ └──────────────────────────┘ └────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ │ │ │ ┌────────────────┐ │ │ │ │ │ │ libcap.so.2 │ │ │ │ │ │ └────────────────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ ▼ ▼ │ │ │ │ ┌──────────────────────────┐ │ │ │ └────────────────▶ │ libc.so.6 │ ◀─────┘ │ │ └──────────────────────────┘ │ │ │ ▲ │ │ │ └───────────────────────────┘ │ ▼ │ ┌────────────────┐ │ │ ld-linux.so.2 │ │ └────────────────┘ └──── [command] What's Next? ╌╌╌╌╌╌╌╌╌╌╌╌ We are working on decompilation and integrating with Ghidra, so in 2.4.0 you should expect that bap will output C code for binaries. But it is not all, we're even working into turning BAP into a program analysis framework that enables analysis of source code programs. And even crazier, we're working on adding compilation capabilities to BAP, i.e., an ability to compile/recompile the input sources. So soon BAP will outlive its name, or we will need to find a new interpretation for the BAP acronym, something like the Best Analysis Platform ;) We also plan to make BAP more available for non-seasoned OCaml users and want to push bap into mainstream Linux distributions and overall lower the entrance barrier. Of course, with the end goal to lure users into installing opam)) Questions and Suggestions ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Please, do not hesitate to ask questions and provide your suggestions and, ideally, join our [community]. Even if you don't plan to work on binary analysis, BAP offers lots of opportunities for writing your toy programs for learning the language, or maybe even student projects. [community] Building Ahrefs codebase with Melange ═════════════════════════════════════ Archive: Javier Chávarri announced ───────────────────────── At Ahrefs, we make extensive use of OCaml and ReScript —previously [known as BuckleScript]. So we have been following the latest developments in the ReScript ecosystem with great interest. A few months ago, [António Monteiro] released [Melange], a fork of ReScript with an emphasis of keeping compatibility with OCaml ecosystem. One of the key features of Melange is that it uses OCaml 4.12, with all the upsides that that entails (ppxlib, let syntax, better errors, …). Besides that, Melange has been modeled recently [as just a `compiler-libs' library], so it can be integrated with other OCaml code in a single opam switch. We decided to give Melange a try recently at Ahrefs, and shared the results of this experiment in a blog post: We are currently looking into how a deeper integration with Dune would look like. If your team or company has tried Melange, or is interested on doing so, we would be very interested to hear your use cases and share experiences. [known as BuckleScript] [António Monteiro] [Melange] [as just a `compiler-libs' library] Lwt 5.4.1 ═════════ Archive: Raphaël Proust announced ──────────────────────── We are glad to announce the release of version 5.4.1 of Lwt: a bugfix-only release. You can update to this version in `opam': ┌──── │ opam update │ opam upgrade lwt └──── Thanks to the contributors for finding and fixing the bugs, leading to this release. Check out the release notes (link above) for a full list. Other OCaml News ════════════════ From the ocamlcore planet blog ────────────────────────────── Here are links from many OCaml blogs aggregated at [OCaml Planet]. • [Beta release of Frama-C 23.0~rc1 (Vanadium)] • [Building Ahrefs codebase with Melange] • [Computing an integer using a Grothendieck topos] • [ ReScript 9.1] • [Tutorial: Format Module of OCaml] • [Tarides project SCoP is selected as one of the brightest Data Portability projects in Europe!] • [Alt-Ergo Users’ Club Annual Meeting (2021)] [OCaml Planet] [Beta release of Frama-C 23.0~rc1 (Vanadium)] [Building Ahrefs codebase with Melange] [Computing an integer using a Grothendieck topos] [ ReScript 9.1] [Tutorial: Format Module of OCaml] [Tarides project SCoP is selected as one of the brightest Data Portability projects in Europe!] [Alt-Ergo Users’ Club Annual Meeting (2021)] Old CWN ═══════ If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] [the archive] [RSS feed of the archives] [online] [Alan Schmitt]