OCaml Weekly News
Hello
Here is the latest OCaml Weekly News, for the week of March 22 to 29, 2022.
Table of Contents
- pyml_bindgen: a CLI app to generate Python bindings directly from OCaml value specifications
- Tarides is hiring!
- For Diversity and the OCaml Community: Outreachy Summer 2022
- Caqti 1.8.0 and related news
- First release of prbnmcn-dagger
- MirageOS 4.0
- OCaml 4.14.0 is released
- ocaml-in-python.0.1.0: Effortless Python bindings for OCaml modules
- Old CWN
pyml_bindgen: a CLI app to generate Python bindings directly from OCaml value specifications
Ryan Moore announced
New releases
Version 0.3.0 and 0.3.1 are now available on GitHub. 0.3.0 has been merged into opam, and a PR for 0.3.1 has been opened. The change log has more details about the changes.
Binding tuples
You can now bind tuples directly. Here's a Python function that takes two lists of points (where each "point" is a
tuple like (x, y)
) and adds them together
def add(points1, points2): return [(x1 + y1, x2 + y2) for (x1, x2), (y1, y2) in zip(points1, points2)]
And you could bind it using tuples from the OCaml side as well.
val add : points1:(int * int) list -> points2:(int * int) list -> unit -> (int * int) list
Note there are some restrictions regarding tuples, which you can read about here, here, or here.
Attributes
You can use attributes on value specifications. Currently the only one supported is py_fun_name
, which allows you
to decouple the Python method name and the generated OCaml function name.
As an example, take the following Python function, which adds to "things".
def add(x, y): return x + y
You could bind multiple OCaml functions to this single function now.
val add_int : x:int -> y:int -> unit -> int [@@py_fun_name add] val add_float : x:float -> y:float -> unit -> float [@@py_fun_name add] val add_string : x:string -> y:string -> unit -> string [@@py_fun_name add]
Python magic methods
This is also nice for binding Python magic
methods. For example, you don't have to use
__init__
as the name of the OCaml function you use to make instances of a Python class. You can bind it to a more
natural name like create
or make
.
val create : name:string -> age:int -> unit -> t [@@py_fun_name __init__]
Using Pytypes.pyobject directly
Sometimes you may not want to bother converting Python types to normal OCaml types at all. You can do that now in
value specifications by using the Pytypes.pyobject
and Py.Object.t
types directly.
Fewer dependencies
re
is now used instead of re2
, which drops the number of dependencies that need to be installed by about half.
Additionally, core
, core_bench
, and bisect_ppx
don't need to be installed if you want to install pyml_bindgen
directly from the git repository, which greatly cuts the required dependencies in this case.
Thanks again to UnixJunkie for spurring many of these updates!
Tarides is hiring!
Thomas Gazagnaire announced
Following the recent announcement about Tarides (joining forces with OCaml Labs and Segfault System), we are now looking to expand our team with experienced software engineers, compassionate team leads and experts in software consulting services. Our ambition is to bring OCaml to a vast set of new developers and industries. We want to make developers more productive by spending less time on fixing bugs and more on writing new features. And we want the software industry to build more robust and performant systems that can last for decades.
We are looking for:
- Experienced Software Engineer(s) to take part in the development of Irmin. You will be part of the team that designs, builds and ships Irmin libraries and applications to our community and customers.
- Team Lead(s) who cares about motivating their team members, supporting their growth and development and successfully delivering the team's objectives on time.
- A Head of Consulting Services to diversify our technical teams and commercial services portfolio. You'll be the first hire for this brand new department and will have the opportunity to help us build our services structure from scratch, including our strategy, processes, tools, and team.
We are always looking for great OCaml enthusiasts to join our team, so even if these job descriptions do not fit your profile precisely, you are welcome to send us a spontaneous application!
For Diversity and the OCaml Community: Outreachy Summer 2022
Archive: https://discuss.ocaml.org/t/for-diversity-and-the-ocaml-community-outreachy-summer-2022/9234/6
Deep in this thread, Aya announced
@pitag and I have resubmitted the PPX derivers project for this Summer 2022 round: Expand OCaml's library of
standard derivers! This is the same project I was the intern for this past Winter 2022 round, where the goal is to
build up a standard derivers library, like ppx_deriving
, using
the updated ppxlib
API.
I'm excited to be supporting @pitag with mentoring, and for the opportunity to stay involved now that my internship has ended :smiley:
Caqti 1.8.0 and related news
"Petter A. Urkedal announced
I am happy to announce the second release of Caqti this year. The reason for the quick succession is partly an adjustment to the new API for request construction and partly that matchable error conditions did not make it into the previous release. You can see the full release notes below.
I would also like to thank OCaml Software Foundation for sponsoring my efforts on the Caqti project this year, also including most of the work that went into the previous release.
One feature in progress is a new driver based on the pure-OCaml pgx which should make it possible, with some additional changes to the way drivers are loaded, to target MirageOS. I am note sure if this can be done in a minor release or will require a Caqti 2 branch.
Release Notes
New features:
- A matchable representation of common causes of errors on the database side is now available, with limitations. It focuses on conditions which seem most likely useful to handle. At the moment we lack extended error codes from SQLite3 needed to make the cause fully precise.
- Expose the underlying error details from database client libraries. This is meant to be use as a last resort, and requires directly linking with the relevant drivers.
- A second set of request construction operators
->.
,->?
,->!
, and->*
were introduced after experience with converting existing code. Given the parameter and result type they return a function which constructs a request directly from a query string. Avoiding the need to compose with@:-
simplifies local opens and usage withList.map
etc. - Environment variables are now expanded in the debug log when using the new request constructors introduced in 1.7.0.
- A new
?tweaks_version
connection parameter has been added to control when the client is ready to adapt to changes in database session parameters or other adjustments of the interaction with specific database systems. [More details available in the documentation.] - Enable foreign key constraint checks for SQLite3 starting at tweaks version 1.7.
Fixes:
- Fixed debug logging to pass the correct driver info to the query callback instead of a dummy driver info which would cause a failure if unsupported.
Deprecations:
- The
-->
operator was renamed to-->!
, with a deprecated alias, for consistency with the new->!
operator. - The old convenience interface for creating requests has been deprecated in favour of the new infix operators and the new query template parser.
- Documented-only deprecations of
Caqti_sql_io
,Caqti_lwt_sql_io
, andCaqti_async_sql_io
have been annotated.
First release of prbnmcn-dagger
Igarnier announced
I'm proud to announce the release of version 0.0.2 of prbnmcn-dagger.
This version adds Sequential Monte-Carlo, a.k.a. particle filters-based inference to the library.
Here's the full changelog:
- Dependency:
prbnmcn-stats.0.0.3
->prbnmcn-stats.0.0.4
- Add beta distribution to Gsl samplers
- Refactor Cps monad
- Add SMC inference
- Simplify handler type, modularize effect definitions away from Cps_monad
- Fix typo: bernouilli -> bernoulli (report by @nilsbecker)
I also wrote the following article: Applying Sequential Monte-Carlo to time series forecasting It contains some use cases for the library, I hope some find it fun :)
To conclude this post, and as a partial answer to @gasche 's question in an older thread, I believe that unlike some other inference techniques, single-shot continuations are enough to implement SMC. Without getting into the details, the implementation is very reminiscent of that of lightweight threading libraries. I look forward to experiment with a fibre-based implementation!
MirageOS 4.0
Thomas Gazagnaire announced
On behalf of the MirageOS team, I am delighted to announce the release of MirageOS 4.0.0! I'd like to send special thanks to @dinosaure and @Lortex who drove that release forward for multiple years.
Since the first release of 2013, MirageOS has made steady progress toward deploying a self-managed internet infrastructure. The project’s initial aim was to self-host as many services as possible aimed at empowering internet users to securely deploy infrastructure to own their data and take back control of their privacy. MirageOS can securely deploy static website hosting with “Let’s Encrypt” certificate provisioning and a secure SMTPstack with security extensions. MirageOS can also deploy decentralised communication infrastructure like Matrix, OpenVPN servers, and TLS tunnels to ensure data privacy or DNS(SEC) servers for better authentication.
The protocol ecosystem now contains hundreds of libraries and services millions of daily users. Over these years, major commercial users have joined the projects. They rely on MirageOS libraries to keep their products secure. For instance, the MirageOS networking code powers Docker Desktop’s VPNKit, which serves the traffic of millions of containers daily. Citrix Hypervisor uses MirageOS to interact with Xen, the hypervisor that powers most of today’s public cloud. Nitrokey is developing a new hardware security module based on MirageOS. Robur develops a unikernel orchestration system for fleets of MirageOS unikernels. Tarides uses MirageOS to improve the Tezos blockchain, and Hyper uses MirageOS to build sensor analytics and an automation platform for sustainable agriculture.
In the coming weeks, our blog will feature in-depth technical content for the new features that MirageOS brings, as well as a tour of the existing community and commercial users of MirageOS. Please reach out if you’d like to tell us about your story.
Install MirageOS 4
The easiest way to install MirageOS 4 is by using the opam version 2.1 and `ocaml>=4.12.1`. Follow the installation guide for more details.
$ opam update $ opam install 'mirage>4'
Note: if you upgrade from MirageOS 3 you will need to manually clean the previous generated files (or call mirage
clean
before upgrading). You would also want to read the full list of API
changes. You can see unikernel examples in
mirage/mirage-skeleton,
roburio/unikernels or
tarides/unikernels.
About MirageOS
MirageOS is a library operating system that constructs unikernels for secure, high-performance, low-energy footprint applications across various hypervisor and embedded platforms. It is available as an open-source project created and maintained by the MirageOS Core Team. A unikernel can be customised based on the target architecture by picking the relevant MirageOS libraries and compiling them into a standalone operating system, which contains strictly the functionality necessary for the target. This minimises the unikernel’s footprint, increasing the security of the deployed operating system.
The MirageOS architecture can be divided into operating system libraries, typed signatures, and a metaprogramming compiler. The operating system libraries implement various functionalities, ranging from low-level network card drivers, to full reimplementations of the TLS protocol, as well as the Git protocol to store versioned data. A set of typed signatures ensures that the OS libraries are consistent and work well in conjunction with each other. Most importantly, MirageOS is also a metaprogramming compiler that can input OCaml source code along with its dependencies, and a deployment target description in order to generate an executable unikernel, i.e., a specialised binary artefact containing only the code needed to run on the target platform. Overall, MirageOS focuses on providing a small, well-defined, typed interface with the system components of the target architecture.
Read the full announcement on mirage.io's blog.
Anil Madhavapeddy then added
For those curious about what some of the MirageOS libraries are, there is a raw Yaml list over at mirage/mirage-repositories listing most of them. Conversion of this Yaml to HTML for the main mirage.io website would be a welcome contribution! :slight_smile:
OCaml 4.14.0 is released
octachron announced
The OCaml team has the pleasure of celebrating the birthday of Alexander Grothendieck by announcing the release of OCaml version 4.14.0.
Some of the highlights in the 4.14.0 release are:
- Integrated support for "go to definitions" in Merlin.
- Standard library: new modules
In_channel
andOut_channel
, many new functions in Seq module, UTF decoding and validation support for strings and bytes. - Runtime optimisation: GC prefetching. Benchmarks show a speedup of around 20% in GC-heavy programs.
- Improved error messages in particular for module-level error.
- Deprecated functions and modules in preparation for OCaml 5. In particular, the Stream and Genlex modules are now deprecated.
Type variables can be explicitly introduced in value and variant constructor declarations. For instance,
val fold: ('acc -> 'elt -> 'acc) -> 'acc -> 'elt list -> 'acc type showable = Show: 'a * ('a -> string) -> showable
can now be written as
val fold: 'acc 'elt. ('acc -> 'elt -> 'acc) -> 'acc -> 'elt list -> 'acc type showable = Show: 'a. 'a * ('a -> string) -> showable
- Tail-call with up to 64 arguments are now guaranteed to be optimized for all architectures.
- Experimental tail modulo cons (TMC) transformation
The full list of changes can be found in the changelog below. (editor’s note: please follow the archive link for the full changelog)
Those releases are available as OPAM switches, and as a source download here:
ocaml-in-python.0.1.0: Effortless Python bindings for OCaml modules
Thierry Martinez announced
I am happy to announce the first release of ocaml-in-python
: this is a Python package that exposes all OCaml
modules as Python libraries, generating bindings on the fly. This can be seen as a dual of
pyml_bindgen
:
pyml_bindgen
binds Python libraries in OCaml, while ocaml-in-python
binds OCaml modules in Python.
It is available from GitHub or via opam
: opam install
ocaml-in-python
Requirements: OCaml
>= 4.13, Python
>= 3.7.
Once installed via opam
, the package should be registered in the Python environment:
either by registering the package with
pip
using the following command (requires Python >=3.8):pip install --editable "`opam var ocaml-in-python:lib`"
or by adding the following definition to the environment:
export PYTHONPATH="`opam var share`/python/:$PYTHONPATH"
Then, we can import ocaml
in Python and use OCaml modules:
Python 3.10.0 (default, Nov 10 2021, 19:16:14) [GCC 7.5.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import ocaml >>> print(ocaml.List.map((lambda x : x + 1), [1, 2, 3])) [2;3;4]
We can for instance compile an OCaml module on the fly from Python.
>>> m = ocaml.compile('let hello x = Printf.printf "Hello, %s!\n%!" x') >>> m.hello('world') Hello, world!
And we can require and use packages via findlib
.
>>> ocaml.require("parmap") >>> from ocaml import Parmap >>> print(Parmap.parmap( ... (lambda x : x + 1), Parmap.A([1, 2, 3]), ncores=2)) [2;3;4]
Details about the conversions are given in
README.md
.
Happy hacking!
Old CWN
If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a look at the archive or the RSS feed of the archives.
If you also wish to receive it every week by mail, you may subscribe online.