Hello Here is the latest OCaml Weekly News, for the week of February 22 to March 01, 2022. Table of Contents ───────────────── data-encoding.0.5 release Tutorial: Roguelike with effect handlers For Diversity and the OCaml Community: Outreachy Summer 2022 Bogue, the OCaml GUI Friday 03/04 Intern presentations – open attendance! Affect: Composable concurrency primitives for OCaml 5.0 Segfault Systems Joins Tarides OCaml User Survey 2022 Old CWN data-encoding.0.5 release ═════════════════════════ Archive: Raphaël Proust announced ──────────────────────── On behalf of [Nomadic Labs], I'm happy to announce the release of data-encoding version 0.5. This new version brings several bug fixes, some increased test coverage, minor improvements in the API, and a major new feature: [Nomadic Labs] Compact encodings: sub-byte tag sizes ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ This new version provides a new set of combinators for _compact_ encodings. These compact encodings will handle all the verbose and error-prone bit-twidling process needed to combine multiple sub-byte discriminators into a single byte-size one. E.g., the encoding `let e1 = either (either bool unit) (option bool)' uses three bits in the shared tag and zero bytes after that; the encoding `let e2 = either int32 int64' uses one bit in the shared tag and either 4 or 8 bytes to represent the integer; the product encoding `let ee = tup2 e1 e2' uses four (3 + 1) bits in the shared tag and either 4 or 8 bytes to represent the integer of `e2'. How to get ╌╌╌╌╌╌╌╌╌╌ The code is available under MIT license on . It can be installed via `opam'. Dario Teixeira asked and Raphaël Proust replied ─────────────────────────────────────────────── Hi @raphael-proust! I have a question regarding the connection between `data-encoding' and `json-data-encoding', also developed at Nomadic Labs. The latter seems tied to JSON, whereas the former is more flexible, supporting also binary encodings. However, since `data-encoding' also supports JSON, doesn't it subsume `json-data-encoding' completely? The `data-encoding' library uses `json-data-encoding' for its JSON backend. It delegates conversion from OCaml values into and from JSON to the primitives provided in the interface of `json-data-encoding'. In a way, yes, as an end-user you don't need to use `json-data-encoding' directly because you can use the `Json' module of `data-encoding' instead. There are three possible reasons why you might add `json-data-encoding' as a (non-transitive) dependency to your project and use it directly in your code: • You want to keep the dependency set and the number of abstraction layers as small as possible. E.g., in order to reduce binary size. • You want some static guarantees that some encodings are only every used for JSON. E.g., in your logging system. • You need to define a JSON encoding which is rejected by `data-encoding' on grounds that it is invalid in binary. Note that • This is very specific to some combinators but basically some combinators will reject their inputs (raise `Invalid_argument') because using the serialiser would lead to undecodable data. Most typically, this happens if you try to concatenate two fields of unknown length. Decoding the result becomes a guessing game as to were one field stops and where the next begins. These could easily be represented as an array in JSON which includes all the delimiters you need to decode it. • There are other workarounds (e.g., prefixing the fields with a length field), but going for the JSON encoding directly is a valid approach if you only need JSON. Raphaël Proust later announced ────────────────────────────── Version 0.5.1 of the data-encoding has just been released. This is a bugfix release making one of the library's internal checks more permissive. Without this fix (i.e., using version 0.5), some valid encodings are rejected (raising `Invalid_argument') by the library. You can update via opam: `opam install data-encoding.0.5.1' Tutorial: Roguelike with effect handlers ════════════════════════════════════════ Archive: art-w announced ─────────────── The recent conversations about [`eio' 0.1] and [agnostic blocking] have made me very curious about effect handlers. The multicore team has done an [awesome] job with their [tutorials], [examples] and [talks], but the laymen have been too quiet for such an exciting feature! Where are all the blog posts about how "you could have invented algebraic effects" and "one-shot continuations are like spaghetti"? In any case, I'm hoping to tease some of you into trying them out with [a simple tutorial about programming a roguelike with effect handlers] :) There's nothing new here besides the fun use-case! So if you already have an intuitive understanding of the syntax and motivations, you may be more interested by [a deeper look at the scope of effect handlers] – and a soft introduction to some less common features of the type system. /(this link was previously posted deep into the `eio' thread)/ I would be grateful if you spot any mistake! I'm also curious of other fun applications for effect handlers… and if you feel like sharing your own surprises and discoveries, I believe it could really help others learn faster :) [`eio' 0.1] [agnostic blocking] [awesome] [tutorials] [examples] [talks] [a simple tutorial about programming a roguelike with effect handlers] [a deeper look at the scope of effect handlers] Kiran Gopinathan then said ────────────────────────── Great blog post! That seems like a very elegant implementation! Funny you should make a rougelike :smiley: , I guess effect handlers + games might be popular for games, because I also had a blog post about effect handlers and their applications, in particular for games, although in my case it was for animations: gasche also replied ─────────────────── Note: the "upstream" status of effect handlers is a little uncertain/confusing right now. Your blog post (I didn't get a chance to read it yet, but it sounds very nice!) uses the experimental syntax of multicore-4.12+effects, but that syntax was intentionally /not/ upstreamed, and it will /not/ be part of OCaml 5.0. I think there is a risk of confusion because the community is aware that Multicore OCaml has effect handlers, and also that Multicore OCaml has been merged upstream. So it can be tempting to believe that the upcoming OCaml release (or maybe one or two releases after that, we said the first Multicore release would be more like a preview) will support effect handlers as a language feature. It will not! Effects as a language feature were removed from Multicore OCaml before the upstream merge. And /no one knows/ if/when they will be supported upstream. So: I think that your blog posts on using effect handlers could have somewhere a short mention that the code is using an experimental extension of OCaml that is not supported by the upstream implementation. The reasoning for this choice is that we want to give a chance to a type system for effect handlers, but that still need quite a bit more time than the Multicore runtime itself. We don't want to encourage the ecosystem to rely on untyped effects, if it means a lot of pain upgrading to typed effects later (or risk having to support both). 5.0 only contains basic support for effect handlers as a /runtime primitive/, but dos /not/ support handlers as a /language feature/. I think they should be considered experimental: you can rely on them for their intended purpose of exposing a flexible interface for concurrent fibers, but uses beyond that may break in the future. So, in a sense, we don't want people to use them. It's of course fine to use experimental features from experimental forks of the OCaml compiler (effect handlers, modular implicits or explicits, runtime type representations and what not), and the people working on these experimental features do benefit from other people trying them and giving them feedback. But we don't want people to depend on it /in production/, whatever that means. (For example, code using it is likely to get stuck on 4.12 forever and never see an upgrade to upcoming OCaml versions, although of course people could choose to port the experimental branch forward.) For Diversity and the OCaml Community: Outreachy Summer 2022 ════════════════════════════════════════════════════════════ Archive: Sonja Heinze announced ────────────────────── Just in case anyone is actually interested in this: the project submission deadline has been extended from March 4th to March 23rd. So the updated timeline now looks as follows: where 2. and 3. probably need to be done a bit in parallel. Bogue, the OCaml GUI ════════════════════ Archive: sanette announced ───────────────── Hi, some new developments. I have implemented a new `Sdl_area' widget where one can conveniently issue any SDL function (from the SDL Renderer API). Here is (below) the new 'labelled graph' example. In this example I am using regular "label" widgets for creating the nodes, and I am using an Sdl_area for drawing the lines. The nice things for labels to be regular widgets is that one can click on them. To demonstrate this, in this example they react to a click by jumping to another random location (with animation). ┌──── │ open Bogue │ module W = Widget │ module L = Layout │ │ let n = 15 (* number of discs *) │ let radius = 20 │ let width = 800 │ let height = 600 │ │ let c = Draw.find_color "#e5b92c" │ let cb = Draw.find_color "#7b6b35" │ let disc_style = Style.( │ create ~border:( │ mk_border ~radius (mk_line ~color:Draw.(opaque c) ~width:1 ~style:Solid ())) │ ~background:(color_bg Draw.(opaque cb)) ()) │ │ let background = L.style_bg Style.( │ of_bg (gradient ~angle:45. Draw.[opaque grey; opaque black])) │ │ let fg = Draw.(opaque white) │ │ let create_disc i (x,y) = │ let w = 2*radius + 1 in │ let bg = Box.create ~style:disc_style ~width:w ~height:w () in │ W.label ~fg (string_of_int i) │ |> L.resident ~background:(L.box_bg bg) ~x:(x-radius) ~y:(y-radius) ~w ~h:w │ │ let move_disc (x,y) d = │ let (x0, y0) = L.xpos d, L.ypos d in │ L.animate_x d (Avar.fromto x0 x); │ L.animate_y d (Avar.fromto y0 y) │ │ let random_center _ = │ radius + Random.int (width - 2*radius), │ radius + Random.int (height - 2*radius) │ │ let area = │ let sdlw = W.sdl_area ~w:width ~h:height () in │ let sdla = W.get_sdl_area sdlw in │ let centers = Array.init n random_center in │ let color = Draw.(opaque grey) in │ let draw_lines renderer = let open Draw in │ for i = 0 to n - 2 do │ let x0, y0 = to_pixels centers.(i) in │ let x1, y1 = to_pixels centers.(i+1) in │ line renderer ~color ~thick:6 ~x0 ~y0 ~x1 ~y1 │ done in │ Sdl_area.add sdla draw_lines; │ let discs = Array.mapi create_disc centers |> Array.to_list in │ (* move the disc when click on it *) │ List.iteri (fun i d -> │ W.on_click ~click:(fun _ -> │ centers.(i) <- random_center 0; │ Sdl_area.update sdla; │ let x,y = centers.(i) in │ move_disc (x - radius, y - radius) d) (L.widget d)) │ discs; │ L.superpose ~w:width ~h:height ~background (L.resident sdlw :: discs) │ │ let board = Bogue.make [] [area] │ │ let () = Bogue.run board └──── Friday 03/04 Intern presentations – open attendance! ════════════════════════════════════════════════════ Archive: Aya announced ───────────── This is Aya, one of the three [Outreachy] interns working on OCaml this winter :camel: After 3 very fast months, our internships are already coming to a close. We have had such a great time working on our projects and learning OCaml that we want to hold an event to mark the end of the internships, and we decided to open it up to the community :tada: As you might have seen in the [initial announcement], @pitag @shonfeder @gs0510 @tmattio and @pkel all volunteered to mentor us from December 2021 to now. Thank you all so so much for mentoring us and introducing us to OCaml :heart: :fire: It's been such an enjoyable experience! We are inviting anyone who is interested to attend a virtual session of 3 short presentations on *Friday, March 4th, 4-5pm CET* (we will post the link to join on Thursday). There will be time for Q&A after each presentation, and the whole session will be recorded and posted online shortly after as well. • @ayc9 will present on updating a standard PPX deriver (mentors: @pitag @shonfeder) • @SaySayo will present on syntax highlighting and other updates to the vscode extension (mentors: @tmattio @gs0510) • @JiaeK will present on building a basic monitoring dashboard for [ocaml.org] (mentors: @tmattio) We hope you can make it! -@ayc9 @SaySayo @JiaeK [Outreachy] [initial announcement] [ocaml.org] Affect: Composable concurrency primitives for OCaml 5.0 ═══════════════════════════════════════════════════════ Archive: Daniel Bünzli announced ─────────────────────── I looked a bit into the kind of fiber abstraction and concurrency structure I would like to use with the new tools OCaml 5.0 is going to offer. You can find some results in affect's [`Fiber'] module. This fiber abstraction supports terminating by returning values or abnormally (by aborting or via a spurious exception). Termination of a fiber is aligned on function scopes: all the fibers spawn by a fiber function have to terminate in order for it to terminate. This means that if your fiber returns a value it waits for its spawns to terminate (in any way) before returning the value. And if your fiber returns abnormally (uncaught eception or explicit abort) it first aborts all its non-terminated spawns before returning abnormally – this provides affect's notion of cancellation. Explicit fiber aborts raise the `Abort' exception in fibers. Combined with a disciplined use of `Fun.protect' and an optional `finally' handler specified at fiber spawn, this lets them release the ressources they may hold when it's time to say goodbye. The module also provides a generic way of blocking and unblocking fibers that you can use to interface with your favourite event loop. It does so without requiring to fiddle with effects, you just need to make judicious use of [`Fiber.block'] and provide a suitable function to `Fiber.run''s built-in scheduler to let it know about fibers that can be unblocked. A grab bag of comments: 1. The first goal of affect is to seek a concurrency and abort structure that are easy to understand, use and compose with event loops. Right now some efficiency and implementation aspects need to be improved. This will likely change the exposed set of primitive effects which doesn't feel exactly right yet (if you want to build your own scheduler). 2. I use abort rather than cancel terminology. From my non-native english speaker perspective, cancelling is more about not doing something that was planned but didn't happen yet. Aborting is more about stopping something that is going on. It also melds better with the uncaught exception case. 3. Say no to `unit' soups! Let fibers return values. 4. At that point I don't feel the need to add a promise/future abstraction to the toolbox. The whole point of direct style is to get rid of this async madness. 5. There's no synchronisation structure yet. Semaphores are always useful for throttling so I'll certainly add that at some point or a more fundamental primitive like an mvar. 6. The [`Funix'] module has a few fiber friendly `Unix' module functions for playing with timers and the network, see [`ping.ml'] for an example of use. In practice you want to be able to use something else than `select(2)' though. There are various ways one could go about this, see for example point 6. in these [design notes]. 7. The [`mouse.ml'] has a basic example on how to interface with the SDL event loop which provides another example on how one goes to interface `Fiber' with event loops. I'm not fully convinced by everything yet. It will certainly need one or two more design rounds. If you try it, feel free to comment or make suggestions on the issue tracker. Home page: API docs: (or `odig doc affect') Install: ┌──── │ opam switch create 5.0.0+trunk │ opam pin add https://erratique.ch/repos/affect.git └──── [`Fiber'] [`Fiber.block'] [`Funix'] [`ping.ml'] [design notes] [`mouse.ml'] Segfault Systems Joins Tarides ══════════════════════════════ Archive: Thomas Gazagnaire announced ─────────────────────────── @kayceesrk and I are delighted to announce that Segfault Systems, a spinout from IIT-Madras, is joining Tarides. Tarides has worked closely with Segfault Systems over the last couple of years, most notably on the award-winning Multicore OCaml project and the upstreaming plans for OCaml 5.0. This alliance furthers the goals of Tarides, bringing the compiler and benchmarking expertise of the Segfault team directly into the Tarides organisation, where it can be commercially funded and supported. All of Segfault Systems’ existing responsibilities and open-source commitments will migrate over to Tarides, where work will continue towards the three main objectives in 2022: • Releasing OCaml 5.0 with support for domains and effect handlers • Supporting the ecosystem to migrate the OCaml community over to OCaml 5.0 • Improving developer productivity for OCaml 5.0 by releasing the best platform tools This alliance will complement the commercial offerings of Tarides – already strengthened by the integration of [OCaml Labs] – and contribute to Tarides’ mission: empowering developers, communities, and organisations to adopt OCaml as their primary programming experience by providing training, expertise, and development services around the OCaml language. Read the full announcement [here], including details of our goals and the focus for 2022. This alliance brings the headcount of Tarides up to 60+ people, all working towards making OCaml the best language for any and every project. Join our team and reach out for commercial services at [https://tarides.com/]. [OCaml Labs] [here] [https://tarides.com/] OCaml User Survey 2022 ══════════════════════ Archive: Kim Nguyễn announced ──────────────────── we are delighted to announce the [OCaml User Survey 2022]. With this survey, the OCSF is trying to get a better picture of the OCaml community and its needs. It would be very helpful if you could take a few minutes (10 to 15) to fill the survey and share it with other OCaml programmers. [https://forms.gle/oKy2Joz1cZhCPNtf6] The survey is run by the [OCaml Software Foundation]. It builds on [the previous iteration] issued in 2020. The results will be published here on discuss and on the [website of the OCSF]. We would like to particularly thank @cjr for his help as well as everyone who commented on the previous survey. We tried our best to take all remarks into account but surely missed something. Don't hesitate to give us your feedback (you can post here or send me a message/email). The survey will remain opened until March 11th 2022 (AOE). [OCaml User Survey 2022] [https://forms.gle/oKy2Joz1cZhCPNtf6] [OCaml Software Foundation] [the previous iteration] [website of the OCSF] Old CWN ═══════ If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] [the archive] [RSS feed of the archives] [online] [Alan Schmitt]