[Caml-list] Attn: Development Editor, Latest OCaml Weekly News

[Alan Schmitt <alan.schmitt@polytechnique.org>]

[-- Attachment #1: Type: text/plain, Size: 14486 bytes --]

Hello

Here is the latest OCaml Weekly News, for the week of September 22 to
29, 2020.

Table of Contents
─────────────────

Opam-repository: security and data integrity posture
jsonoo 0.1.0
Interesting OCaml Articles
Rehabilitating Packs using Functors and Recursivity
the OCaml Software Foundation
dual 0.1.0
Old CWN


Opam-repository: security and data integrity posture
════════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/opam-repository-security-and-data-integrity-posture/6478/1>


Chas Emerick said, spawning a huge thread
─────────────────────────────────────────

  In connection with [another thread] discussing the fact that
  Bitbucket's closure of mercurial support had affected the availability
  of around 60+ projects' published versions, I learned of a number of
  facts about how the opam repository is arranged, and how it is managed
  that are concerning.

  In summary, it seems that opam / opam-repository:

  1. Never retains "published" artifacts, only links to them as provided
     by library authors.
  2. Allows very weak hashes (even md5).
  3. Allows authors to _update_ artifact URLs and hashes of previously
     "published" versions.
  4. Offers scant support for individually signing artifacts or
     metadata.

  All of these are quite dangerous. As a point of reference, the
  ecosystems I am most familiar with using prior to OCaml (JVM and
  Javascript) each had very serious documented failures and exploits
  (and many many more quiet ones) until their respective package
  managers (Maven Central et al., and npm) plugged the above
  vulnerabilities that opam-repository suffers from.

  To make things concrete, without plugging the above (and especially
  items 1-3):

  • the availability and integrity of published libraries can be
    impacted by third-party hosting services changing or going offline
    (as in the case of the Bitbucket closure)
  • the integrity of libraries can be impacted by authors
    non-maliciously publishing updates to already-released versions,
    affecting functionality, platform compatibility, build
    reproducibility, or all of the above (anecdotes of which were shared
    with me when talking about this issue earlier today)
  • the integrity of libraries can be impacted by malicious authors
    publishing updates to already-released versions
  • the integrity of libraries can be impacted by malicious non-authors
    changing the contents at tarball URLs to include changed code that
    could e.g. exfiltrate sensitive data from within the organizations
    that use those libraries. This is definitely the nuclear nightmare
    scenario, and unfortunately opam is wide open to it thanks to
    artifacts not being retained authoritatively and [essential
    community libraries] continuing to use md5 in 2020.

  Seeing that this has been well-established policy for years was
  honestly quite shocking (again, in comparison to other languages'
  package managers that have had these problems licked for a very very
  long time). I understand that opam and its repository probably have
  human-decades of work put into them, and that these topics have been
  discussed here and there (in somewhat piecemeal fashion AFAICT), so
  I'm certain I have not found (nevermind read) all of the prior art,
  but I thought it reasonable to open a thread to gauge what the
  projects' posture is in general.


[another thread]
<https://discuss.ocaml.org/t/bitbucket-stopped-supporting-mercurial-repositories/6324/3?u=cemerick>

[essential community libraries]
<https://github.com/ocaml/opam-repository/blob/master/packages/core/core.v0.14.0/opam>


Hannes Mehnert replied
──────────────────────

  first of all thanks for your post raising this issue, which is
  important for me as well.

  I've been evaluating and working on improving the security of the
  opam-repository over the years, including to not use `curl –insecure`
  (i.e. properly validate TLS certificates) - the WIP result is [conex],
  which aims at cryptographically signed community repositories without
  single points of failures (threshold signatures for delegations,
  built-in key rollover, …) - feel free to read the blog posts or OCaml
  meeting presentations. Unfortunately it still has not enough traction
  to be deployed and mandatory for the main opam repository. Without
  cryptopgraphic signatures (and an established public key
  infrastructure), the hashes used in opam-repository and opam are more
  checksums (i.e. integrity protection) than for security. Threat models
  - I recommend to read section [1.5.2 "goals to protect against
  specific attacks"] - that's what conex above is based on and attempts
  to mitigate. I'll most likely spend some time on improving conex over
  the next year, and finally deploying it on non-toy repositories.

  In the meantime, what you're mentioning:
  1. "Never retains 'published' artifacts" <- this is not true, the
     opam.ocaml.org host serves as an artifact cache, and is used by
     opam when you use the default repository. This also means that the
     checksums and the tarballs are usually taken from the same host ->
     the one who has access there may change anything arbitrarily for
     all opam users.
  2. "Weak hashes" <- this is true, I'd appreciate if (a) opam would
     warn (configurable to error out) if a package which uses weak
     checksum algorithms, and (b) Camelus or some other CI step would
     warn when md5/sha1 are used
  3. "Authors can modify URLs and hashes" <- sometimes (when a
     repository is renamed or moved on GitHub) the GitHub auto-generated
     tarball has a different checksum. I'd appreciate to, instead of
     updating that meta-data in the opam-repository to add new
     patch-versions (1.2.3-1 etc.) with the new URL & hash - there could
     as well be a CI job / Camelus check what is allowed to be modified
     in an edit of a package (I think with the current state of the
     opam-repository, "adding upper bounds" on dependencies needs to be
     allowed, but not really anything else).
  4. I'm not sure I understand what you mean - is it about cryptographic
     signatures and setting up a public key infrastructure?


[conex] <https://github.com/hannesm/conex>

[1.5.2 "goals to protect against specific attacks"]
<https://github.com/theupdateframework/specification/blob/master/tuf-spec.md#the-update-framework-specification>


Anton Kochkov said
──────────────────

  Closely related issue is
  <https://discuss.ocaml.org/t/how-to-setup-local-opam-mirror/4423>,
  since the integrity checks and verification will become even more
  important if there will be multiple mirrors in the future.


jsonoo 0.1.0
════════════

  Archive: <https://discuss.ocaml.org/t/ann-jsonoo-0-1-0/6480/1>


Max LANTAS announced
────────────────────

  Hello! I am announcing the first release of `jsonoo', a JSON library
  for Js_of_ocaml.

  <https://github.com/mnxn/jsonoo>
  <https://opam.ocaml.org/packages/jsonoo>

  This library provides a very similar API to the excellent BuckleScript
  library, [bs-json] by [glennsl]. Unlike bs-json, this port of the
  library tries to follow OCaml naming conventions and be easier to
  interface with other OCaml types like `Hashtbl.t' . This library
  passes a nearly equivalent test suite.

  This project is part of ongoing work to port [vscode-ocaml-platform]
  to Js_of_ocaml.

  Generated documentation can be found [here].


[bs-json] <https://github.com/glennsl/bs-json>

[glennsl] <https://github.com/glennsl>

[vscode-ocaml-platform]
<https://github.com/ocamllabs/vscode-ocaml-platform>

[here] <https://mnxn.github.io/jsonoo/jsonoo/Jsonoo/index.html>


Interesting OCaml Articles
══════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/interesting-ocaml-articles/1867/62>


Ryan Slade announced
────────────────────

  <https://blog.darklang.com/fizzboom-benchmark/>


Rehabilitating Packs using Functors and Recursivity
═══════════════════════════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/rehabilitating-packs-using-functors-and-recursivity/6497/1>


OCamlPro announced
──────────────────

  Our new blogpost is about the redemption of packs in the OCaml
  ecosystem. This first part shows our work to generate functor units
  and functor packs : [Rehabilitating Packs using Functors and
  Recursivity, part 1.]

        Packs in the OCaml ecosystem are kind of an outdated
        concept (options `-pack' and `-for-pack' the OCaml manual,
        and their main utility has been overtaken by the
        introduction of module aliases in OCaml 4.02. What if we
        tried to redeem them and give them a new youth and utility
        by adding the possibility to generate functors or
        recursive packs?

        This blog post covers the functor units and functor packs,
        while the next one will be centered around recursive
        packs. Both RFCs are currently developed by JaneStreet and
        OCamlPro. This idea was initially introduced by functor
        packs (Fabrice Le Fessant) and later generalized by
        functorized namespaces (Pierrick Couderc /et al/.).


[Rehabilitating Packs using Functors and Recursivity, part 1.]
<https://www.ocamlpro.com/2020/09/24/rehabilitating-packs-using-functors-and-recursivity-part-1/>


the OCaml Software Foundation
═════════════════════════════

  Archive:
  <https://discuss.ocaml.org/t/ann-the-ocaml-software-foundation/4476/19>


gasche announced
────────────────

  We were all very busy during the last semester, and have been mostly
  quiet on the foundation activities, but of course our actions were
  running in the background. Some highlights:

  • Kate @kit-ty-kate Deplaix has worked on opam-repository QA for the
    OCaml 4.11 release, and the work and results are just as superb as
    for 4.10. We will fund Kate to work again on the upcoming 4.12
    release.

  • We are funding ongoing maintenance work on [ocaml-rs] (a port of the
    OCaml FFI library from C to Rust) by its author and maintainer, Zach
    @zshipko Shipko. Zach did a big round of cleanup changes this
    summer, improving the overall design of the library and completing
    its feature set.

  • We are funding @JohnWhitington (the author of [OCaml from the Very
    Beginning]) to do some technical writing work for OCaml
    documentation. His contributions so far have been very diverse, from
    a script to harmonize the documentation of List and ListLabels (and
    Array and ArrayLabels, etc.) in the standard library, to small
    cleanups and improvement to ocaml.org web pages. One focus of his
    work is the upcoming documentation page "Up and running with OCaml",
    taking complete newcomers through the basic setup, using the
    toplevel and building and running a Hello World. ([ocaml.org#1165],
    [rendered current state])

  • Two [Outreachy] internships were supervised this summer, focusing on
    the compiler codebase. Florian @Octachron Angeletti (INRIA)
    supervised an intern on adding a JSON format for some compiler
    messages (we expect PRs to be submitted soon). Vincent @vlaviron
    Laviron and Guillaume @zozozo Bury (OCamlPro) supervised an intern
    on reducing mutable state in the internal implementation.

  • Inspired by [this Discuss thread], we are funding experimental work
    by @sanette on the HTML rendering of the OCaml manual. This work is
    in the process of being reviewed for upstreaming in the OCaml
    compiler distribution. ([#9755].) This is a better end-result than I
    had initially expected.

  (We also had a couple non-highlights. For example, we funded a sprint
  (physical development meeting) for the [Owl] contributors, with
  Marcello @mseri Seri doing all the organization work; it was planned
  for the end of March, and had to be postponed due to the pandemic.)


[ocaml-rs] <https://github.com/zshipko/ocaml-rs/>

[OCaml from the Very Beginning] <http://ocaml-book.com/>

[ocaml.org#1165] <https://github.com/ocaml/ocaml.org/pull/1165>

[rendered current state]
<https://github.com/johnwhitington/ocaml.org/blob/up-and-running/site/learn/tutorials/up_and_running.md>

[Outreachy] <https://outreachy.org>

[this Discuss thread]
<https://discuss.ocaml.org/t/suggestions-for-ocaml-documentation/4504>

[#9755] <https://github.com/ocaml/ocaml/pull/9755>

[Owl] <https://github.com/owlbarn>


dual 0.1.0
══════════

  Archive: <https://discuss.ocaml.org/t/ann-dual-0-1-0/6512/1>


Jason Nielsen announced
───────────────────────

  I’ve released [dual] which is now up on opam.  It is a dual numbers
  library which includes a one dimensional root finder (via Newton's
  method).


[dual] <https://github.com/drjdn/ocaml_dual>


Old CWN
═══════

  If you happen to miss a CWN, you can [send me a message] and I'll mail
  it to you, or go take a look at [the archive] or the [RSS feed of the
  archives].

  If you also wish to receive it every week by mail, you may subscribe
  [online].

  [Alan Schmitt]


[send me a message] <mailto:alan.schmitt@polytechnique.org>

[the archive] <http://alan.petitepomme.net/cwn/>

[RSS feed of the archives] <http://alan.petitepomme.net/cwn/cwn.rss>

[online] <http://lists.idyll.org/listinfo/caml-news-weekly/>

[Alan Schmitt] <http://alan.petitepomme.net/>


[-- Attachment #2: Type: text/html, Size: 26248 bytes --]