From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by c5ff346549e7 (Postfix) with ESMTPS id F29FC5D5 for ; Tue, 29 Sep 2020 07:02:54 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.77,317,1596492000"; d="scan'208,217";a="469976127" Received: from prod-listesu18.inria.fr (HELO sympa.inria.fr) ([128.93.162.160]) by mail2-relais-roc.national.inria.fr with ESMTP; 29 Sep 2020 09:02:50 +0200 Received: by sympa.inria.fr (Postfix, from userid 20132) id 474C1E0B4B; Tue, 29 Sep 2020 09:02:50 +0200 (CEST) Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id B6AE2E00E5 for ; Tue, 29 Sep 2020 09:02:42 +0200 (CEST) Authentication-Results: mail2-smtp-roc.national.inria.fr; spf=Pass smtp.pra=alan.schmitt@polytechnique.org; spf=Pass smtp.mailfrom=SRS0=mcYN=DG=polytechnique.org=alan.schmitt@bounces.m4x.org; spf=Pass smtp.helo=postmaster@mx1.polytechnique.org IronPort-PHdr: =?us-ascii?q?9a23=3ALPi8lxcgXvcWKTGul0RQIs6glGMj4u6mDksu8pMi?= =?us-ascii?q?zoh2WeGdxc25ZhKN2/xhgRfzUJnB7Loc0qyK6v+mAzNLuM/c+Fk5M7V0Hycfjs?= =?us-ascii?q?sXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aFRrwLxd6?= =?us-ascii?q?KfroEYDOkcu3y/qy+5rOaAlUmTaxe7x/IAi0oAnLqMUbg4RuJrssxhbJv3BFZ/?= =?us-ascii?q?lYyWR0KFyJgh3y/N2w/Jlt8yRRv/Iu6ctNWrjkcqo7ULJVEi0oP3g668P3uxbD?= =?us-ascii?q?SxCP5mYHXWUNjhVIGQnF4wrkUZr3ryD3q/By2CiePc3xULA0RTGv5LplRRP0lC?= =?us-ascii?q?sKMSMy/2bKhMxtl6JbuAyuqABjw4DaZ4GVMeBxfqLbfdgHQWZMUcJcWylHD4+8?= =?us-ascii?q?bIUPCfQBPedDr4n8vFQOqQWxDhSiBePo0D9Hm2T21rA+3+kvDQ3K2QotFM8Mvn?= =?us-ascii?q?vJttX4LKccX/6owqfGzjvMYO9Z1Czn54TUaB0su+2AUa5yfMfX1EIhFxnFjlKV?= =?us-ascii?q?qYH9Iz6V0v4Cs26G5OR9Se2vi2snqwBtojiz28whjZPGhoYPxVDC7yl525o6Jd?= =?us-ascii?q?29SE56fd6kDIBdtzmdN4tyQsIiX39ntzo6yr0AuJ67ZTUKx4o9yx7YcfyHfJGF?= =?us-ascii?q?7xT+X+mePTl2nmhqeK6jhxms60igzPXxW9W33VpWoCRIltfCum0C2hHX9sSKV/?= =?us-ascii?q?9w80i91DqS1w3e6fxILFwomaTUNpIvwrA9moQNvEnNACL4lln6gayUe0gi5+Om?= =?us-ascii?q?5ePnYrD8qZ+dMY95khn+Pboymsy+HeQ3LBAOX2+e+eS5yrLv50v5T6tWjvEula?= =?us-ascii?q?nWrIrVJcEfpqKjBA9VyIkj5w6wDzenzNQYnWQHI0lfdB2biIjpPknCIP/5Dfej?= =?us-ascii?q?g1SsjSxky+rHPr3mGpnCM2LMkK3ifbZ58UFczgUzwcpD6JJTD7ENOPXzVVPru9?= =?us-ascii?q?zdCh85Kxa0w+H9BNph0YMeXHqDAqiFP6zItF+I4vsjLPKLZI8SuzbxMeQq5/nr?= =?us-ascii?q?jXMhnl8dYLWp3YEJZ3+iAvtmI0WYbWDrgtcbHmcGpgs+Q/HqiV2GVT5ffXGyX7?= =?us-ascii?q?gz5jw9FYmoDp/DS5iwjLCf2Cq3BIBaanxJB1yWH3rka5+IVvkDZS6KP8NsnCEI?= =?us-ascii?q?WaK/R4Ih2hyirhL2x6Z9IubJ+CAUqZTu38Vv6eLJjxE97zl0Atyd026TS2F0mX?= =?us-ascii?q?sFRzo53axiu0B90lCD0ax4gvxEC9Nc+/NJUgE7NZ7F0ux1Fcr+WgXbfteGUFqm?= =?us-ascii?q?Q9OmDi8tTt8p3tMCfUJwF8+/ghzf0CemGbEYm6CRCJE6/a/Qx33xKNx8y3bC2q?= =?us-ascii?q?khlV4mQs5XOGO7mqBx6hTfCpbMk0qFlamkbbwR0iDM+mqb1WqOu0VYUQ5sUarb?= =?us-ascii?q?QX8fZk3WrdXg5kPfUbCiE7MnMhFOycOaMKRKbsfmjVNcSPf4JNveY2exm2asBR?= =?us-ascii?q?aU3b6Dd43qe3gb3CrBFkcEiBof/XOJOAkxHCuhpHjeDDN2GVL1f0zs6fV+qG+8?= =?us-ascii?q?TkIs0w6FdUhh176s9h4RhPycUO8T06kfuCYhrjV0BEyy08jXC9qGvQphfb9Tbc?= =?us-ascii?q?kz4FddhirlsFk3OoOmZeg2gkEYWwBouQXo2gkhTs0Kmtcs5jtimAFtL4qc0Uhd?= =?us-ascii?q?bHWZ0YH0PvvQMGakuFikYqvSn1Xfy8q++6EV6f1+pU+n9AquE1IK93R8z8IT1G?= =?us-ascii?q?GW54nDCAQVQdT8TxUZ7R9/8pjeay913IjU0HxwLeHguzvL3ZQyD+sgywq8V89Y?= =?us-ascii?q?NLKYGQTyFcwDGsXoL/YlzQv6JikYNfxfofZnd/itcOGLjfH6ZbsyrHedlW1Cpb?= =?us-ascii?q?tF/AeM+i57ELWa2oZchemf2hqbWjz8ila4r82xnppLN2hLTziPjBP8DYsUXZVc?= =?us-ascii?q?OIMCCGOgOcqyn4osjZnwXXVV7ximW0NA39WmK0PLMw7NmDZI3EFSmkSJ3DOixm?= =?us-ascii?q?UlwSkuqruD0SfOxeX7aRdBPXREFjBv?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AfAQCs2nJflyIeaIFgHgEBCxIMgzJSR?= =?us-ascii?q?gFeVjIshD2JAotWAoVBjWGFZxABAwENGAEOBQECBAEBgVaCMYJ0Ah4GAQUzEwI?= =?us-ascii?q?QAQEFAQEBAgEDAwQBEwEBAQEBCBYGhV8HJgyCNwwMAwODJQEYCQQGZSMDFAEGA?= =?us-ascii?q?wIRARcBFAoXARIUBkyCQAGCfAUKmVube38zhDsBFgEOCYQXgUINAhOBFoVOS4M?= =?us-ascii?q?DCIQKD4FNP4NzbIJFDAsBAQEBARaBJAEBgzeCYASQHgUik0SSMBRLKgeCaoEOB?= =?us-ascii?q?AuGXYEBiweGd4MNgSiOEY5NIZJogXiGCAd3CIFlhgiKboRVgUEqgWcMBzMaMEO?= =?us-ascii?q?CaQlgDYRUiVYBARaDToJkgXU7hUQ/MwIBAQkFJQIGAQkBAQMJdQEBBRMLAYsBL?= =?us-ascii?q?YIXAQE?= X-IPAS-Result: =?us-ascii?q?A0AfAQCs2nJflyIeaIFgHgEBCxIMgzJSRgFeVjIshD2JAot?= =?us-ascii?q?WAoVBjWGFZxABAwENGAEOBQECBAEBgVaCMYJ0Ah4GAQUzEwIQAQEFAQEBAgEDA?= =?us-ascii?q?wQBEwEBAQEBCBYGhV8HJgyCNwwMAwODJQEYCQQGZSMDFAEGAwIRARcBFAoXARI?= =?us-ascii?q?UBkyCQAGCfAUKmVube38zhDsBFgEOCYQXgUINAhOBFoVOS4MDCIQKD4FNP4Nzb?= =?us-ascii?q?IJFDAsBAQEBARaBJAEBgzeCYASQHgUik0SSMBRLKgeCaoEOBAuGXYEBiweGd4M?= =?us-ascii?q?NgSiOEY5NIZJogXiGCAd3CIFlhgiKboRVgUEqgWcMBzMaMEOCaQlgDYRUiVYBA?= =?us-ascii?q?RaDToJkgXU7hUQ/MwIBAQkFJQIGAQkBAQMJdQEBBRMLAYsBLYIXAQE?= X-IronPort-AV: E=Sophos;i="5.77,317,1596492000"; d="scan'208,217";a="469976078" X-MGA-submission: =?us-ascii?q?MDGAiY5n/27dtEsI7Rjz3fIWh2ciQ+X1p40J8w?= =?us-ascii?q?CL98ndSd5X5RiZ7XPr+h7cvTdtCdhAnqhWlDgScsvpEXI6L9cWqNEMHb?= =?us-ascii?q?sVQICq+UPzLg0GOK/lKPFpmSMYL1PQKmluDrHrXDBtgirwkMNvUiq5fL?= =?us-ascii?q?oMwAlePFQ+mdoeFuZ1b88J+A=3D=3D?= Received: from mx1.polytechnique.org ([129.104.30.34]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Sep 2020 09:02:40 +0200 Received: from set (set.irisa.fr [131.254.10.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ssl.polytechnique.org (Postfix) with ESMTPSA id 403C9564F04; Tue, 29 Sep 2020 09:02:38 +0200 (CEST) From: Alan Schmitt To: "lwn" , "cwn" , caml-list@inria.fr, comp@lists.orbitalfox.eu Date: Tue, 29 Sep 2020 09:02:38 +0200 Message-ID: <878sct9ib5.fsf@m4x.org> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=-=-=" X-AV-Checked: ClamAV using ClamSMTP at svoboda.polytechnique.org (Tue Sep 29 09:02:39 2020 +0200 (CEST)) X-Spam-Flag: No, tests=bogofilter, spamicity=0.000000, queueID=ED0C4564F05 X-Org-Mail: alan.schmitt.1995@polytechnique.org Subject: [Caml-list] Attn: Development Editor, Latest OCaml Weekly News Reply-To: Alan Schmitt X-Loop: caml-list@inria.fr X-Sequence: 18251 Errors-To: caml-list-owner@inria.fr Precedence: list Precedence: bulk Sender: caml-list-request@inria.fr X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Here is the latest OCaml Weekly News, for the week of September 22 to 29, 2020. Table of Contents =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80 Opam-repository: security and data integrity posture jsonoo 0.1.0 Interesting OCaml Articles Rehabilitating Packs using Functors and Recursivity the OCaml Software Foundation dual 0.1.0 Old CWN Opam-repository: security and data integrity posture =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90=E2=95=90 Archive: Chas Emerick said, spawning a huge thread =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 In connection with [another thread] discussing the fact that Bitbucket's closure of mercurial support had affected the availability of around 60+ projects' published versions, I learned of a number of facts about how the opam repository is arranged, and how it is managed that are concerning. In summary, it seems that opam / opam-repository: 1. Never retains "published" artifacts, only links to them as provided by library authors. 2. Allows very weak hashes (even md5). 3. Allows authors to _update_ artifact URLs and hashes of previously "published" versions. 4. Offers scant support for individually signing artifacts or metadata. All of these are quite dangerous. As a point of reference, the ecosystems I am most familiar with using prior to OCaml (JVM and Javascript) each had very serious documented failures and exploits (and many many more quiet ones) until their respective package managers (Maven Central et al., and npm) plugged the above vulnerabilities that opam-repository suffers from. To make things concrete, without plugging the above (and especially items 1-3): =E2=80=A2 the availability and integrity of published libraries can be impacted by third-party hosting services changing or going offline (as in the case of the Bitbucket closure) =E2=80=A2 the integrity of libraries can be impacted by authors non-maliciously publishing updates to already-released versions, affecting functionality, platform compatibility, build reproducibility, or all of the above (anecdotes of which were shared with me when talking about this issue earlier today) =E2=80=A2 the integrity of libraries can be impacted by malicious authors publishing updates to already-released versions =E2=80=A2 the integrity of libraries can be impacted by malicious non-aut= hors changing the contents at tarball URLs to include changed code that could e.g. exfiltrate sensitive data from within the organizations that use those libraries. This is definitely the nuclear nightmare scenario, and unfortunately opam is wide open to it thanks to artifacts not being retained authoritatively and [essential community libraries] continuing to use md5 in 2020. Seeing that this has been well-established policy for years was honestly quite shocking (again, in comparison to other languages' package managers that have had these problems licked for a very very long time). I understand that opam and its repository probably have human-decades of work put into them, and that these topics have been discussed here and there (in somewhat piecemeal fashion AFAICT), so I'm certain I have not found (nevermind read) all of the prior art, but I thought it reasonable to open a thread to gauge what the projects' posture is in general. [another thread] [essential community libraries] Hannes Mehnert replied =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 first of all thanks for your post raising this issue, which is important for me as well. I've been evaluating and working on improving the security of the opam-repository over the years, including to not use `curl =E2=80=93insec= ure` (i.e. properly validate TLS certificates) - the WIP result is [conex], which aims at cryptographically signed community repositories without single points of failures (threshold signatures for delegations, built-in key rollover, =E2=80=A6) - feel free to read the blog posts or O= Caml meeting presentations. Unfortunately it still has not enough traction to be deployed and mandatory for the main opam repository. Without cryptopgraphic signatures (and an established public key infrastructure), the hashes used in opam-repository and opam are more checksums (i.e. integrity protection) than for security. Threat models - I recommend to read section [1.5.2 "goals to protect against specific attacks"] - that's what conex above is based on and attempts to mitigate. I'll most likely spend some time on improving conex over the next year, and finally deploying it on non-toy repositories. In the meantime, what you're mentioning: 1. "Never retains 'published' artifacts" <- this is not true, the opam.ocaml.org host serves as an artifact cache, and is used by opam when you use the default repository. This also means that the checksums and the tarballs are usually taken from the same host -> the one who has access there may change anything arbitrarily for all opam users. 2. "Weak hashes" <- this is true, I'd appreciate if (a) opam would warn (configurable to error out) if a package which uses weak checksum algorithms, and (b) Camelus or some other CI step would warn when md5/sha1 are used 3. "Authors can modify URLs and hashes" <- sometimes (when a repository is renamed or moved on GitHub) the GitHub auto-generated tarball has a different checksum. I'd appreciate to, instead of updating that meta-data in the opam-repository to add new patch-versions (1.2.3-1 etc.) with the new URL & hash - there could as well be a CI job / Camelus check what is allowed to be modified in an edit of a package (I think with the current state of the opam-repository, "adding upper bounds" on dependencies needs to be allowed, but not really anything else). 4. I'm not sure I understand what you mean - is it about cryptographic signatures and setting up a public key infrastructure? [conex] [1.5.2 "goals to protect against specific attacks"] Anton Kochkov said =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80 Closely related issue is , since the integrity checks and verification will become even more important if there will be multiple mirrors in the future. jsonoo 0.1.0 =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90 Archive: Max LANTAS announced =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80 Hello! I am announcing the first release of `jsonoo', a JSON library for Js_of_ocaml. This library provides a very similar API to the excellent BuckleScript library, [bs-json] by [glennsl]. Unlike bs-json, this port of the library tries to follow OCaml naming conventions and be easier to interface with other OCaml types like `Hashtbl.t' . This library passes a nearly equivalent test suite. This project is part of ongoing work to port [vscode-ocaml-platform] to Js_of_ocaml. Generated documentation can be found [here]. [bs-json] [glennsl] [vscode-ocaml-platform] [here] Interesting OCaml Articles =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90 Archive: Ryan Slade announced =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80 Rehabilitating Packs using Functors and Recursivity =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90 Archive: OCamlPro announced =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80 Our new blogpost is about the redemption of packs in the OCaml ecosystem. This first part shows our work to generate functor units and functor packs : [Rehabilitating Packs using Functors and Recursivity, part 1.] Packs in the OCaml ecosystem are kind of an outdated concept (options `-pack' and `-for-pack' the OCaml manual, and their main utility has been overtaken by the introduction of module aliases in OCaml 4.02. What if we tried to redeem them and give them a new youth and utility by adding the possibility to generate functors or recursive packs? This blog post covers the functor units and functor packs, while the next one will be centered around recursive packs. Both RFCs are currently developed by JaneStreet and OCamlPro. This idea was initially introduced by functor packs (Fabrice Le Fessant) and later generalized by functorized namespaces (Pierrick Couderc /et al/.). [Rehabilitating Packs using Functors and Recursivity, part 1.] the OCaml Software Foundation =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95= =90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90= =E2=95=90=E2=95=90=E2=95=90=E2=95=90 Archive: gasche announced =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 We were all very busy during the last semester, and have been mostly quiet on the foundation activities, but of course our actions were running in the background. Some highlights: =E2=80=A2 Kate @kit-ty-kate Deplaix has worked on opam-repository QA for = the OCaml 4.11 release, and the work and results are just as superb as for 4.10. We will fund Kate to work again on the upcoming 4.12 release. =E2=80=A2 We are funding ongoing maintenance work on [ocaml-rs] (a port o= f the OCaml FFI library from C to Rust) by its author and maintainer, Zach @zshipko Shipko. Zach did a big round of cleanup changes this summer, improving the overall design of the library and completing its feature set. =E2=80=A2 We are funding @JohnWhitington (the author of [OCaml from the V= ery Beginning]) to do some technical writing work for OCaml documentation. His contributions so far have been very diverse, from a script to harmonize the documentation of List and ListLabels (and Array and ArrayLabels, etc.) in the standard library, to small cleanups and improvement to ocaml.org web pages. One focus of his work is the upcoming documentation page "Up and running with OCaml", taking complete newcomers through the basic setup, using the toplevel and building and running a Hello World. ([ocaml.org#1165], [rendered current state]) =E2=80=A2 Two [Outreachy] internships were supervised this summer, focusi= ng on the compiler codebase. Florian @Octachron Angeletti (INRIA) supervised an intern on adding a JSON format for some compiler messages (we expect PRs to be submitted soon). Vincent @vlaviron Laviron and Guillaume @zozozo Bury (OCamlPro) supervised an intern on reducing mutable state in the internal implementation. =E2=80=A2 Inspired by [this Discuss thread], we are funding experimental = work by @sanette on the HTML rendering of the OCaml manual. This work is in the process of being reviewed for upstreaming in the OCaml compiler distribution. ([#9755].) This is a better end-result than I had initially expected. (We also had a couple non-highlights. For example, we funded a sprint (physical development meeting) for the [Owl] contributors, with Marcello @mseri Seri doing all the organization work; it was planned for the end of March, and had to be postponed due to the pandemic.) [ocaml-rs] [OCaml from the Very Beginning] [ocaml.org#1165] [rendered current state] [Outreachy] [this Discuss thread] [#9755] [Owl] dual 0.1.0 =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2= =95=90=E2=95=90 Archive: Jason Nielsen announced =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80 I=E2=80=99ve released [dual] which is now up on opam. It is a dual numbe= rs library which includes a one dimensional root finder (via Newton's method). [dual] Old CWN =E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90=E2=95=90 If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] [the archive] [RSS feed of the archives] [online] [Alan Schmitt] --=-=-= Content-Type: text/html; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable OCaml Weekly News

OCaml Weekly News

Previous Week Up Next Week

Hello

Here is the latest OCaml Weekly News, for the week of September 22 to 29, 2= 020.

Opam-repository: security and data integrity posture

Archive: https://discuss.ocaml.org/t/opam-repositor= y-security-and-data-integrity-posture/6478/1

Chas Emerick said, spawning a huge thread

In connection with another thread discussing the fact that Bitbucket's closure of mercurial support had affec= ted the availability of around 60+ projects' published versions, I learned of a number of facts abo= ut how the opam repository is arranged, and how it is managed that are concerning.

In summary, it seems that opam / opam-repository:

  1. Never retains "published" artifacts, only links to them as provided by = library authors.
  2. Allows very weak hashes (even md5).
  3. Allows authors to update artifact URLs= and hashes of previously "published" versions.
  4. Offers scant support for individually signing artifacts or metadata.

All of these are quite dangerous. As a point of reference, the ecosystems I= am most familiar with using prior to OCaml (JVM and Javascript) each had very serious documented failur= es and exploits (and many many more quiet ones) until their respective package managers (Maven Centra= l et al., and npm) plugged the above vulnerabilities that opam-repository suffers from.

To make things concrete, without plugging the above (and especially items 1= -3):

  • the availability and integrity of published libraries can be impacted b= y third-party hosting services changing or going offline (as in the case of= the Bitbucket closure)
  • the integrity of libraries can be impacted by authors non-maliciously p= ublishing updates to already-released versions, affecting functionality, pl= atform compatibility, build reproducibility, or all of the above (anecdotes= of which were shared with me when talking about this issue earlier today)<= /li>
  • the integrity of libraries can be impacted by malicious authors publish= ing updates to already-released versions
  • the integrity of libraries can be impacted by malicious non-authors cha= nging the contents at tarball URLs to include changed code that could e.g. = exfiltrate sensitive data from within the organizations that use those libr= aries. This is definitely the nuclear nightmare scenario, and unfortunately= opam is wide open to it thanks to artifacts not being retained authoritati= vely and essential community libraries continuing= to use md5 in 2020.

Seeing that this has been well-established policy for years was honestly qu= ite shocking (again, in comparison to other languages' package managers that have had these problem= s licked for a very very long time). I understand that opam and its repository probably have human-d= ecades of work put into them, and that these topics have been discussed here and there (in somewhat= piecemeal fashion AFAICT), so I'm certain I have not found (nevermind read) all of the prior art, but = I thought it reasonable to open a thread to gauge what the projects' posture is in general.

Hannes Mehnert replied

first of all thanks for your post raising this issue, which is important fo= r me as well.

I've been evaluating and working on improving the security of the opam-repo= sitory over the years, including to not use `curl –insecure` (i.e. properly validate TLS ce= rtificates) - the WIP result is conex, which aims at crypt= ographically signed community repositories without single points of failures (threshold signatures for de= legations, built-in key rollover, …) - feel free to read the blog posts or OCaml meeting pre= sentations. Unfortunately it still has not enough traction to be deployed and mandatory for the main opa= m repository. Without cryptopgraphic signatures (and an established public key infrastructure), t= he hashes used in opam-repository and opam are more checksums (i.e. integrity protection) tha= n for security. Threat models - I recommend to read section 1.5.2 "goals to protect against specific attacks" - that's wh= at conex above is based on and attempts to mitigate. I'll most likely spend= some time on improving conex over the next year, and finally deploying it on non-toy rep= ositories.

In the meantime, what you're mentioning:

  1. "Never retains 'published' artifacts" <- this is not true, the opam.= ocaml.org host serves as an artifact cache, and is used by opam when you us= e the default repository. This also means that the checksums and the tarbal= ls are usually taken from the same host -> the one who has access there = may change anything arbitrarily for all opam users.
  2. "Weak hashes" <- this is true, I'd appreciate if (a) opam would warn= (configurable to error out) if a package which uses weak checksum algorith= ms, and (b) Camelus or some other CI step would warn when md5/sha1 are used=
  3. "Authors can modify URLs and hashes" <- sometimes (when a repository= is renamed or moved on GitHub) the GitHub auto-generated tarball has a dif= ferent checksum. I'd appreciate to, instead of updating that meta-data in t= he opam-repository to add new patch-versions (1.2.3-1 etc.) with the new UR= L & hash - there could as well be a CI job / Camelus check what is allo= wed to be modified in an edit of a package (I think with the current state = of the opam-repository, "adding upper bounds" on dependencies needs to be a= llowed, but not really anything else).
  4. I'm not sure I understand what you mean - is it about cryptographic sig= natures and setting up a public key infrastructure?

Anton Kochkov said

Closely related issue is https://discuss.ocaml.org/t/how-to-setup-local-op= am-mirror/4423, since the integrity checks and verification will become even more important if there = will be multiple mirrors in the future.

jsonoo 0.1.0

Archive: ht= tps://discuss.ocaml.org/t/ann-jsonoo-0-1-0/6480/1

Max LANTAS announced

Hello! I am announcing the first release of jsonoo, a JSON lib= rary for Js_of_ocaml.

https://github.com/mnxn/jsonoo https://opam.ocaml.org/p= ackages/jsonoo

This library provides a very similar API to the excellent BuckleScript libr= ary, bs-json by glennsl. Unlike bs-json, this port of the library tries to follow OCaml naming conventions and be ea= sier to interface with other OCaml types like Hashtbl.t . This library passes a nearly equi= valent test suite.

This project is part of ongoing work to port vscode-ocaml= -platform to Js_of_ocaml.

Generated documentation can be found here.

Rehabilitating Packs using Functors and Recursivity

Archive: https://discuss.ocaml.org/t/rehabilitating= -packs-using-functors-and-recursivity/6497/1

OCamlPro announced

Our new blogpost is about the redemption of packs in the OCaml ecosystem. T= his first part shows our work to generate functor units and functor packs : Rehabilitating Packs using Functors and Recursivity, part 1.

Packs in the OCaml ecosystem are kind of an outdated concept (options -pack and -for-pack the OCaml manual, and their main utility has been overtaken by the introduction= of module aliases in OCaml 4.02. What if we tried to redeem them and give them a new youth and u= tility by adding the possibility to generate functors or recursive packs?

This blog post covers the functor units and functor packs, while the next o= ne will be centered around recursive packs. Both RFCs are currently developed by JaneStreet and OCamlP= ro. This idea was initially introduced by functor packs (Fabrice Le Fessant) and later genera= lized by functorized namespaces (Pierrick Couderc et al.).

the OCaml Software Foundation

Archive: https://discuss.ocaml.org/t/ann-the-ocaml-software-foundati= on/4476/19

gasche announced

We were all very busy during the last semester, and have been mostly quiet = on the foundation activities, but of course our actions were running in the background. Some = highlights:

  • Kate @kit-ty-kate Deplaix has worked on opam-repository QA for the OCam= l 4.11 release, and the work and results are just as superb as for 4.10. We= will fund Kate to work again on the upcoming 4.12 release.
  • We are funding ongoing maintenance work on ocaml-rs (a port of the OCaml FFI library from C t= o Rust) by its author and maintainer, Zach @zshipko Shipko. Zach did a big = round of cleanup changes this summer, improving the overall design of the l= ibrary and completing its feature set.
  • We are funding @JohnWhitington (the author of OCaml from the Very Beginning) to do some technical writing w= ork for OCaml documentation. His contributions so far have been very divers= e, from a script to harmonize the documentation of List and ListLabels (and= Array and ArrayLabels, etc.) in the standard library, to small cleanups an= d improvement to ocaml.org web pages. One focus of his work is the upcoming= documentation page "Up and running with OCaml", taking complete newcomers = through the basic setup, using the toplevel and building and running a Hell= o World. (ocaml.or= g#1165, rendered current state<= /a>)
  • Two Outreachy internships were su= pervised this summer, focusing on the compiler codebase. Florian @Octachron= Angeletti (INRIA) supervised an intern on adding a JSON format for some co= mpiler messages (we expect PRs to be submitted soon). Vincent @vlaviron Lav= iron and Guillaume @zozozo Bury (OCamlPro) supervised an intern on reducing= mutable state in the internal implementation.
  • Inspired by this Discuss thread, we are funding experimental = work by @sanette on the HTML rendering of the OCaml manual. This work is in= the process of being reviewed for upstreaming in the OCaml compiler distri= bution. (#9755.) T= his is a better end-result than I had initially expected.

(We also had a couple non-highlights. For example, we funded a sprint (phys= ical development meeting) for the Owl contributors, with M= arcello @mseri Seri doing all the organization work; it was planned for the end of March, and had to be postp= oned due to the pandemic.)

Old CWN

If you happen to miss a CWN, you can send me a message and I'll mail it to you, or go take a loo= k at the archive or the RSS feed of the archives<= /a>.

If you also wish to receive it every week by mail, you may subscribe online.

Alan Schmitt

--=-=-=--