From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <9front-bounces@9front.inri.net> X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: from 9front.inri.net (9front.inri.net [168.235.81.73]) by inbox.vuxu.org (Postfix) with ESMTP id E7C2F21725 for ; Wed, 8 May 2024 22:38:23 +0200 (CEST) Received: from mail-ay.bbox.fr ([194.158.98.9]) by 9front; Wed May 8 16:35:40 -0400 2024 Received: from cauchy.polynum.local (213-44-244-59.abo.bbox.fr [213.44.244.59]) by mail-ay.bbox.fr (Postfix) with ESMTP id 4EDFF56 for <9front@9front.org>; Wed, 8 May 2024 22:35:33 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.11.0 mail-ay.bbox.fr 4EDFF56 Received: from cauchy.polynum.local (localhost [127.0.0.1]) by cauchy.polynum.local (8.16.1/8.15.2) with ESMTP id 448KYHpY029487 for <9front@9front.org>; Wed, 8 May 2024 22:34:17 +0200 (CEST) Received: (from tlaronde@localhost) by cauchy.polynum.local (8.16.1/8.14.9/Submit) id 448KYHR0014089 for 9front@9front.org; Wed, 8 May 2024 22:34:17 +0200 (CEST) X-Authentication-Warning: cauchy.polynum.local: tlaronde set sender to laronde.thierry@bbox.fr using -f Date: Wed, 8 May 2024 22:34:17 +0200 From: tlaronde@kergis.com To: 9front@9front.org Message-ID: References: <328dc131-044d-408d-a040-512a44ae6e7b@fjrhome.net> <9df183e7-7a94-4d58-9a68-2dbc0e73018f@posixcafe.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-VADE-SPAMSTATE: clean X-VADE-SPAMSCORE: 0 X-VADE-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvledrvdeftddgudeglecutefuodetggdotefrodftvfcurfhrohhfihhlvgemuceuqfgfjgfifgfgufdpucfqfgfvpdcuggftfghnshhusghstghrihgsvgenuceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtredttddtvdenucfhrhhomhepthhlrghrohhnuggvsehkvghrghhishdrtghomhenucggtffrrghtthgvrhhnpeeiveefieevhfekudduveduueefvdeltdeufeeggeehvddthffhhedvkefhkefhteenucffohhmrghinhepkhgvrhhgihhsrdgtohhmnecukfhppedvudefrdeggedrvdeggedrheelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepvddufedrgeegrddvgeegrdehledphhgvlhhopegtrghutghhhidrphholhihnhhumhdrlhhotggrlhdpmhgrihhlfhhrohhmpehlrghrohhnuggvrdhthhhivghrrhihsegssghogidrfhhrpdhnsggprhgtphhtthhopedupdhrtghpthhtoheplehfrhhonhhtseelfhhrohhnthdrohhrgh List-ID: <9front.9front.org> List-Help: X-Glyph: ➈ X-Bullshit: secure virtualized polling table persistence interface Subject: Re: [9front] Enabling a service Reply-To: 9front@9front.org Precedence: bulk On Wed, May 08, 2024 at 09:46:43PM +0200, Roberto E. Vargas Caballero wrote: > On Wed, May 08, 2024 at 11:10:00AM -0500, Jacob Moody wrote: > > > No matter how you cut you are going to have to deal with multiple people merging > > their cache in to the single root of truth with potentially latent updates. > > Either you have to serialize all mutations at the source of truth (and at that point > > you are latent bound) or you have to be clever about merging. A lot of ink has been > > spilled about this problem in the scope of web programming (CRDs iirc), perhaps > > that may serve as some inspiration. > > I would suggest to take a look to AFS (Andrew File System, that I think had also > an evoultion but I don't remember the name) and how it works, and the implications. > Sometimes you can merge the changes, and other times you have to request I bought, long ago, a book about the AFS system. But it was no explanation about how things were done, but only the overall organization and how to administrate it. To be honest, the conclusion was: for the "big" picture, it was simply what I would have thought almost at first. But there were no indication of what supplementary features pop up when thinking about filesystems considering a WORM, registering file content history by blocks, but adding another layer for tentative writes (I call them "minutes") for files that are tentative files and that have to be backed-up without history and differently, and a special command to "commit" the file, that is to indicate that this version has to be saved. All in all several combined levels: 1) A rock solid low level WORM filesystem, with deduplication of blocks and history, with a canonical name for a file and a way to retrieve a specific version of the file; 2) A mechanism providing a way of scripting a policy allowing to create a namespace and allowing to pile storages (being anywhere: local or remote), with the upper level being a local cached (and back-upped with no history) of the current version, commiting being a special command saying that this version is to be saved in a versioning WORM; 3) One server (that could simply be special instances of 2) ) to contact to centralize requests for sharing, knowing where the files are in whatever version, with scriptable policy about the delay for requiring the canonical version in order to save it in one or more back-upped WORM systems a changed file, with the ability to reconstruct information from local caches when a version of the file is missing in the authoritative servers, and the possibility to redirect a request to another local cache (if allowed by administration) even if the canonical version has not been for the moment saved in an authoritative server. But all this (except perhaps for the "tentative" layer that is not exactly a "/tmp") exists more or less. The problem is to try to make the thing reliable and efficient. I wonder too if the aliases of the file should be in the low level filesystem (like hard links), or if the filesystem should have a canonical name (and only one), and the user aliases for a same content (several user names for the same content; or several user names for different versions of the same content) should be a separate feature, the filesystem keeping only one pair (some uniq id + some canonical utf-8 userlevel name---its birthname). Plan9 has already some chunks of all this. And AFS, if I read correctly, was not a panacea and, if I'm not mistaken, is declining or is even more or less orphaned. The WORM and deduplication of blocks was, for me, a feature discovered with Plan9. And I still think it's a great idea and a basis for VCS. And it seems not many have got it right in this area---from what I read, it was supposed to exist with ZFS but it does not widely exist in the implementations and I seem to recall that even when it is existing, its use is discouraged. -- Thierry Laronde http://www.kergis.com/ http://kertex.kergis.com/ Key fingerprint = 0FF7 E906 FBAF FE95 FD89 250D 52B1 AE95 6006 F40C