From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 11381 invoked from network); 6 Feb 2021 01:56:13 -0000 Received: from minnie.tuhs.org (45.79.103.53) by inbox.vuxu.org with ESMTPUTF8; 6 Feb 2021 01:56:13 -0000 Received: by minnie.tuhs.org (Postfix, from userid 112) id 427CE9C2E4; Sat, 6 Feb 2021 11:56:06 +1000 (AEST) Received: from minnie.tuhs.org (localhost [127.0.0.1]) by minnie.tuhs.org (Postfix) with ESMTP id 390A29BA45; Sat, 6 Feb 2021 11:55:46 +1000 (AEST) Authentication-Results: minnie.tuhs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=iitbombay-org.20150623.gappssmtp.com header.i=@iitbombay-org.20150623.gappssmtp.com header.b="QUGPzFK1"; dkim-atps=neutral Received: by minnie.tuhs.org (Postfix, from userid 112) id 912D09BA45; Sat, 6 Feb 2021 11:55:43 +1000 (AEST) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) by minnie.tuhs.org (Postfix) with ESMTPS id 547029BA3F for ; Sat, 6 Feb 2021 11:55:42 +1000 (AEST) Received: by mail-ot1-f51.google.com with SMTP id r21so288333otk.13 for ; Fri, 05 Feb 2021 17:55:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/Qx7RKpw+LhDeyxArEX+1g/bxxLSm9DxgBiqy/mpmI8=; b=QUGPzFK10w4v67oJaJcd9t4YJlHcAzzzExs49WaBzqVmTUNwCqeqfeGtbFIFsZc5zX bPZO92hkSzSUDmTmDOKop/Nn/e8jeskolYkg9MUjQ+MMe4Me1n/v/P+vR/6FrfNuHRHT 40FM0er3qdscP/FWUv2UtE7AYC1qgFoHhte7vJQpNAayXxQ15+ylOyYhubRBgCkIt89z sqLSBLpygFl1W24rKlqVK5F8rHk9MJN5qYmMuYjdvSLSDeWXRMVtk7dsgGFyKXjDMgSJ EEbUO9RSua9xlxJA16oS8z4p2adCsF4Kj7543+zdHxCebkpXoHQzi+gHZai6uykmujgP Xujg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=/Qx7RKpw+LhDeyxArEX+1g/bxxLSm9DxgBiqy/mpmI8=; b=eT+3lrfQbxF1SYEVX5prW70hG62gpwCWQGUJmxT/s7E3VbTG3l/mTqFBx48iKuVedp gew73EOGFh1w+a5QSbdOr05Ay0SA+t10yQtZfKPyM124QnfSlbaFBkYTdTaZxwfVWQIj o6sMY4Okrm1K93mUWr0uLge5JRJXOZcnJVi9ZiaOn4fROE18p/PwYrjq2yHhRXiagoNE FeNskLaGTo7UqS7uFPKzuG8fNvUWrbUqIPLAagr/dlM8okQKTjDitKfDnNyLdPVd2KgZ e+zAatMqHKTl8frytTkUAVpHBTDwNsV2bBIRsLa150Pph9obsIiCrXIfXD+E+3Htomc/ gECw== X-Gm-Message-State: AOAM530hRN/zkbun4Hhw8qnGt8sQVjezq50TCsSFPAh/jDaDTz14VQIc /pkQLxB0w8mmW2Fc9CET0fwLcmyyomi0SOjkZxY= X-Google-Smtp-Source: ABdhPJyWQyZZjqqDIb4yt7Gs4AuP0B4IAWHnmVo2H7lObVn0Pb9anY5NnRNoBtHcAw3Uk+67GUzEFw== X-Received: by 2002:a9d:640b:: with SMTP id h11mr5282577otl.224.1612576541417; Fri, 05 Feb 2021 17:55:41 -0800 (PST) Received: from unknowna45e60f0be01.attlocal.net (172-125-77-130.lightspeed.sntcca.sbcglobal.net. [172.125.77.130]) by smtp.gmail.com with ESMTPSA id e6sm1264445otl.63.2021.02.05.17.55.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Feb 2021 17:55:40 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) From: Bakul Shah In-Reply-To: <31039.1612574314@hop.toad.com> Date: Fri, 5 Feb 2021 17:55:38 -0800 Content-Transfer-Encoding: 7bit Message-Id: References: <20210205003315.GK13701@mcvoy.com> <26D923CF-5319-4207-BA28-6EFA0E3BB1F8@iitbombay.org> <20210205141820.GO13701@mcvoy.com> <31039.1612574314@hop.toad.com> To: John Gilmore X-Mailer: Apple Mail (2.3654.60.0.2.21) Subject: Re: [TUHS] FreeBSD behind the times? (was: Favorite unix design principles?) X-BeenThere: tuhs@minnie.tuhs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: The Unix Heritage Society mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: TUHS main list Errors-To: tuhs-bounces@minnie.tuhs.org Sender: "TUHS" Please see my followup message. My fault for mixing two separate things (what a user should not do vs how the kernel can still provide coherence). > On Feb 5, 2021, at 5:18 PM, John Gilmore wrote: > > On Thu, Feb 04, 2021 at 09:17:54PM -0800, Bakul Shah wrote: >> Write(2)ing to a mapped page sounds pretty dodgy. Likely to get you >> in trouble in any case. Similarly read(2)ing. > > Uh, no. You misunderstand completely. > > The purpose of the kernel is to provide a reliable interface to system > facilities, that lets processes NOT DEPEND on what other processes are > doing. > > The decision about whether Tool X uses mmap() versus read() to access a > file, or mmap() versus write() to change one, is a decision that DOES > NOT DEPEND on what Tool Y is doing. Tools X and Y may have been written > by different groups in different decades. Tool X may have been written > to use stdio, which used read(). Three years later, stdio got rewritten > to use mmap() for speed, but that's invisible to the author of Tool X. > And maybe an end user in 2025 decides to use both Tool X and Tool Y on > the same file. So only much later will any malign interactions between > read/write and mmap actually be noticed by end users. And the fix is > not to create new dependencies between Tool X, stdio, and Tool Y. It is > to fix the kernel so they do not depend on each other! > > Here is a real-life example from my own experience. > > There is a long-standing bug in the Linux kernel, in which the inotify() > system call simply didn't work on nested file systems. This caused a > long-standing bug in Ubuntu, which I reported in 2012 here: > > https://bugs.launchpad.net/ubuntu/+source/rpcbind/+bug/977847 > > The symptom was that after booting from a LiveCD image, "apt-get > install" for system services (in my case an NFS client package) wouldn't > work. Turned out the system startup scripts used inotify() to notice > and start newly installed system services. The root cause was that > inotify failed because the root file system was an "overlayfs" that > overlaid a RAMdisk on top of the read-only LiveCD file system. The > people who implemented "overlayfs" didn't think inotify() was important, > or they thought it would be too much work to make it actually meet its > specs, so they just made it ignore changes to the files in the overlaid > file system. So the startup daemon's inotify() would never report the > creation of new files about the new services, because those files were > in the overlaying RAM disk, and so it would not start them and the user > would notice the error. > > The underlying overlayfs bug was reported in 2011 here: > > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/882147 > > As far as I know it has never been fixed. (The bug report was > closed in 2019 for one of the usual bogus reasons.) > > The problem came because real tools (like systemd, or the tail command) > actually started using inotify, assuming that as a well documented > kernel interface, it would actually meet its specs. And because a > completely unrelated other real tool (like the LiveCD installer) > actually started using overlayfs, assuming that as a well documented > kernel interface, it too would actually meet its specs. And then one > day somebody tried to use both those tools together and they failed. > > That's why telling people "Don't use mmap() on the same file that you > use read() on" is an invalid attitude for a Real Kernel Maintainer. > Props to Larry McVoy for caring about this. Boos to the Linux > maintainers of overlayfs who didn't give a shit. > > John >