From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from tb-mx1.topicbox.com (localhost.local [127.0.0.1]) by tb-mx1.topicbox.com (Postfix) with ESMTP id 318E211C8E22 for <9fans@9fans.net>; Fri, 10 Jan 2020 14:25:01 -0500 (EST) (envelope-from mycroftiv@sphericalharmony.com) Received: from tb-mx1.topicbox.com (localhost [127.0.0.1]) by tb-mx1.topicbox.com (Authentication Milter) with ESMTP id F824CB0341F; Fri, 10 Jan 2020 14:25:01 -0500 ARC-Seal: i=1; a=rsa-sha256; cv=none; d=topicbox.com; s=arcseal; t= 1578684300; b=iZaFcINk6GuLCWhFvP7jfWkIS/dTUzxWMwEVX5hNiZbHCpYOtX yhKvab6AMDaYO09hjcMTnPlNBoMQZ0kypq+UT0VtzJv+/k8TPuwImeDQCOPyk2o8 vln5eSak97eXTlz9sZ0fCq57xr5uyih6WyDjxbphVl/08/hgbAGbD1Kcm2BQsF9V tMR+xhJIjyh1T8uCP6WehsQ5+NQZvFfdmMJTaY9sYOTRrfMajujvu2hNwewMcc/S mJ0SABUCLYeptlgqO1OzGhjJEwjVvHCaDdh4Gh760u5Wtjt+mr/qLGBdnh27THl+ WY/45Jd7wkW2d0XIkKXz+KihUFPiB90+D9xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d= topicbox.com; h=from:message-id:date:to:subject:mime-version :content-type:content-transfer-encoding; s=arcseal; t= 1578684300; bh=A/df1J/B54l0KM0/00+PO5xQN1aMhMgviDiRshebxOs=; b=n YLwOvoMFQgZQnplE94FZofL1bacH31bQfzVDWWY+m71J4ltc47BNi8JoMwn8Y6zG OvlBDd1mnPqCqBf9VYRIjPG7DPptNz/lwhZacOfMBSwWi0uILqsTu/LFxIaAmEak /Jld98/fExeJdiadSiOwBdvXFUGZZuJQORgmmnFndG0wXqe2Sm9do9MO0eA4K90o f2kJFyrBVQUYfYaEzyGKJDiE4kqTEvOB7b7x3RQuV/gjxeriV4R4eIu+Tpw6wHa5 NEz0dozK+bEs7aZ/+lEHCdwiP2ZAHjbItrKbvM7XhZOvUN/cG/hwxquUTA7QJefF /OH9uDUrpBI/EWrMM4H5A== ARC-Authentication-Results: i=1; tb-mx1.topicbox.com; arc=none (no signatures found); dkim=pass (2048-bit rsa key sha256) header.d=sphericalharmony-com.20150623.gappssmtp.com header.i=@sphericalharmony-com.20150623.gappssmtp.com header.b=fsqL5fPQ header.a=rsa-sha256 header.s=20150623 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=sphericalharmony.com; iprev=pass smtp.remote-ip=209.85.166.175 (mail-il1-f175.google.com); spf=none smtp.mailfrom=mycroftiv@sphericalharmony.com smtp.helo=mail-il1-f175.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=i4DU3I1f; x-ptr=pass smtp.helo=mail-il1-f175.google.com policy.ptr=mail-il1-f175.google.com; x-return-mx=pass header.domain=sphericalharmony.com policy.is_org=yes (MX Record found); x-return-mx=pass smtp.domain=sphericalharmony.com policy.is_org=yes (MX Record found); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 Authentication-Results: tb-mx1.topicbox.com; arc=none (no signatures found); dkim=pass (2048-bit rsa key sha256) header.d=sphericalharmony-com.20150623.gappssmtp.com header.i=@sphericalharmony-com.20150623.gappssmtp.com header.b=fsqL5fPQ header.a=rsa-sha256 header.s=20150623 x-bits=2048; dmarc=none policy.published-domain-policy=none policy.applied-disposition=none policy.evaluated-disposition=none (p=none,d=none,d.eval=none) policy.policy-from=p header.from=sphericalharmony.com; iprev=pass smtp.remote-ip=209.85.166.175 (mail-il1-f175.google.com); spf=none smtp.mailfrom=mycroftiv@sphericalharmony.com smtp.helo=mail-il1-f175.google.com; x-aligned-from=pass (Address match); x-google-dkim=pass (2048-bit rsa key) header.d=1e100.net header.i=@1e100.net header.b=i4DU3I1f; x-ptr=pass smtp.helo=mail-il1-f175.google.com policy.ptr=mail-il1-f175.google.com; x-return-mx=pass header.domain=sphericalharmony.com policy.is_org=yes (MX Record found); x-return-mx=pass smtp.domain=sphericalharmony.com policy.is_org=yes (MX Record found); x-tls=pass smtp.version=TLSv1.2 smtp.cipher=ECDHE-RSA-AES256-GCM-SHA384 smtp.bits=256/256; x-vs=clean score=0 state=0 X-ME-VSCause: gggruggvucftvghtrhhoucdtuddrgedufedrvdeifedguddvtdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecunecujfgurhephffkff fvufggtgfgsehtjeejtddttddvnecuhfhrohhmpehmhigtrhhofhhtihhvsehsphhhvghr ihgtrghlhhgrrhhmohhnhidrtghomhenucffohhmrghinhepmhgrrhgtrdhinhhfohdple hgrhhiuggthhgrnhdrohhrghenucfkphepvddtledrkeehrdduieeirddujeehpdeiiedr udekledrudefjedrudefvdenucfrrghrrghmpehinhgvthepvddtledrkeehrdduieeird dujeehpdhhvghlohepmhgrihhlqdhilhduqdhfudejhedrghhoohhglhgvrdgtohhmpdhm rghilhhfrhhomhepoehmhigtrhhofhhtihhvsehsphhhvghrihgtrghlhhgrrhhmohhnhi drtghomhequcfukfgkgfepuddvfeegtdenucevlhhushhtvghrufhiiigvpedt X-ME-VSCategory: clean Received-SPF: none (sphericalharmony.com: No applicable sender policy available) receiver=tb-mx1.topicbox.com; identity=mailfrom; envelope-from="mycroftiv@sphericalharmony.com"; helo=mail-il1-f175.google.com; client-ip=209.85.166.175 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tb-mx1.topicbox.com (Postfix) with ESMTPS for <9fans@9fans.net>; Fri, 10 Jan 2020 14:24:59 -0500 (EST) (envelope-from mycroftiv@sphericalharmony.com) Received: by mail-il1-f175.google.com with SMTP id f5so2654930ilq.5 for <9fans@9fans.net>; Fri, 10 Jan 2020 11:24:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sphericalharmony-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:date:to:subject:mime-version :content-transfer-encoding; bh=A/df1J/B54l0KM0/00+PO5xQN1aMhMgviDiRshebxOs=; b=fsqL5fPQc4CVBaZ1nY4SnqglXp4lK4n84svTr+c8Zu44ou3a4mOlHum4y8e59OwTxd NGlTUWx1iA8oSCubgIogQ9Oh/x065NaJrcau2Aw7ProZm4Z/WrFaQ4UnLeICsCrAj+TR 2TCM5ZnY41JUwEnjYi//1rR/qZqWVVaZoJG77Xs/wQAHQGRqv7anob9CAm7AD2hjVp0M SHAs4PToNVhUhAY6+U4UQIIUpOcYt1BGyhYj2KMYgsiGVmdbIG1PvMYlyxEtS5VBzQCz uCSRvhCGeU+cjLbwHd6DY34IyjQQ8FnTbh/EBvzTaFAS0fzaTXiXAUYC9vp52aIPIP+z fG8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:date:to:subject:mime-version :content-transfer-encoding; bh=A/df1J/B54l0KM0/00+PO5xQN1aMhMgviDiRshebxOs=; b=i4DU3I1fYQ8o7Q8XE+YzH6R9EaICEjWKA0t8dmW3ipxN2eslk54nvvKRC+CrOBtONO plRx6Y98zX7F9hca7veZ/tvXR4IEg+H3dhySX3yv6A3qXfiipG79RFjy7N3J6zl87Mne Vu07jIQIdSo58IRsUnNPdzoWJqR6hK+aMwzb04dsJHR8ifguHAVQTv2p4+e2zg7yh2t1 beIzqO3eREUsPtdlyz1x6Uh9jOTqq6KX7i/EN7DtBNQ4pDDQgLGr7QNsu8eGz3atAwXe LxssFgvpIAi8zMKZq9NMKwPXdgTN7OeArOjumZY4P8ltzCd3emkYzEjkylx0KAQ/jAHz PTNA== X-Gm-Message-State: APjAAAWE8Wd/QfDfxdMDwyMeyjZPq7wiB91PEPxSlcmLmEQc2Z6AEQ7k 2S3qWK51eW1f6SFOqEKYsEXX8RnH0I0= X-Google-Smtp-Source: APXvYqzuqpgp9A+uCRxViupxEvpg896WC69tHU8B+pC6HnMNWDhargfCi9M5jP6yhGWSlbRTb1WO5A== X-Received: by 2002:a05:6e02:80c:: with SMTP id u12mr3969665ilm.273.1578684298850; Fri, 10 Jan 2020 11:24:58 -0800 (PST) Received: from opti.home (66-189-137-132.dhcp.mdsn.wi.charter.com. [66.189.137.132]) by smtp.gmail.com with ESMTPSA id g27sm952763ild.26.2020.01.10.11.24.58 for <9fans@9fans.net> (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 10 Jan 2020 11:24:58 -0800 (PST) From: mycroftiv@sphericalharmony.com X-Google-Original-From: glenda@sphericalharmony.com Message-ID: Date: Fri, 10 Jan 2020 12:29:56 -0600 To: 9fans@9fans.net Subject: notes on fossil, ANTS, and 9front/Bell labs controversies MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Topicbox-Policy-Reasoning: allow: sender is a member Topicbox-Message-UUID: e9c72a76-33de-11ea-872c-f354491722b3 This is also posted at http://doc.9gridchan.org/blog/200110.fossil.9front This post is a survey of technical and community issues connected to Plan 9 root file servers. The author is not a member of any Plan 9 tribe save Grid. Opinions are entirely my own and a lot of unsourced claims will be made based on my years of participation and observation of the Plan 9 community. ANTS installer: fossil root for 9front code The ANTS iso installer scripts include fossil+venti root install option. This makes it easier to use the 9front codebase with fossil+venti root. The 9front developers have made a conscious and deliberate choice to remove fossil support from their distribution, so undoubtedly they don't see this as an improvement. I have been using a primarily 9front codebase on top of fossil+venti root fileservers for my home grid of systems and for most of the 9gridchan.org servers for the past several years with good results. My datasets are small and my results may not translate well to users with much larger quantities of data. The latest ANTS release contains support tooling for venti replication as well as installer support. - 64-bit 9ants5.64.iso.gz https://files.9gridchan.org/9ants5.64.iso.gz This distribution of Plan 9 has admittedly flaky maintainership, but competent users should be able to track 9front changes even if mycroftiv has wandered off to some distant hypercube. A highly subjective historical view When 9front forked from the Bell Labs distribution of Plan 9, one of the major changes was removing the option to use the fossil file server as a root file system and making cwfs64x the primary fs in the installer. In the late 2000s era of Plan 9, many users reported unreliable behavior of the fossil+venti combination. The resources being given to Plan 9 by the corporate heirs to Bell Labs were clearly insufficient to properly develop and debug the operating system. As I understand it, there were two engineers working part-time on the public Plan 9 infrastructure. Without sufficient resource investment, even the best efforts of skilled developers may not scale to the needs of a full operating system project. At the same time, the late and sorely missed Uriel was attempting to grow an independent community of people dedicated to the software principles espoused by the unix tradition and embodied in Plan 9. Unfortunately, in my view, Uriel's health problems influenced his communication, and his passionate desire to see Plan 9 succeed and thrive led him to debates full of angry sarcasm and increasing bitterness. The way it all played out was to create divisions and bad feelings between the corporate and university Plan 9 developers and researchers and much of the grass-roots Plan 9 community. Fossil-venti unreliability was a focal point of dissatisfaction. When 9front forked, removing fossil support from the installer was part of a clear message that the standard Bell Labs setup was not reliable. Discussions of the motivations and necessity for the 9front fork have often referenced poor user experiences with fossil+venti in the time prior to the fork. My own experience of trying to use fossil+venti in the 2000s era supports this perspective. In fact I found things so unreliable that a major factor in the software that became ANTS was "how to deal with the failure of the root filesystem" because that was a regular and expected occurence in my use of Plan 9. - The bug and fix In 2012, after 9front had already forked, Richard Miller found and fixed a bug in fossil that I believe was the primary source of many fossil-venti reliability issues. He wrote a very nice post describing the bug and its solution. https://marc.info/?l=9fans&m=133243392017939&w=2 I believe the precise nature of this bug explains a lot of things. As I understand it, the condition for triggering the bug is making changes to filesystem data during the archival snapshot process. For experienced Plan 9 users with always-on fileservers set to do a daily snapshot in the middle of the night, things were reliable. For a new user who installed Plan 9 and started immediately messing around and installing software while the initial archival snapshot is happening, there was a good chance that corruption would sneak into the fs from the very beginning. This matches my memory of frequent troubles with installs that seemed to break right away. In 2015, I migrated ANTS to 9front, and because I have always used and valued the powerful operations that venti-backed fossil rootscores support, I decided to figure out how to use 9front/ants with a fossil root. I was expecting trouble because of the prevailing sentiment in the 9front community that was fossil was a no-good very-bad filesystem, and my own previous struggles. In practice however, what I discovered was that fossil seemed to be working just fine. I worked out a system for getting 9front+ants+fossil onto vultr vps nodes and started hosting 9gridchan.org websites on this platform as well as using it for my home systems. Between 2015 and now, I believe I have accumulated 25+ machine-years of trouble free 9front/ants/fossil service on my home systems and public servers. Most of these systems only store a few gigabytes of data total so the workload is light however. Other users with large data sets have expressed some disappointment with performance. - The Tribes Most people understand that the Plan 9 user community is both small, and strongly fractured into tribes who seem to harbor personal grudges toward each other. It's easy to understand why the sense of humor and strongly critical perspective of many "9front tribe" people rub some the wrong way - especially the people who are the actual target of satire. People just aren't really into being criticized and satirized as a general rule. It's also easy to understand why what can only be referred to as the "Old Boys Club" of Plan 9 has critics. Plan 9 has always had overtones of elitism - from my reading, Rob Pike's writings and commentary set a lot of the tone which Uriel amplified. The "haughty style" of commentary on computing, in which the Sad and Fallen state of computing into complexity and muddled design is Lamented by the Sage goes back a long way - Dijkstra in particular perpetrated many famous examples. Humans love having social status, and nothing signals high social status more than condescending to the un-enlightened. New users arriving in the Plan 9 community in the 2000s discovered a mismatch between the rhetoric of how much superior everything was in the Plan 9 world, and the reality of unreliable root fileservers, challenging and complicated administration with documentation focused on Murray Hill's system, and lots of rumors of intriguing code and research projects that didn't seem to feed into the publicly available Plan 9 experience. The operating system seemed rather stagnant and buggy, but the veterans of the Plan 9 community seemed to talk in a parallel universe in which Plan 9 was perfectly meeting all of their computing needs and any issues were the result of user error and inexperience. - The Present Day, SHA1, community So, 9front tribe doesn't want to use fossil, and labs/legacy tribe doesn't want to use 9front code. I believe that both of these perspectives at this point are based more on human social conflicts than solid technical motivations. I am trying to offer a solution to users who are interested in exploring the most practical ways to use Plan 9. The major fly in the ointment is the outdated sha1 hash function used by venti. As a result of this yet-to-be-fixed shortcoming, the world of venti-backed public plan 9 services is not yet ready for large scale production use. For personal and hobby use howeveer, I believe the sha1 issues are manageable. I hope that this post can serve a constructive purpose in building mutual understanding between humans and increasing code-sharing and collaboration. Given the way things often go, I'm probably screwing up and just pissing everyone on both sides off more by presenting my own warped and subjective view of things. I recall being told to "stop alternating history" for sharing my perspective on the saga of Plan 9 community and software - so please offer your own corrections and perspective, especially if you think I have made any factual mistakes or you have first hand knowledge and experience to share. I've always had pretty good interactions with other humans who use Plan 9 and I'm really into the whole "tolerant and diverse community" thing. I'd like a lot more people from different places and with different backgrounds and cultures and life experiences to get into Plan 9 and find it to be a fun and welcoming experience. I've been trying since 2008 to help people learn to use Plan 9 to communicate and connect and cooperate and have better and more direct control over their software and system, outside of systems of hierarchical authority and control and exploitation and coercion. I've made a lot of mistakes in my own communication and community building efforts. 2020 seems like a Year of Change and I'm hoping for positive change in the Plan 9 community. I'm particularly excited for the reincarnation of a real-life Plan 9 conference/hackathon event. Humans do better in real life social situations than internet diatribes and perhaps the time is ripe for the ever-blooming Tree of Files to send forth new fruits. Peace, love, hypercubes, and ultrafilters to all Mycroftiv