9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* notes on fossil, ANTS, and 9front/Bell labs controversies
@ 2020-01-10 18:29 mycroftiv
  2020-01-10 21:36 ` [9fans] " hiro
  2020-01-11  4:36 ` Lucio De Re
  0 siblings, 2 replies; 8+ messages in thread
From: mycroftiv @ 2020-01-10 18:29 UTC (permalink / raw)
  To: 9fans

This is also posted at http://doc.9gridchan.org/blog/200110.fossil.9front

This post is a survey of technical and community issues connected to
Plan 9 root file servers.  The author is not a member of any Plan 9
tribe save Grid.  Opinions are entirely my own and a lot of unsourced
claims will be made based on my years of participation and observation
of the Plan 9 community.

ANTS installer: fossil root for 9front code

The ANTS iso installer scripts include fossil+venti root install
option.  This makes it easier to use the 9front codebase with
fossil+venti root.  The 9front developers have made a conscious and
deliberate choice to remove fossil support from their distribution, so
undoubtedly they don't see this as an improvement.  I have been using
a primarily 9front codebase on top of fossil+venti root fileservers
for my home grid of systems and for most of the 9gridchan.org servers
for the past several years with good results.  My datasets are small
and my results may not translate well to users with much larger
quantities of data.  The latest ANTS release contains support tooling
for venti replication as well as installer support.

- 64-bit 9ants5.64.iso.gz 
https://files.9gridchan.org/9ants5.64.iso.gz

This distribution of Plan 9 has admittedly flaky maintainership, but
competent users should be able to track 9front changes even if
mycroftiv has wandered off to some distant hypercube.

A highly subjective historical view

When 9front forked from the Bell Labs distribution of Plan 9, one of
the major changes was removing the option to use the fossil file
server as a root file system and making cwfs64x the primary fs in the
installer.  In the late 2000s era of Plan 9, many users reported
unreliable behavior of the fossil+venti combination.  The resources
being given to Plan 9 by the corporate heirs to Bell Labs were clearly
insufficient to properly develop and debug the operating system.  As I
understand it, there were two engineers working part-time on the
public Plan 9 infrastructure.  Without sufficient resource investment,
even the best efforts of skilled developers may not scale to the needs
of a full operating system project.

At the same time, the late and sorely missed Uriel was attempting to
grow an independent community of people dedicated to the software
principles espoused by the unix tradition and embodied in Plan 9.
Unfortunately, in my view, Uriel's health problems influenced his
communication, and his passionate desire to see Plan 9 succeed and
thrive led him to debates full of angry sarcasm and increasing
bitterness.  The way it all played out was to create divisions and bad
feelings between the corporate and university Plan 9 developers and
researchers and much of the grass-roots Plan 9 community.

Fossil-venti unreliability was a focal point of dissatisfaction.  When
9front forked, removing fossil support from the installer was part of
a clear message that the standard Bell Labs setup was not reliable.
Discussions of the motivations and necessity for the 9front fork have
often referenced poor user experiences with fossil+venti in the time
prior to the fork.  My own experience of trying to use fossil+venti in
the 2000s era supports this perspective.  In fact I found things so
unreliable that a major factor in the software that became ANTS was
"how to deal with the failure of the root filesystem" because that was
a regular and expected occurence in my use of Plan 9.

- The bug and fix

In 2012, after 9front had already forked, Richard Miller found and
fixed a bug in fossil that I believe was the primary source of many
fossil-venti reliability issues.  He wrote a very nice post describing
the bug and its solution.

https://marc.info/?l=9fans&m=133243392017939&w=2

I believe the precise nature of this bug explains a lot of things.  As
I understand it, the condition for triggering the bug is making
changes to filesystem data during the archival snapshot process.  For
experienced Plan 9 users with always-on fileservers set to do a daily
snapshot in the middle of the night, things were reliable.  For a new
user who installed Plan 9 and started immediately messing around and
installing software while the initial archival snapshot is happening,
there was a good chance that corruption would sneak into the fs from
the very beginning.  This matches my memory of frequent troubles with
installs that seemed to break right away.

In 2015, I migrated ANTS to 9front, and because I have always used and
valued the powerful operations that venti-backed fossil rootscores
support, I decided to figure out how to use 9front/ants with a fossil
root.  I was expecting trouble because of the prevailing sentiment in
the 9front community that was fossil was a no-good very-bad
filesystem, and my own previous struggles.  In practice however, what
I discovered was that fossil seemed to be working just fine.  I worked
out a system for getting 9front+ants+fossil onto vultr vps nodes and
started hosting 9gridchan.org websites on this platform as well as
using it for my home systems.

Between 2015 and now, I believe I have accumulated 25+ machine-years
of trouble free 9front/ants/fossil service on my home systems and
public servers.  Most of these systems only store a few gigabytes of
data total so the workload is light however.  Other users with large
data sets have expressed some disappointment with performance.

- The Tribes

Most people understand that the Plan 9 user community is both small,
and strongly fractured into tribes who seem to harbor personal grudges
toward each other.  It's easy to understand why the sense of humor and
strongly critical perspective of many "9front tribe" people rub some
the wrong way - especially the people who are the actual target of
satire.  People just aren't really into being criticized and satirized
as a general rule.

It's also easy to understand why what can only be referred to as the
"Old Boys Club" of Plan 9 has critics.  Plan 9 has always had
overtones of elitism - from my reading, Rob Pike's writings and
commentary set a lot of the tone which Uriel amplified.  The "haughty
style" of commentary on computing, in which the Sad and Fallen state
of computing into complexity and muddled design is Lamented by the
Sage goes back a long way - Dijkstra in particular perpetrated many
famous examples.  Humans love having social status, and nothing
signals high social status more than condescending to the
un-enlightened.

New users arriving in the Plan 9 community in the 2000s discovered a
mismatch between the rhetoric of how much superior everything was in
the Plan 9 world, and the reality of unreliable root fileservers,
challenging and complicated administration with documentation focused
on Murray Hill's system, and lots of rumors of intriguing code and
research projects that didn't seem to feed into the publicly available
Plan 9 experience.  The operating system seemed rather stagnant and
buggy, but the veterans of the Plan 9 community seemed to talk in a
parallel universe in which Plan 9 was perfectly meeting all of their
computing needs and any issues were the result of user error and
inexperience.

- The Present Day, SHA1, community

So, 9front tribe doesn't want to use fossil, and labs/legacy tribe
doesn't want to use 9front code.  I believe that both of these
perspectives at this point are based more on human social conflicts
than solid technical motivations.  I am trying to offer a solution to
users who are interested in exploring the most practical ways to use
Plan 9.  The major fly in the ointment is the outdated sha1 hash
function used by venti.  As a result of this yet-to-be-fixed
shortcoming, the world of venti-backed public plan 9 services is not
yet ready for large scale production use.  For personal and hobby use
howeveer, I believe the sha1 issues are manageable.

I hope that this post can serve a constructive purpose in building
mutual understanding between humans and increasing code-sharing and
collaboration.  Given the way things often go, I'm probably screwing
up and just pissing everyone on both sides off more by presenting my
own warped and subjective view of things.  I recall being told to
"stop alternating history" for sharing my perspective on the saga
of Plan 9 community and software - so please offer your own
corrections and perspective, especially if you think I have made any
factual mistakes or you have first hand knowledge and experience to
share.

I've always had pretty good interactions with other humans who use
Plan 9 and I'm really into the whole "tolerant and diverse community"
thing.  I'd like a lot more people from different places and with
different backgrounds and cultures and life experiences to get into
Plan 9 and find it to be a fun and welcoming experience.  I've been
trying since 2008 to help people learn to use Plan 9 to communicate
and connect and cooperate and have better and more direct control over
their software and system, outside of systems of hierarchical
authority and control and exploitation and coercion.

I've made a lot of mistakes in my own communication and community
building efforts.  2020 seems like a Year of Change and I'm hoping for
positive change in the Plan 9 community.  I'm particularly excited for
the reincarnation of a real-life Plan 9 conference/hackathon event.
Humans do better in real life social situations than internet
diatribes and perhaps the time is ripe for the ever-blooming Tree of
Files to send forth new fruits.

Peace, love, hypercubes, and ultrafilters to all

Mycroftiv

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-01-13  9:34 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-10 18:29 notes on fossil, ANTS, and 9front/Bell labs controversies mycroftiv
2020-01-10 21:36 ` [9fans] " hiro
2020-01-11 10:31   ` Steve Simon
2020-01-11 10:41     ` Rodrigo G. López
2020-01-11 15:56       ` Steve Simon
2020-01-11 19:32         ` Rodrigo G. López
2020-01-13  9:33           ` Richard Miller
2020-01-11  4:36 ` Lucio De Re

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).