From mboxrd@z Thu Jan  1 00:00:00 1970
To: 9fans@cse.psu.edu
Subject: Re: [9fans] bell-labs website and plan9
From: "Russ Cox" <rsc@swtch.com>
Date: Mon,  9 Apr 2007 11:50:41 -0400
In-Reply-To: <9ab217670704090750h19fd808dw171a046d088df5f@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Message-Id: <20070409155043.6EC651E8C1F@holo.morphisms.net>
Topicbox-Message-UUID: 4233b608-ead2-11e9-9d60-3106f5b1d025

devon:
> One thing that this will need is support for a '*' target for -s (and
> I guess for -c, too, just for consistency) to replica/applylog. I have
> a patch for this, and I'll submit it shortly. This flag would make
> applylog ignore all client-side (or server-side) changes, though I
> suppose the latter is already possible.

Since the arguments to -c and -s are just string prefixes, you 
can use -c '' or -s '' already.  I admit it is non-standard.

However, you'd be better off not using replica/pull as
the input to a differ, but instead using replica's change file
/n/sources/plan9/dist/replica/plan9.log.  Replica(8) describes
the format:

   A replica is further described on the server by a textual
   log listing creation and deletion of files and changes to
   file contents and metadata.  Each line is of the form:

      time gen verb path serverpath mode uid gid mtime length

   The time and gen fields are both decimal numbers, providing
   an ordering for log entries so that incremental tools need
   not process the whole log each time they are run.  The verb,
   a single character, describes the event: addition of a file
   (a), deletion of a file (d), a change to a file's contents
   (c), or a change to a file's metadata (m).  Path is the file
   path on the client; serverpath the path on the server (these
   are different when the optional fifth field in a proto file
   line is given; see proto(2)). Mode, uid, gid, and mtime are
   the files metadata as in the Dir structure (see stat(5)).
   For deletion events, the metadata is that of the deleted
   file.  For other events, the metadata is that after the
   event.

It would be easiest to pick the two most recent dumps in
/n/sourcesdump, diff the plan9.log files to pick out the
new lines, and then run diffs between the two different
dump roots.  This is what we used to do when we (I) annotated
all the changes to produce /n/sources/extra/changes.
But it was far too much work to do the annotations and
not enough people cared, so we stopped that particular
experiment.

erik:
> i have also noticed that replica/applylog has a problem.  when i started
> experimenting with copying history from our old fileserver to the new
> one, i started using replica/updatedb and replica/applylog.  updatedb
> worked very well, but applylog hung for me pretty consistantly.

Did you ever use acid to get a stack trace from the `hung' applylogs?
The only threading in applylog is an implementation of something
like fcp to copy files using multiple outstanding 9P read requests.
Since no one else seems to have had problems, I would guess that
there were just some requests that made your file server thrash.
But stack traces would make the answer very clear.

ron:
> My take is that bringing in mercurial, and then using mercurial with
> mounted file systems, instead of ssh, would be quite neat. And, we are
> close to having it. We've got python, almost. We've just about got
> python modules -- actually, I've had them for months. Lots of bits
> work that did not used to. We really are pretty close.

Echoing Ron, I think having Mercurial would be great and doable.
If someone makes a Mercurial client work, I will be happy to make
a Mercurial repository that mirrors sources automatically.

Also echoing Ron, a venti-based SCM sounds similar to git,
which is an SCM built on top of a hash-addressed object store
(that happens not to be named venti).  It would be nice to know
you're not reinventing git, especially since in my experience
the fact that git is hash-addressed makes many things a lot
harder and slower (although I am sure it has advantages).

devon:
> In fact, I seem to recall the last time Uriel was yelling about this
> on the list, Russ offered to do exactly this. Is this offer still
> available? It'd save me some time and effort trying to figure out how
> to revert a replica/pull.

I put a copy of the script we used to use in
/n/sources/contrib/rsc/makemail.  Use at your own risk.
It probably deserves to be rewritten in a better language.

devon again:
> So maybe to go further down that path (and I would
> _love_ to get input from Bell Labs people because they're really the
> ones that have the power to yay or nay any of this), is replica/*
> still an ideal manner for getting updates? Are there potentially
> better ways to do this?

There are things that replica doesn't do very well.  I wish you
could tell it to back up, or to back out local changes, and so on,
but the core functionality works well, and I would be wary of
falling into the CADT Model trap (ask Google).

devon again:
> The scenario being, I want to test my hypothetical replica/applylog
> action parser / diff generator. I run pull, which grabs stuff, but
> I've made changes to xyz.c and that file doesn't get modified because
> of local changes. So I want to roll back my log so I can run pull
> again with -s /sys/src/cmd/xyz.c

Your thinking is stuck in the maze of twisty little passages
that is modern Unix and its version control systems.
Replica is a file distribution mechanism, not a version 
control system.

On Plan 9, one would do this with the dump file system.
9fs sourcesdump and look around -- you've got all the info
there you could possibly want to produce diffs.  You don't
even need to copy anything to your local machine.

Russ