Re: [9fans] bell-labs website and plan9

Re: [9fans] bell-labs website and plan9

[erik quanstrom]

On Mon Apr  9 11:50:41 EDT 2007, rsc@swtch.com wrote:
> erik:
> > i have also noticed that replica/applylog has a problem.  when i started
> > experimenting with copying history from our old fileserver to the new
> > one, i started using replica/updatedb and replica/applylog.  updatedb
> > worked very well, but applylog hung for me pretty consistantly.
> 
> Did you ever use acid to get a stack trace from the `hung' applylogs?
> The only threading in applylog is an implementation of something
> like fcp to copy files using multiple outstanding 9P read requests.
> Since no one else seems to have had problems, I would guess that
> there were just some requests that made your file server thrash.
> But stack traces would make the answer very clear.

i apologize for not having a backtrace, they looked uninformative at the
time.  what i rememer was that applylog was not doing any i/o at the time.
(unless it was reading the same blocks over and over.)
in once instance, applylog had the same /proc/$pid/fd for 4 hrs
and generated no system load at all. the 

one problem i do see that was not my case (i was working on two successive
days from the dump) is that there is no maximum number
of tries to keep a file from changing underfoot.  a log file competing with
a slow link could be problematic.

restarting it where it left off (with an initial line number) generally
fixed the problem.  i didn't mention it at the time because i 
didn't get to the bottom of the problem.

i'll try to recreate the problem with a backtrace, but anyone else is welcome
to beat me to it.

- erik

Re: [9fans] bell-labs website and plan9

[Francisco J Ballesteros]

The way it's done is backwards compatible in the sense that no app/fs knows
that anyone is speaking Op. All our tools speak styx, and the inferno
kernel/emu is
the client of choice. Op is used to brigde separate islands so that
nobody else has
to care.

On 4/10/07, Uriel <uriel99@gmail.com> wrote:
> One word: latency.
>
> By the way, is anyone working on the solutions to the latency problem
> we discussed at IWP9? I know nemo has a somewhat different solution
> with OP but I still would like to see something backwards compatible
> with existing 9P.
>
> uriel
>
> On 4/10/07, erik quanstrom <quanstro@coraid.com> wrote:
> > i don't understand why ssh "using a seperate protocol" implies that it should
> > be significantly faster than 9p.  could you explain why you think this?
> >
> > On Mon Apr  9 19:27:33 EDT 2007, bsdaemon@comcast.net wrote:
> >
> > > On Mon, Apr 09, 2007 at 08:22:27AM -0700, ron minnich wrote:
> > > >My take is that bringing in mercurial, and then using mercurial with
> > > >mounted file systems, instead of ssh, would be quite neat. And, we are
> > > >close to having it.
> > >
> > > I'd think that it would be practically a no-op, but I'm not sure of hg's
> > > locking semantics. I'd say that it would be a very good idea to support
> > > cpu and ssh, though, since ssh uses a separate protocol which should be
> > > significantly faster than just doing the work over 9P. Again, I'm not
> > > sure of the details.
> >
>
>

Re: [9fans] bell-labs website and plan9

[Kris Maglione]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On Mon, Apr 09, 2007 at 08:15:56PM -0400, erik quanstrom wrote:
>i don't understand why ssh "using a seperate protocol" implies that it should
>be significantly faster than 9p.  could you explain why you think this?

I don't know the specifics, but I do know that there is a protocol used 
for HTTP and SSH which are designed to speed up operations on the 
repository. I think it relates to the fact that parsing the FS would 
require many synchronous round-trips which would be faster locally. I 
believe that the paper elaborates.

-- 
Kris Maglione

When you need to knock on wood is when you realize the
world's composed of aluminum and vinyl.

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: [9fans] bell-labs website and plan9

[Uriel]

One word: latency.

By the way, is anyone working on the solutions to the latency problem
we discussed at IWP9? I know nemo has a somewhat different solution
with OP but I still would like to see something backwards compatible
with existing 9P.

uriel

On 4/10/07, erik quanstrom <quanstro@coraid.com> wrote:
> i don't understand why ssh "using a seperate protocol" implies that it should
> be significantly faster than 9p.  could you explain why you think this?
>
> On Mon Apr  9 19:27:33 EDT 2007, bsdaemon@comcast.net wrote:
>
> > On Mon, Apr 09, 2007 at 08:22:27AM -0700, ron minnich wrote:
> > >My take is that bringing in mercurial, and then using mercurial with
> > >mounted file systems, instead of ssh, would be quite neat. And, we are
> > >close to having it.
> >
> > I'd think that it would be practically a no-op, but I'm not sure of hg's
> > locking semantics. I'd say that it would be a very good idea to support
> > cpu and ssh, though, since ssh uses a separate protocol which should be
> > significantly faster than just doing the work over 9P. Again, I'm not
> > sure of the details.
>

Re: [9fans] bell-labs website and plan9

[erik quanstrom]

i don't understand why ssh "using a seperate protocol" implies that it should
be significantly faster than 9p.  could you explain why you think this?

On Mon Apr  9 19:27:33 EDT 2007, bsdaemon@comcast.net wrote:

> On Mon, Apr 09, 2007 at 08:22:27AM -0700, ron minnich wrote:
> >My take is that bringing in mercurial, and then using mercurial with
> >mounted file systems, instead of ssh, would be quite neat. And, we are
> >close to having it.
> 
> I'd think that it would be practically a no-op, but I'm not sure of hg's 
> locking semantics. I'd say that it would be a very good idea to support 
> cpu and ssh, though, since ssh uses a separate protocol which should be 
> significantly faster than just doing the work over 9P. Again, I'm not 
> sure of the details.

Re: [9fans] bell-labs website and plan9

[ron minnich]

On 4/9/07, Kris Maglione <bsdaemon@comcast.net> wrote:
>I'd say that it would be a very good idea to support
> cpu and ssh, though, since ssh uses a separate protocol which should be
> significantly faster than just doing the work over 9P.

I'm not sure I understand what you mean here. But, that said, I will
try to see what hg will want ...

BTW, I do recommend that you all take a look at Xen 3 and Plan 9. I
just needed to check something out and having a plan 9 instance up and
running in 6 seconds is really, really nice.

Thanks

ron

Re: [9fans] bell-labs website and plan9

[Kris Maglione]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

On Mon, Apr 09, 2007 at 08:22:27AM -0700, ron minnich wrote:
>My take is that bringing in mercurial, and then using mercurial with
>mounted file systems, instead of ssh, would be quite neat. And, we are
>close to having it.

I'd think that it would be practically a no-op, but I'm not sure of hg's 
locking semantics. I'd say that it would be a very good idea to support 
cpu and ssh, though, since ssh uses a separate protocol which should be 
significantly faster than just doing the work over 9P. Again, I'm not 
sure of the details.

-- 
Kris Maglione

There's no time like the present for postponing
what you don't want to do.

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: [9fans] bell-labs website and plan9

[Christopher Nielsen]

if there isn't, we should start one. i am in san francisco.
though, i am about to depart for fire season, but i'll be back some
time in november.

On 4/9/07, ron minnich <rminnich@gmail.com> wrote:
>
> Gee, is there a CA-based plan 9 user's group?
>
> ron
>


-- 
Christopher Nielsen
"They who can give up essential liberty for temporary
safety, deserve neither liberty nor safety." --Benjamin Franklin

Re: [9fans] bell-labs website and plan9

[Paweł Lasek]

On 4/9/07, ron minnich <rminnich@gmail.com> wrote:
>
> > As far as an SCM goes, it would be nice to have one, but I'm sure it
> > has to run in Plan 9. The big problem that I have with hg and my own
> > vcs is that neither is really suited for maintaining multiple
> > branches.
>
> Are you sure about Hg in this case? The xen tree is about the size of
> the plan 9 kernel, for sure, and maybe all of /sys/src, and there are
> lots of Xen trees out there. I have not used Hg enough to say, but my
> observation is that it has worked well for xen in distributed repo
> mode.

AFAIK, Sun moves the whole OpenSolaris tree (kernel, userspace, docs,
EVERYTHING) into Mercurial. They IIRC use "forest" add-on to manage
multiple trees (for different parts of the system) at once.

I think Hg with it's tree backed up into Venti would be great thing to have :-)

And for big projects... I heard something about Vesta, but judging
from the webpage it's really complex system and Unix only.

> ron
>

-- 
Paul Lasek

Re: [9fans] bell-labs website and plan9

[ron minnich]

On 4/9/07, andrey mirtchovski <mirtchovski@gmail.com> wrote:
> as far as i remember webls was written by dan cross.
>
> the non-standard thing of 9grid's httpd is the use of full regular
> expressions in its rewrite file as well as other minor tweaks.
>

Sorry, I was confused; but AFAIK the "andrey/aki version" has some
nice features. Can they be included in the distro?

thanks

ron

Re: [9fans] bell-labs website and plan9

[andrey mirtchovski]

as far as i remember webls was written by dan cross.

the non-standard thing of 9grid's httpd is the use of full regular
expressions in its rewrite file as well as other minor tweaks.

Re: [9fans] bell-labs website and plan9

[geoff]

Everybody should have webls; it's
/n/sources/plan9/sys/src/cmd/ip/httpd/webls.c and
/n/sources/plan9/386/bin/ip/httpd/webls.

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, ron minnich <rminnich@gmail.com>:
> On 4/9/07, Uriel <uriel99@gmail.com> wrote:
>
> > It also would be nice if ron and the other LANL folks could reveal
> > (and release) the state of their gcc port.
> >
>
> Sorry about that. What's at 9grid.net is the latest from me.
>
> Note that I am no longer at LANL, but am at Sandia Labs in livermore, ca.
>
> Gee, is there a CA-based plan 9 user's group?

There was last year when I lived there. I'm in New York now :(

> ron

Re: [9fans] bell-labs website and plan9

[ron minnich]

On 4/9/07, Uriel <uriel99@gmail.com> wrote:

> It also would be nice if ron and the other LANL folks could reveal
> (and release) the state of their gcc port.
>

Sorry about that. What's at 9grid.net is the latest from me.

Note that I am no longer at LANL, but am at Sandia Labs in livermore, ca.

Gee, is there a CA-based plan 9 user's group?

ron

Re: [9fans] bell-labs website and plan9

[ron minnich]

On 4/9/07, Devon H. O'Dell <devon.odell@gmail.com> wrote:
>  I've asked you several times off-list about the status of
> Python and hg and getting these to work, but I haven't received a
> response yet, so I don't know if you got them.

I owe you an apology for that ... I'm sorry. What I've been trying to
do is find a way to apply $$$ to someone to
1) get my python stuff reworked in a more correct fashion
2) get the work done to put that code back in the mainstream.

This has not happened as soon as I might have hoped, because everyone
is (always) overworked.

I have put my python work to date at 9grid.net, I think this is it,
tell me if I screwed up!
http://9grid.net/magic/webls?dir=/rminnich/src

(and we should all ask andrey and aki to give us webls ... :-)

>Any
> information on the hg status and how one can help with that would be
> nice.

status: For Hg, I needed ssh2. We don't have it. So I got
paramiko-tools. It needed pycrypto. I got those. Pycrypto needed
multiple fixes to the Python port, which I made; I got the pycrypto
stuff to actually work. (and, along the way, realized that Python
plays C tricks that ought not to be played, but that's another story
... these Python C tricks required me to hack things so that
type-safety was set to "OFF OFF OFF OFF OFF") Then I prepared to
release it. Then I realized, that, me releasing crypto would be a
*really* *stupid* thing to do, given where I work and the state of
export control and so on and so forth. SO, ... I stopped.

Then you made me realize that I was being stupid, and should have
thought of Hg port in terms of file trees, not ssh, and now I have to
go look at this again. On plan 9, if you don't first do these tools
with either file trees or 9p, you are being dumb, and I was being dumb
in not thinking in those terms.

But, we still really need an ssh2 transport, at some point, because
that's what The Rest of The World, in their blindness, uses for most
of their Hg repos.

> As far as an SCM goes, it would be nice to have one, but I'm sure it
> has to run in Plan 9. The big problem that I have with hg and my own
> vcs is that neither is really suited for maintaining multiple
> branches.

Are you sure about Hg in this case? The xen tree is about the size of
the plan 9 kernel, for sure, and maybe all of /sys/src, and there are
lots of Xen trees out there. I have not used Hg enough to say, but my
observation is that it has worked well for xen in distributed repo
mode.

Thanks, and sorry I was so uncommunicative on the python stuff.

ron

Re: [9fans] bell-labs website and plan9

[Russ Cox]

> should expect this to work. Specific example, /sys/src/cmd/ip/ping.c
> was modified for my hushping patch, but pull didn't grab it because I
> had modified the copy there. If I run pull again with -s '', will it
> get that file this time?

You get one override per line in plan9.log.
If you have already run replica -c sys/src/cmd/ip/ping.c,
then future pulls will not worry about the change that
just happened.  Future changes will make new conflicts
that you'll have to resolve.  If you haven't done that,
then you can still run pull -s sys/src/cmd/ip/ping.c
and will get the new one.

Russ

Re: [9fans] bell-labs website and plan9

[Uriel]

On 4/9/07, Devon H. O'Dell <devon.odell@gmail.com> wrote:
> 2007/4/9, Uriel <uriel99@gmail.com>:
> > There are 75 people subscribed to the plan9changes[1] mailing list, is
> > that not enough people that care?
>
> Can we not go down this road? I'm offering to take up maintaining
> /n/sources/extra/changes for the People Who Do. Arguing about the past
> isn't going to help us progress into the future.

I am not arguing about the past, but I don't like people rewriting
history, if russ didn't want to keep updating the changelog, that is
fine, it was nice that he did it for a while, and it is his choice how
he spends his time; but claiming that nobody cared is ridiculous.

And I am pointing how many people care right now. If you or anyone is
going to maintain /n/sources/extra/changes, that will be really
fantastic and will make many people happy.

Of course, I don't think this is the right way to do things, the
easiest and most useful way to do it is for the person who makes the
change to writes down what the change is supposed to do, but I guess
this is a crazy suggestion in the Plan 9 universe.

An hg port would be very useful, and I would not mind if Plan 9 used
to maintain its codebase and distribute changes (I don't like replica,
you can check 9fans archives for some of the reasons, but in short: it
is slow, unreliable, and awkward to use.)

Once we find out if the code ron put in 9grid.net (now mirrored in
sources) is the latest, I might try to help dho get python and hg
running on Plan 9. Dho and me are still waiting to hear back from ron
about this after many unanswered private emails.

It also would be nice if ron and the other LANL folks could reveal
(and release) the state of their gcc port.

Best wishes

uriel

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, Uriel <uriel99@gmail.com>:
> > This is what we used to do when we (I) annotated
> > all the changes to produce /n/sources/extra/changes.
> > But it was far too much work to do the annotations and
> > not enough people cared, so we stopped that particular
> > experiment.
>
> There are 75 people subscribed to the plan9changes[1] mailing list, is
> that not enough people that care?

Can we not go down this road? I'm offering to take up maintaining
/n/sources/extra/changes for the People Who Do. Arguing about the past
isn't going to help us progress into the future.

> uriel
>
> [1] http://groups.google.com/group/plan9changes

--dho

Re: [9fans] bell-labs website and plan9

[Uriel]

> This is what we used to do when we (I) annotated
> all the changes to produce /n/sources/extra/changes.
> But it was far too much work to do the annotations and
> not enough people cared, so we stopped that particular
> experiment.

There are 75 people subscribed to the plan9changes[1] mailing list, is
that not enough people that care?

uriel

[1] http://groups.google.com/group/plan9changes

Re: [9fans] bell-labs website and plan9

[matt]

I don't know which bits are missing from Ron's port but the PyPy.org 
project has been making a pure python version of python.

I've been using bits of it on Nokia Series 60 Python which says it's 2.2 
but doesn't have all the 2.2 libs [such as Pickle, collections, mutex]), 
still no crypt though, so no using Newsham's py9p

The other plan 9 python is 2.2 also  
http://plan9.bell-labs.com/sources/extra/python.iso.bz2

Just FYI

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, Russ Cox <rsc@swtch.com>:
> devon:
> > One thing that this will need is support for a '*' target for -s (and
> > I guess for -c, too, just for consistency) to replica/applylog. I have
> > a patch for this, and I'll submit it shortly. This flag would make
> > applylog ignore all client-side (or server-side) changes, though I
> > suppose the latter is already possible.
>
> Since the arguments to -c and -s are just string prefixes, you
> can use -c '' or -s '' already.  I admit it is non-standard.

Ah, ok. I did a cursory glance over the matching code and ISTR it
using strcmp so I assumed it was a full match. In this case, I'll send
a patch for the manpage describing this (after I read it again and
determine that I didn't just miss something).

> However, you'd be better off not using replica/pull as
> the input to a differ, but instead using replica's change file
> /n/sources/plan9/dist/replica/plan9.log.  Replica(8) describes
> the format:
> [snip]
> It would be easiest to pick the two most recent dumps in
> /n/sourcesdump, diff the plan9.log files to pick out the
> new lines, and then run diffs between the two different
> dump roots.  This is what we used to do when we (I) annotated
> all the changes to produce /n/sources/extra/changes.
> But it was far too much work to do the annotations and
> not enough people cared, so we stopped that particular
> experiment.

I'll look into this. It does indeed seem to be a much more achievable goal.

> erik:
>[snip: not for me]
>
> ron:
> > My take is that bringing in mercurial, and then using mercurial with
> > mounted file systems, instead of ssh, would be quite neat. And, we are
> > close to having it. We've got python, almost. We've just about got
> > python modules -- actually, I've had them for months. Lots of bits
> > work that did not used to. We really are pretty close.
>
> Echoing Ron, I think having Mercurial would be great and doable.
> If someone makes a Mercurial client work, I will be happy to make
> a Mercurial repository that mirrors sources automatically.

Well, I prefer to not duplicate work, and the Python that I get from
Ron's website needs coaxing to build. Also, after doing that, setup.py
needed extra modules compiled in to the config. After that, it
setup.py needed the signal modules and that's when I got fed up. If
there's a Python port that builds a working python when I type `mk',
I'll start looking at it again, but I even had to edit the mkfile to
add the -x flag to 8l so it would do the dynamic linking magic.

I would be glad to work on getting this done, but I get the feeling
from Ron's postings that the code he has working is different than the
code I got from him. Ron: if you could clarify this for me, I'd really
appreciate it, because I'd definitely like to get the ball rolling on
this subject.

> Also echoing Ron, a venti-based SCM sounds similar to git,
> which is an SCM built on top of a hash-addressed object store
> (that happens not to be named venti).  It would be nice to know
> you're not reinventing git, especially since in my experience
> the fact that git is hash-addressed makes many things a lot
> harder and slower (although I am sure it has advantages).

I may be. I have never used git, and probably never will. I'm actually
trying to make something that is more similar to Perforce, but I'm not
sure that this is possible with doing what I'm doing using a vac
backend. Right now it's more experimental. If anybody would like to
use it for anything, what I currently have in
/n/sources/contrib/dho/vcs _does_ work for all intents and purposes,
it's just incomplete. However, I think 90% of the groundwork is there.

> devon:
> > In fact, I seem to recall the last time Uriel was yelling about this
> > on the list, Russ offered to do exactly this. Is this offer still
> > available? It'd save me some time and effort trying to figure out how
> > to revert a replica/pull.
>
> I put a copy of the script we used to use in
> /n/sources/contrib/rsc/makemail.  Use at your own risk.
> It probably deserves to be rewritten in a better language.

I will take a look at this.

> devon again:
> > So maybe to go further down that path (and I would
> > _love_ to get input from Bell Labs people because they're really the
> > ones that have the power to yay or nay any of this), is replica/*
> > still an ideal manner for getting updates? Are there potentially
> > better ways to do this?
>
> There are things that replica doesn't do very well.  I wish you
> could tell it to back up, or to back out local changes, and so on,
> but the core functionality works well, and I would be wary of
> falling into the CADT Model trap (ask Google).

Right, I did ask Google just now, and that makes sense. It is a valid
worry. I guess that the issue is that we currently sort of use replica
for distributing versions of our system, and, as you say below, it's
not a version control utility. More below.

> devon again:
> > The scenario being, I want to test my hypothetical replica/applylog
> > action parser / diff generator. I run pull, which grabs stuff, but
> > I've made changes to xyz.c and that file doesn't get modified because
> > of local changes. So I want to roll back my log so I can run pull
> > again with -s /sys/src/cmd/xyz.c
>
> Your thinking is stuck in the maze of twisty little passages
> that is modern Unix and its version control systems.
> Replica is a file distribution mechanism, not a version
> control system.

Well, the problem is that replica/pull won't update a file if it has
already found a conflict. (At least, I couldn't get it to earlier). I
will try re-retrieving with -s ''; I guess my question is whether I
should expect this to work. Specific example, /sys/src/cmd/ip/ping.c
was modified for my hushping patch, but pull didn't grab it because I
had modified the copy there. If I run pull again with -s '', will it
get that file this time?

However, as I said, we do use replica to keep systems `up to date', so
we do have some sort of versioned mentality in its usage. But it would
be nice to actually have some sort of SCM (hg, vcs, cvs or
what-have-you) as an option as well.

> On Plan 9, one would do this with the dump file system.
> 9fs sourcesdump and look around -- you've got all the info
> there you could possibly want to produce diffs.  You don't
> even need to copy anything to your local machine.

I'll follow your earlier advice for diff generation and take a look at
your makemail script.

> Russ

Thanks for the information; it seems rather useful.

--dho

Re: [9fans] bell-labs website and plan9

[Martin Neubauer]

* Devon H. O'Dell (devon.odell@gmail.com) wrote:
> The scenario being, I want to test my hypothetical replica/applylog
> action parser / diff generator. I run pull, which grabs stuff, but
> I've made changes to xyz.c and that file doesn't get modified because
> of local changes. So I want to roll back my log so I can run pull
> again with -s /sys/src/cmd/xyz.c
> 
> This wouldn't be an issue if we added a '*' option to replica/pull for
> the -c and -s flags. But I'll send a patch for that shortly and see if
> that gets committed.
> 
> --dho

I'm not sure how serious that really is. You can always run ``pull -n''
(actually, I regularly do); and if you positively want to replace every
locally modified file you can pipe the output through some trivial awk.

	Martin

Re: [9fans] bell-labs website and plan9

[Russ Cox]

devon:
> One thing that this will need is support for a '*' target for -s (and
> I guess for -c, too, just for consistency) to replica/applylog. I have
> a patch for this, and I'll submit it shortly. This flag would make
> applylog ignore all client-side (or server-side) changes, though I
> suppose the latter is already possible.

Since the arguments to -c and -s are just string prefixes, you 
can use -c '' or -s '' already.  I admit it is non-standard.

However, you'd be better off not using replica/pull as
the input to a differ, but instead using replica's change file
/n/sources/plan9/dist/replica/plan9.log.  Replica(8) describes
the format:

   A replica is further described on the server by a textual
   log listing creation and deletion of files and changes to
   file contents and metadata.  Each line is of the form:

      time gen verb path serverpath mode uid gid mtime length

   The time and gen fields are both decimal numbers, providing
   an ordering for log entries so that incremental tools need
   not process the whole log each time they are run.  The verb,
   a single character, describes the event: addition of a file
   (a), deletion of a file (d), a change to a file's contents
   (c), or a change to a file's metadata (m).  Path is the file
   path on the client; serverpath the path on the server (these
   are different when the optional fifth field in a proto file
   line is given; see proto(2)). Mode, uid, gid, and mtime are
   the files metadata as in the Dir structure (see stat(5)).
   For deletion events, the metadata is that of the deleted
   file.  For other events, the metadata is that after the
   event.

It would be easiest to pick the two most recent dumps in
/n/sourcesdump, diff the plan9.log files to pick out the
new lines, and then run diffs between the two different
dump roots.  This is what we used to do when we (I) annotated
all the changes to produce /n/sources/extra/changes.
But it was far too much work to do the annotations and
not enough people cared, so we stopped that particular
experiment.

erik:
> i have also noticed that replica/applylog has a problem.  when i started
> experimenting with copying history from our old fileserver to the new
> one, i started using replica/updatedb and replica/applylog.  updatedb
> worked very well, but applylog hung for me pretty consistantly.

Did you ever use acid to get a stack trace from the `hung' applylogs?
The only threading in applylog is an implementation of something
like fcp to copy files using multiple outstanding 9P read requests.
Since no one else seems to have had problems, I would guess that
there were just some requests that made your file server thrash.
But stack traces would make the answer very clear.

ron:
> My take is that bringing in mercurial, and then using mercurial with
> mounted file systems, instead of ssh, would be quite neat. And, we are
> close to having it. We've got python, almost. We've just about got
> python modules -- actually, I've had them for months. Lots of bits
> work that did not used to. We really are pretty close.

Echoing Ron, I think having Mercurial would be great and doable.
If someone makes a Mercurial client work, I will be happy to make
a Mercurial repository that mirrors sources automatically.

Also echoing Ron, a venti-based SCM sounds similar to git,
which is an SCM built on top of a hash-addressed object store
(that happens not to be named venti).  It would be nice to know
you're not reinventing git, especially since in my experience
the fact that git is hash-addressed makes many things a lot
harder and slower (although I am sure it has advantages).

devon:
> In fact, I seem to recall the last time Uriel was yelling about this
> on the list, Russ offered to do exactly this. Is this offer still
> available? It'd save me some time and effort trying to figure out how
> to revert a replica/pull.

I put a copy of the script we used to use in
/n/sources/contrib/rsc/makemail.  Use at your own risk.
It probably deserves to be rewritten in a better language.

devon again:
> So maybe to go further down that path (and I would
> _love_ to get input from Bell Labs people because they're really the
> ones that have the power to yay or nay any of this), is replica/*
> still an ideal manner for getting updates? Are there potentially
> better ways to do this?

There are things that replica doesn't do very well.  I wish you
could tell it to back up, or to back out local changes, and so on,
but the core functionality works well, and I would be wary of
falling into the CADT Model trap (ask Google).

devon again:
> The scenario being, I want to test my hypothetical replica/applylog
> action parser / diff generator. I run pull, which grabs stuff, but
> I've made changes to xyz.c and that file doesn't get modified because
> of local changes. So I want to roll back my log so I can run pull
> again with -s /sys/src/cmd/xyz.c

Your thinking is stuck in the maze of twisty little passages
that is modern Unix and its version control systems.
Replica is a file distribution mechanism, not a version 
control system.

On Plan 9, one would do this with the dump file system.
9fs sourcesdump and look around -- you've got all the info
there you could possibly want to produce diffs.  You don't
even need to copy anything to your local machine.

Russ

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, erik quanstrom <quanstro@coraid.com>:
> > available? It'd save me some time and effort trying to figure out how
> > to revert a replica/pull.
> >
> > Speaking of which, how _does_ one replica/pull to a previous date?
> >
> > --dho
>
> there are lots of ways to do this, but a particular senerio would be helpful.
> for example, if you wanted to pull only up to date x, then you could just edit
> the log and delete entries that come later.  you can always mount sourcesdump
> and either copy or mount what you'd like if you have a problem with a particular
> package.
>
> - erik

The scenario being, I want to test my hypothetical replica/applylog
action parser / diff generator. I run pull, which grabs stuff, but
I've made changes to xyz.c and that file doesn't get modified because
of local changes. So I want to roll back my log so I can run pull
again with -s /sys/src/cmd/xyz.c

This wouldn't be an issue if we added a '*' option to replica/pull for
the -c and -s flags. But I'll send a patch for that shortly and see if
that gets committed.

--dho

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, erik quanstrom <quanstro@coraid.com>:
> On Mon Apr  9 11:08:49 EDT 2007, devon.odell@gmail.com wrote:
> > > what do you mean by this?
> >
> > ``I don't know'' and I don't really care either. Apparently he doesn't
> > like that it takes forever and that it has the habit of wiping things
> > out sometimes. That's my understanding of his explanation, anyway.
> >
> > I'd rather not go down this thread though because I don't really want
> > to talk about why this work isn't happening, but rather what can be
> > done to make it happen and actually getting it done.
>
> rather a worked-up response for a simple question.

I didn't mean to come across as inflamed or sarcastic, rather to imply
that I don't really care what Uriel's complaints are about because
they don't address the issue. Also, my experience with this list is
that tangents like this end up going off into oblivion with no
resolution and I don't want that to happen for this issue. So again,
apologies if I seemed worked up or irritated; that wasn't the
intention.

> for the record, i think many people on this list are interested in
> fixing things.  if this is a pet project of yours, why don't you work
> on fixing it?  unfortunately, i don't have very much spare time right now.
> there are drivers to write.

I was originally trying to get Uriel to put his code where his mouth
is (or at least where it used to be). And I'm trying to work on fixing
it -- right now by analyzing what has been said and done in the past,
since this is certainly not a new `issue'. If there are already
utilities to grab diffs like this and / or a means to send diffs in
the mail, that would be an easy and ideal solution and would save time
:).

I know that discussing it to death on a mailing list is more on the
wasting time scale and if I think it starts getting to that point,
I'll just quit and write my own stuff to do it. (And to some degree, I
already am with vcs)

> i have also noticed that replica/applylog has a problem.  when i started
> experimenting with copying history from our old fileserver to the new
> one, i started using replica/updatedb and replica/applylog.  updatedb
> worked very well, but applylog hung for me pretty consistantly.
>
> my thought was that applylog has a threading deadlock, but i didn't spent
> much time thinking about it.  one thing that does help quite a bit  is to use
> replica/compactdb on your local database.  that is in /dist/replica/plan9.db.
>
> - erik

I've only just started getting into the replica code, so I'm not sure
what this would be indicative of (I'm sure your understanding is far
better than mine). So maybe to go further down that path (and I would
_love_ to get input from Bell Labs people because they're really the
ones that have the power to yay or nay any of this), is replica/*
still an ideal manner for getting updates? Are there potentially
better ways to do this?

--dho

Re: [9fans] bell-labs website and plan9

[erik quanstrom]

> available? It'd save me some time and effort trying to figure out how
> to revert a replica/pull.
> 
> Speaking of which, how _does_ one replica/pull to a previous date?
> 
> --dho

there are lots of ways to do this, but a particular senerio would be helpful.
for example, if you wanted to pull only up to date x, then you could just edit
the log and delete entries that come later.  you can always mount sourcesdump
and either copy or mount what you'd like if you have a problem with a particular
package.

- erik

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, ron minnich <rminnich@gmail.com>:
> possibly inflammatory question: the ideas I've seen proposed so far
> don't seem superior in any way to, e.g., mercurial.
>
> Please note, I am sorry if this letter sounds offensive, I do not
> intend it to be.
>
> If, somehow, the proposed ideas represent a huge leap in 'something',
> it would be nice to know what 'something' is; I'm not getting it from
> the descriptions. So far, it all sounds like a bit of a kludge to
> cover for our lack of tools and time to build them.
>
> If, instead, the proposed ideas are more of the usual "There is a
> better non-Plan 9 system but we don't run it because we (a) can't
> compile it (b) don't have software to run it (c) we didn't invent it
> (i.e. NIH)", well, then, it's time to start bringing those better
> systems in, and put our efforts elsewhere.

That's in part what vcs is supposed to be, but I don't think it's
really going to be good for anything other than small projects any
time soon. I've asked you several times off-list about the status of
Python and hg and getting these to work, but I haven't received a
response yet, so I don't know if you got them. I got hg partially
working, but then ran into dependencies on signal handlers and didn't
take the time to remove them / change them to use Notes. Any
information on the hg status and how one can help with that would be
nice.

> My take is that bringing in mercurial, and then using mercurial with
> mounted file systems, instead of ssh, would be quite neat. And, we are
> close to having it. We've got python, almost. We've just about got
> python modules -- actually, I've had them for months. Lots of bits
> work that did not used to. We really are pretty close.
>
> Take the best of both worlds, in other words, and create something
> new. But, if you can take other's good work, that can be a timesaver.
>
> thanks
>
> ron

As far as an SCM goes, it would be nice to have one, but I'm sure it
has to run in Plan 9. The big problem that I have with hg and my own
vcs is that neither is really suited for maintaining multiple
branches. At this point, vcs is small and incomplete enough to make
this doable, but it's going to be way too slow anyway. And it's not
going to be in a complete enough state to manage an OS for quite some
time to come, either.

--dho

Re: [9fans] bell-labs website and plan9

[erik quanstrom]

On Mon Apr  9 11:08:49 EDT 2007, devon.odell@gmail.com wrote:
> > what do you mean by this?
> 
> ``I don't know'' and I don't really care either. Apparently he doesn't
> like that it takes forever and that it has the habit of wiping things
> out sometimes. That's my understanding of his explanation, anyway.
> 
> I'd rather not go down this thread though because I don't really want
> to talk about why this work isn't happening, but rather what can be
> done to make it happen and actually getting it done.

rather a worked-up response for a simple question.

for the record, i think many people on this list are interested in
fixing things.  if this is a pet project of yours, why don't you work
on fixing it?  unfortunately, i don't have very much spare time right now.
there are drivers to write.

i have also noticed that replica/applylog has a problem.  when i started
experimenting with copying history from our old fileserver to the new
one, i started using replica/updatedb and replica/applylog.  updatedb
worked very well, but applylog hung for me pretty consistantly.

my thought was that applylog has a threading deadlock, but i didn't spent
much time thinking about it.  one thing that does help quite a bit  is to use 
replica/compactdb on your local database.  that is in /dist/replica/plan9.db.

- erik

Re: [9fans] bell-labs website and plan9

[ron minnich]

possibly inflammatory question: the ideas I've seen proposed so far
don't seem superior in any way to, e.g., mercurial.

Please note, I am sorry if this letter sounds offensive, I do not
intend it to be.

If, somehow, the proposed ideas represent a huge leap in 'something',
it would be nice to know what 'something' is; I'm not getting it from
the descriptions. So far, it all sounds like a bit of a kludge to
cover for our lack of tools and time to build them.

If, instead, the proposed ideas are more of the usual "There is a
better non-Plan 9 system but we don't run it because we (a) can't
compile it (b) don't have software to run it (c) we didn't invent it
(i.e. NIH)", well, then, it's time to start bringing those better
systems in, and put our efforts elsewhere.

My take is that bringing in mercurial, and then using mercurial with
mounted file systems, instead of ssh, would be quite neat. And, we are
close to having it. We've got python, almost. We've just about got
python modules -- actually, I've had them for months. Lots of bits
work that did not used to. We really are pretty close.

Take the best of both worlds, in other words, and create something
new. But, if you can take other's good work, that can be a timesaver.

thanks

ron

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, erik quanstrom <quanstro@coraid.com>:
> On Mon Apr  9 10:50:37 EDT 2007, devon.odell@gmail.com wrote:
> > I've been asking uriel to write an rc script for me to run, but he
> > doesn't seem to want to. The premise is to parse output of
> > replica/pull and provide diff -c output for all changed non-binary
> > files. He doesn't seem to want to do this because he says he doesn't
> > like how replica/pull [doesn't] work.
> >
>
> what do you mean by this?

``I don't know'' and I don't really care either. Apparently he doesn't
like that it takes forever and that it has the habit of wiping things
out sometimes. That's my understanding of his explanation, anyway.

I'd rather not go down this thread though because I don't really want
to talk about why this work isn't happening, but rather what can be
done to make it happen and actually getting it done.

> - erik

--dho

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, Devon H. O'Dell <devon.odell@gmail.com>:
> 2007/4/9, Benn Newman <newmanbe@sdf.lonestar.org>:
> > On 05/04/07, geoff@plan9.bell-labs.com <geoff@plan9.bell-labs.com> wrote:
> > > > it seems like Bell Labs/Lucent doesn't really do anything with Plan
> > > > 9 these days except host the server.
> > >
> > > You obviously haven't done a replica/pull lately.  ☺
> > >
> > It would be really nice if you made a ChangeLog or used patch(1) so
> > the rest of us could easily tell what you did. ☺
>
> If there's a better way for me to get diffs (or if they can be piped
> directly to my Inbox), I'd be glad to maintain a ChangeLog.

In fact, I seem to recall the last time Uriel was yelling about this
on the list, Russ offered to do exactly this. Is this offer still
available? It'd save me some time and effort trying to figure out how
to revert a replica/pull.

Speaking of which, how _does_ one replica/pull to a previous date?

--dho

Re: [9fans] bell-labs website and plan9

[erik quanstrom]

On Mon Apr  9 10:50:37 EDT 2007, devon.odell@gmail.com wrote:
> I've been asking uriel to write an rc script for me to run, but he
> doesn't seem to want to. The premise is to parse output of
> replica/pull and provide diff -c output for all changed non-binary
> files. He doesn't seem to want to do this because he says he doesn't
> like how replica/pull [doesn't] work.
> 

what do you mean by this?

- erik

Re: [9fans] bell-labs website and plan9

[Devon H. O'Dell]

2007/4/9, Benn Newman <newmanbe@sdf.lonestar.org>:
> On 05/04/07, geoff@plan9.bell-labs.com <geoff@plan9.bell-labs.com> wrote:
> > > it seems like Bell Labs/Lucent doesn't really do anything with Plan
> > > 9 these days except host the server.
> >
> > You obviously haven't done a replica/pull lately.  ☺
> >
> It would be really nice if you made a ChangeLog or used patch(1) so
> the rest of us could easily tell what you did. ☺

I've been asking uriel to write an rc script for me to run, but he
doesn't seem to want to. The premise is to parse output of
replica/pull and provide diff -c output for all changed non-binary
files. He doesn't seem to want to do this because he says he doesn't
like how replica/pull [doesn't] work.

Basically, the premise is to take the output of replica/pull, find
non-binary files, run history on sourcesdump for those files, diff -c
the two, and plop that in some file. I'll read through it and create
comments on what the changes are. If I have a question, I'll ask
someone. And I'll manually maintain a ChangeLog of sorts.

One thing that this will need is support for a '*' target for -s (and
I guess for -c, too, just for consistency) to replica/applylog. I have
a patch for this, and I'll submit it shortly. This flag would make
applylog ignore all client-side (or server-side) changes, though I
suppose the latter is already possible.

If there's a better way for me to get diffs (or if they can be piped
directly to my Inbox), I'd be glad to maintain a ChangeLog.

That said, I've also been working on a vac-based SCM. See also
/n/sources/contrib/dho/vcs/

--dho

> --
> Benn Newman

Re: [9fans] bell-labs website and plan9

[Benn Newman]

On 05/04/07, geoff@plan9.bell-labs.com <geoff@plan9.bell-labs.com> wrote:
> > it seems like Bell Labs/Lucent doesn't really do anything with Plan
> > 9 these days except host the server.
>
> You obviously haven't done a replica/pull lately.  ☺
>
It would be really nice if you made a ChangeLog or used patch(1) so
the rest of us could easily tell what you did. ☺
-- 
Benn Newman

Re: [9fans] bell-labs website and plan9

[geoff]

> it seems like Bell Labs/Lucent doesn't really do anything with Plan
> 9 these days except host the server.

You obviously haven't done a replica/pull lately.  ☺

Re: [9fans] bell-labs website and plan9

[W B Hacker]

John Floren wrote:
> On 4/5/07, pedro henrique antunes de oliveira <ph.rpguo@gmail.com> wrote:
>> why the www.bell-labs.com doesnt talks too much about plan9 (I, 
>> really, dont
>> know if it talks about it, i never found anything about it there).
>>
>>  Anyone knows?
>>
> 
> Because they have other fish to fry? I don't know for sure, but it
> seems like Bell Labs/Lucent doesn't really do anything with Plan 9
> these days except host the server.
> Can somebody clue us in on this? Maybe one of the earlier coders can
> give some info?
> Thanks
> 
> John

The life-cycle is explained on the website:

http://plan9.bell-labs.com/plan9/

See also:

http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs

http://www.vitanuova.com/company/background.html

Bill

Re: [9fans] bell-labs website and plan9

[pedro henrique antunes de oliveira]

[-- Attachment #1: Type: text/plain, Size: 6 bytes --]

nice

[-- Attachment #2: Type: text/html, Size: 10 bytes --]

Re: [9fans] bell-labs website and plan9

[Fazlul Shahriar]

http://www.alcatel-lucent.com/wps/portal/!ut/p/kcxml/04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLd4y38DIESYGZzgH6kShiBvGOCJEgfW99X4_83FT9AP2C3NCIckdHRQA0CwaZ/delta/base64xml/L3dJdyEvd0ZNQUFzQUMvNElVRS82X0FfNDdE

scroll to the bottom. It's there, just hidden somewhere.

fhs

Re: [9fans] bell-labs website and plan9

[pedro henrique antunes de oliveira]

[-- Attachment #1: Type: text/plain, Size: 169 bytes --]

i wasnt talk about this.
i just want to know why in the bell-labs website
www.bell-labs.com
there arent anything about plan9 (if there are, they are almost nothing)

[-- Attachment #2: Type: text/html, Size: 224 bytes --]

Re: [9fans] bell-labs website and plan9

[W B Hacker]

pedro henrique antunes de oliveira wrote:
> why the www.bell-labs.com doesnt talks too much about plan9 (I, really, 
> dont
> know if it talks about it, i never found anything about it there).
> 
> Anyone knows?
> 

Dunno why you are having trouble finding it.

First hit Google produces, out of 'about 2,130,000' should lead you to the rest:

http://plan9.bell-labs.com/plan9/

Bill

Re: [9fans] bell-labs website and plan9

[John Floren]

On 4/5/07, pedro henrique antunes de oliveira <ph.rpguo@gmail.com> wrote:
> why the www.bell-labs.com doesnt talks too much about plan9 (I, really, dont
> know if it talks about it, i never found anything about it there).
>
>  Anyone knows?
>

Because they have other fish to fry? I don't know for sure, but it
seems like Bell Labs/Lucent doesn't really do anything with Plan 9
these days except host the server.
Can somebody clue us in on this? Maybe one of the earlier coders can
give some info?
Thanks

John
-- 
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn

[9fans] bell-labs website and plan9

[pedro henrique antunes de oliveira]

[-- Attachment #1: Type: text/plain, Size: 163 bytes --]

why the www.bell-labs.com doesnt talks too much about plan9 (I, really, dont
know if it talks about it, i never found anything about it there).

Anyone knows?

[-- Attachment #2: Type: text/html, Size: 214 bytes --]