[9fans] FS for sharing between Linux and Plan 9

[9fans] FS for sharing between Linux and Plan 9

[Jeremy Jackins]

Hello, first off I'm new to this list and OS so I apologize for my
ignorance on many things.
I have an external HDD that I would like to use as a common backup and
synchronization point between a Linux machine and a Plan 9 machine.
Until recently I was using btrfs on this disk for the snapshot
support, and was only sharing it between Linux machines.

I like Fossil/Venti, but if I use this on the HDD I lose the ability
to take the disk outside of my home network and plug it into a Linux
machine (right?)
I could use something like FAT which is supported by everything but
then I lose the nice features of a modern file system like Fossil or
btrfs.

Is there an obvious solution which I am missing? My goal essentially
is to keep all my important files in this central location, easily
update it with new changes from machine A, then update machine B with
these changes. If something goes wrong, I would like to be able to
restore my files as they were at some (more or less) arbitrary point
in time to machine X. If possible I'd like to be able to unplug it and
move it around, maybe to somewhere where there isn't a Plan 9 machine.

Thanks,
Jeremy

Re: [9fans] FS for sharing between Linux and Plan 9

[smiley]

Jeremy Jackins <jeremyjackins@gmail.com> writes:

> Until recently I was using btrfs on this disk for the snapshot
> support, and was only sharing it between Linux machines.

I know it's kinda OT, but was there a particular reason you wanted more
than btrfs could offer?

> I like Fossil/Venti, but if I use this on the HDD I lose the ability
> to take the disk outside of my home network and plug it into a Linux
> machine (right?)

There's a port of Plan 9 utilities to Linux userspace.  It's called Plan
9 Port (aka p9p).  See http://swtch.com/plan9port/.  Linux machines with
p9p should be able to talk to fossil/venti.

Of course, if your "external drive" contains a CPU (i.e., one of the NAS
boxen being sold as "external drives"), the "drive" itself may be able
to run 9p.  :)

> Is there an obvious solution which I am missing? My goal essentially
> is to keep all my important files in this central location, easily
> update it with new changes from machine A, then update machine B with
> these changes.

Would replica meet your needs?  Under some circumstances, rsync
outperforms replica.  For some purposes, a DVCS such as Mercurial (which
has also been ported to P9) is more appropriate.  It depends on a mix of
factors.

--
+---------------------------------------------------------------+
|Smiley       <smiley@icebubble.org>    PGP key ID:    BC549F8B |
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA  3489 DAB7 555A BC54 9F8B|
+---------------------------------------------------------------+

Re: [9fans] FS for sharing between Linux and Plan 9

[andrey mirtchovski]

I've been using the ufs version written in Go by Latchesar Ionkov
(go9p.googlecode.com) recently with great success. It connects the
underlying OS with Plan 9 on a hosted virtual box image, linked via a
bridged network. It's not completely bug-free so you can't expect to,
for example, do a complete build of Go over it, but it is at worst
1.5* slower and at best 2* faster than a native fossil.

Re: [9fans] FS for sharing between Linux and Plan 9

[Jeremy Jackins]

On Sun, Oct 30, 2011 at 7:03 PM,  <smiley@icebubble.org> wrote:
> I know it's kinda OT, but was there a particular reason you wanted more
> than btrfs could offer?

No, I like btrfs, but as far as I know Plan 9 doesn't support it.

> There's a port of Plan 9 utilities to Linux userspace.  It's called Plan
> 9 Port (aka p9p).  See http://swtch.com/plan9port/.  Linux machines with
> p9p should be able to talk to fossil/venti.

Okay, thanks, I've been using p9p to play around with acme and such on
linux, but I didn't know it could do this.

> Of course, if your "external drive" contains a CPU (i.e., one of the NAS
> boxen being sold as "external drives"), the "drive" itself may be able
> to run 9p.  :)

Hm, I don't think so. Maybe external drive was the wrong term, what I
have is actually just a little enclosure thing that I pop a normal
3.5" drive into and power on, which then has a USB port.

> Would replica meet your needs?  Under some circumstances, rsync
> outperforms replica.  For some purposes, a DVCS such as Mercurial (which
> has also been ported to P9) is more appropriate.  It depends on a mix of
> factors.

Thanks, I'll have a look at replica.

On Sun, Oct 30, 2011 at 7:35 PM, andrey mirtchovski
<mirtchovski@gmail.com> wrote:
> I've been using the ufs version written in Go by Latchesar Ionkov
> (go9p.googlecode.com) recently with great success. It connects the
> underlying OS with Plan 9 on a hosted virtual box image, linked via a
> bridged network. It's not completely bug-free so you can't expect to,
> for example, do a complete build of Go over it, but it is at worst
> 1.5* slower and at best 2* faster than a native fossil.

Interesting. Incidentally I've been wondering what the status of Go on
Plan 9 is lately. cat-v.org tells me "Status: Being integrated with
mainline", but they haven't updated in a couple weeks and I can't seem
to find much other info.

Re: [9fans] FS for sharing between Linux and Plan 9

[smiley]

smiley@icebubble.org writes:

> Jeremy Jackins <jeremyjackins@gmail.com> writes:

>> I like Fossil/Venti, but if I use this on the HDD I lose the ability
>> to take the disk outside of my home network and plug it into a Linux
>> machine (right?)
>
> There's a port of Plan 9 utilities to Linux userspace.  It's called Plan
> 9 Port (aka p9p).  See http://swtch.com/plan9port/.  Linux machines with
> p9p should be able to talk to fossil/venti.

Hm.  It looks like I might have been wrong about that.  Digging a little
deeper, it seems like plan9port doesn't have any fdisk, prep, or fossil.
Is that correct?  (And if so, why?)  I know that Linux doesn't have any
kernel-level support for Plan 9 partitions or the fossil file system.
But can't that stuff be done in userspace, too?

--
+---------------------------------------------------------------+
|Smiley       <smiley@icebubble.org>    PGP key ID:    BC549F8B |
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA  3489 DAB7 555A BC54 9F8B|
+---------------------------------------------------------------+

Re: [9fans] FS for sharing between Linux and Plan 9

[Russ Cox]

On Fri, Nov 4, 2011 at 4:50 PM,  <smiley@icebubble.org> wrote:
> Hm.  It looks like I might have been wrong about that.  Digging a little
> deeper, it seems like plan9port doesn't have any fdisk, prep, or fossil.
> Is that correct?  (And if so, why?)  I know that Linux doesn't have any
> kernel-level support for Plan 9 partitions or the fossil file system.
> But can't that stuff be done in userspace, too?

You can port it if you want.  Plan9port tries to integrate well with
the host operating system.  Part of that means using the same
file tree, and the host OS cannot talk to fossil.

There is disk code that knows how to read Plan 9 partition tables.
The venti server uses it to let you write /dev/sdC0:arenas.

Russ

Re: [9fans] FS for sharing between Linux and Plan 9

[David du Colombier]

> > Hm.  It looks like I might have been wrong about that.  Digging a
> > little deeper, it seems like plan9port doesn't have any fdisk,
> > prep, or fossil. Is that correct?  (And if so, why?)  I know that
> > Linux doesn't have any kernel-level support for Plan 9 partitions
> > or the fossil file system. But can't that stuff be done in
> > userspace, too?
> 
> You can port it if you want.  Plan9port tries to integrate well with
> the host operating system.  Part of that means using the same
> file tree, and the host OS cannot talk to fossil.
> 
> There is disk code that knows how to read Plan 9 partition tables.
> The venti server uses it to let you write /dev/sdC0:arenas.

I started a port of Fossil to plan9port few months ago.

It seems to be working fine, but:

 - the console doesn't work,
 - it doesn't handle Plan 9 partitions tables,
 - I haven't wrote the manuals.

I hope to fix all of these problems in the future. The lack of
console is obviously a big issue.

Beware this is not exactly the Plan 9 Fossil. This is the modified
Fossil I use on Plan 9. This Fossil is based on libventi and
libthread, instead of liboventi.
This is why liboventi haven't been ported.

I considered it is too experimental to be submitted to plan9port,
but the patches are available online:

 - [1] fossil-plan9: import Fossil from Plan 9,
 - [2] fossil-new: modified Fossil,
 - [3] fossil-new-p9p: required plan9port changes on Fossil.

However, I recently submitted two needed patches to libventi:

 - [4] libventi-redial: add redial and reconn functions,
 - [5] libventi-noarchive: add VtEntryNoArchive.

These two patches add features, currently part of liboventi,
missing from libventi, but useful to Fossil.

[1] http://www.9legacy.org/p9p/patch/fossil-plan9.diff
[2] http://www.9legacy.org/p9p/patch/fossil-new.diff
[3] http://www.9legacy.org/p9p/patch/fossil-new-p9p.diff
[4] http://codereview.appspot.com/download/issue5325046_1.diff
[5] http://codereview.appspot.com/download/issue5298088_1.diff

If you want to contribute, I will be happy to receive patches
and bug reports.

-- 
David du Colombier

Re: [9fans] FS for sharing between Linux and Plan 9

[smiley]

David du Colombier <0intro@gmail.com> writes:

> I started a port of Fossil to plan9port few months ago.

> Beware this is not exactly the Plan 9 Fossil. This is the modified
> Fossil I use on Plan 9. This Fossil is based on libventi and

> I considered it is too experimental to be submitted to plan9port,
> but the patches are available online:

>From this, I understand that you've concocted an alpha (beta?) version
of fossil for plan9port.  I'm wondering: Is the on-disk format of this
"modified Fossil" compatible with P9 fossil?  If so, it might work as
the Plan9-Linux-universal file system the OP's looking for after all
(less any bugs he might encounter in such young code).

> I hope to fix all of these problems in the future. The lack of
> console is obviously a big issue.

For the OP's purposes (sharing an fs between Plan 9 and Linux), this
might actually be tenable.  The fossil console is typically used for
administrative tasks, not generally during routine use (i.e., just
storing and reading back files).  If the OP's needs for fossilcons can
wait until he's at a P9 box, it might be a practical setup... at least
until the console is ported.

--
+---------------------------------------------------------------+
|Smiley       <smiley@icebubble.org>    PGP key ID:    BC549F8B |
|Fingerprint: 9329 DB4A 30F5 6EDA D2BA  3489 DAB7 555A BC54 9F8B|
+---------------------------------------------------------------+

Re: [9fans] FS for sharing between Linux and Plan 9

[David du Colombier]

> From this, I understand that you've concocted an alpha (beta?)
> version of fossil for plan9port.

Actually, it was an easy job. I did it in few hours and, as
you can see, the modifications were trivial, thank to the
very good environment offered by plan9port.

It's unfinished just because I haven't spent much time on the
remaining issues.

> I'm wondering: Is the on-disk format of this "modified Fossil"
> compatible with P9 fossil?

Yes of course. The file system format is identical.
Actually, the modifications are only internal and there
are no user-visible changes.

Keep in mind I did these changes primarily for Plan 9. My goal
was firstly to get rid of the liboventi and secondly to benefit
from the libthread to replace locks with channels.

The version currently available is the first part. It doesn't
address any of the known Fossil issues, like the well-known
deadlock on flushwait, since I still haven't figured how to
properly fix this problem using the current locking mechanisms.

I am currently working on the second part and I already fixed
these problems by replacing some locks with channels, but it
is currently too experimental to be distributed.

It just happened I ported my modified Fossil to plan9port and
not the original one.

> If so, it might work as the Plan9-Linux-universal file system the
> OP's looking for after all (less any bugs he might encounter in such
> young code).

Yes, it allows some Unix to read and write to a Plan 9 file system.

> For the OP's purposes (sharing an fs between Plan 9 and Linux), this
> might actually be tenable.  The fossil console is typically used for
> administrative tasks, not generally during routine use (i.e., just
> storing and reading back files).  If the OP's needs for fossilcons can
> wait until he's at a P9 box, it might be a practical setup... at least
> until the console is ported.

The lack of console is more problematic than you think.

It's not a problem when you start Fossil, because you can use options
-f and -c to execute commands from a file or command-line.
However, you will not be able, for example, to execute "sync" or "halt"
on a running file system, which is somewhat embarrassing.

That's why I want to fix this problem.

--
David du Colombier

Re: [9fans] FS for sharing between Linux and Plan 9

[David du Colombier]

Command-line option -t is now working, so you can use
the console this way.

--
David du Colombier