[9fans] fuse bashing

[9fans] fuse bashing

[Russ Cox]

Fuse has a nice crunchy layer of GNU crud around it
which makes it hard to swallow, but at its core, it's
very close to 9P (or the version 8 network file system).

You open /dev/fuse, hand the fd to the kernel to mount
it somewhere, and then you read and write fuse messages
on the fd.  All the kernel vfs operations end up as fuse 
messages, including close.  (It just isn't used in examples,
but it does exist.  It's called release.)  The main failing I see
from a protocol standpoint seems to be the lack of a flush
message.  Instead, fuse just puts interrupted requests on
a list and assumes they'll come back eventually.  
This wouldn't be appropriate for things like interrupting
a read from /dev/tty.  For mounting file systems on a 
Unix system, fuse is probably a better protocol than 9P.
It appears to match the kernel better, so the implementation
should be simpler.  (As soon as you want to talk between
different operating systems or send the messages over
the network, that's where 9P starts to make more sense.)

Fuse's main problem is that they bundle the protocol inside
the library loop, so what you see from the outside is the
equivalent of 9P+lib9p.  You don't ever get to see the protocol
by itself.  That's often good enough, but sometimes it's nice
to deal with the raw messages.  Another big problem with
fuse is the lack of easily-found documentation.  

If you download a fuse distribution and look in
kernel/fuse_kernel.h you can see the messages.

Russ

Re: [9fans] fuse bashing

[Ronald G Minnich]

In fact, fuse may be just good enough, and v9fs came in just late 
enough, that in the long term, fuse is what "wins" on linux, in the 
sense that most people may end up using fuse, since there are already 
pretty good examples for people to follow. We'll see.

ron

Re: [9fans] fuse bashing

[David Leimbach]

On 1/23/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> In fact, fuse may be just good enough, and v9fs came in just late
> enough, that in the long term, fuse is what "wins" on linux, in the
> sense that most people may end up using fuse, since there are already
> pretty good examples for people to follow. We'll see.
>
> ron
>

FUSE also currently works fine on FreeBSD 6.  I have it here.

Dave

Re: [9fans] fuse bashing

[Eric Van Hensbergen]

On 1/23/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> In fact, fuse may be just good enough, and v9fs came in just late
> enough, that in the long term, fuse is what "wins" on linux, in the
> sense that most people may end up using fuse, since there are already
> pretty good examples for people to follow. We'll see.
>

Lucho's made some good progress on this front.  We've proposed a
tutorial at OLS (which may also generate a DeveloperWorks tutorial)
which will cover how to write synthetic file systems for 9P and also
cover some of the tradeoffs between 9P and FUSE.

       -eric

Re: [9fans] fuse bashing

[Bruce Ellis]

More protocols!  More conversions!

Call me old-fashioned but i still use styx whenever possible.
Some people like metallica, i like berlioz.

brucee

On 1/24/06, Eric Van Hensbergen <ericvh@gmail.com> wrote:
> On 1/23/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> > In fact, fuse may be just good enough, and v9fs came in just late
> > enough, that in the long term, fuse is what "wins" on linux, in the
> > sense that most people may end up using fuse, since there are already
> > pretty good examples for people to follow. We'll see.
> >
>
> Lucho's made some good progress on this front.  We've proposed a
> tutorial at OLS (which may also generate a DeveloperWorks tutorial)
> which will cover how to write synthetic file systems for 9P and also
> cover some of the tradeoffs between 9P and FUSE.
>
>       -eric
>

Re: [9fans] fuse bashing

[David Leimbach]

On 1/23/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> In fact, fuse may be just good enough, and v9fs came in just late
> enough, that in the long term, fuse is what "wins" on linux, in the
> sense that most people may end up using fuse, since there are already
> pretty good examples for people to follow. We'll see.
>
> ron
>

Well are there differences in maximum file sizes exposable via 9p vs
FUSE?  I'd like to see more of a feature breakdown.

I don't know too much about the limitations of 9P... and this choir
isn't likely to sing them [no offense but it is called 9fans after
all]

Dave

Re: [9fans] fuse bashing

[Ronald G Minnich]

David Leimbach wrote:

> I don't know too much about the limitations of 9P... and this choir
> isn't likely to sing them [no offense but it is called 9fans after
> all]

The big complaints I know of so far on 9P are

- there is no posix file locking (sorry, but people want it) although 
the 'only allow one open at a time' is a pretty damned good substitute

- no ACLs (I'm convinced that the stat and wstat could be trivially
	extended to support this --- 9p2000.acl)

- doesn't fit linux vfs semantics too well (just a joke, son, but true
    too -- sometimes you have to fit a good thing onto a broken thing)

That's about all I've hit so far. I spent about 5 years hacking on nfs, 
and I have to say 9p is a way better protocol.

ron

Re: [9fans] fuse bashing

[Dan Cross]

On Mon, Jan 23, 2006 at 01:33:35PM -0700, Ronald G Minnich wrote:
> The big complaints I know of so far on 9P are
> 
> - there is no posix file locking (sorry, but people want it) although 
> the 'only allow one open at a time' is a pretty damned good substitute
> 
> - no ACLs (I'm convinced that the stat and wstat could be trivially
> 	extended to support this --- 9p2000.acl)
> 
> - doesn't fit linux vfs semantics too well (just a joke, son, but true
>    too -- sometimes you have to fit a good thing onto a broken thing)
> 
> That's about all I've hit so far. I spent about 5 years hacking on nfs, 
> and I have to say 9p is a way better protocol.

I'm sure some crackhead somewhere has added ``doesn't do readlink or
symlink'' to that list.

	- Dan C.

Re: [9fans] fuse bashing

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

On 1/23/06, Dan Cross <cross@math.psu.edu> wrote:
>
> On Mon, Jan 23, 2006 at 01:33:35PM -0700, Ronald G Minnich wrote:
> > The big complaints I know of so far on 9P are
> >
> > - there is no posix file locking (sorry, but people want it) although
> > the 'only allow one open at a time' is a pretty damned good substitute
> >
> > - no ACLs (I'm convinced that the stat and wstat could be trivially
> >       extended to support this --- 9p2000.acl)
> >
> > - doesn't fit linux vfs semantics too well (just a joke, son, but true
> >    too -- sometimes you have to fit a good thing onto a broken thing)
> >
> > That's about all I've hit so far. I spent about 5 years hacking on nfs,
> > and I have to say 9p is a way better protocol.
>
> I'm sure some crackhead somewhere has added ``doesn't do readlink or
> symlink'' to that list.



They might if they're on a unix system with no private namespaces.

[-- Attachment #2: Type: text/html, Size: 1322 bytes --]

Re: [9fans] fuse bashing

[Ronald G Minnich]

Dan Cross wrote:

> I'm sure some crackhead somewhere has added ``doesn't do readlink or
> symlink'' to that list.


When I first did v9fs in 1996 or so, I left out symlink/readlink. Life 
was impossible. I had to put 'em in :-(

They are also in the version on 2.6.15

hey, you and I had this 'on crack' discussion last time.

ron

Re: [9fans] fuse bashing

[Enache Adrian]

On Mon, Jan 23, 2006 at 06:28:21PM -0500, Dan Cross wrote:
> > - there is no posix file locking (sorry, but people want it) although 
> > the 'only allow one open at a time' is a pretty damned good substitute
> > 
> > - no ACLs (I'm convinced that the stat and wstat could be trivially
> > 	extended to support this --- 9p2000.acl)
> > 
> > - doesn't fit linux vfs semantics too well (just a joke, son, but true
> >    too -- sometimes you have to fit a good thing onto a broken thing)
> > 
> I'm sure some crackhead somewhere has added ``doesn't do readlink or
> symlink'' to that list.

- doesn't have a rename/move operation.

Regards,
Adi

Re: [9fans] fuse bashing

[Christoph Lohmann]

Good day.

On Wed, 25 Jan 2006 21:58:35 +0200
Enache Adrian <enache@rdslink.ro> wrote:

> - doesn't have a rename/move operation.

Maybe 9P2000.move will do that.

But Tmove is really needed. I move my whole home dir around every day,
so I adding this operation will increase my productivity by 40%!

Sincerely,

Christoph

Re: [9fans] fuse bashing

[Ronald G Minnich]

Christoph Lohmann wrote:

> Maybe 9P2000.move will do that.
> 
> But Tmove is really needed. I move my whole home dir around every day,
> so I adding this operation will increase my productivity by 40%!

so, you move it, and move it, and move it. So, one might say, at times 
you re-move it. To we need Tremove too :-)

IT is april fools!

ron

Re: [9fans] fuse bashing

[Bruce Ellis]

I have found a that a good way to handle out-of-band
messages is to have a pair of co-operating processses,
one at each end.  One makes a pipe and opens one end,
the other opens the other end of the pipe.  Then any
messages that you would like to add to 9P2k are
handled by the minions.

brucee

On 1/26/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> Christoph Lohmann wrote:
>
> > Maybe 9P2000.move will do that.
> >
> > But Tmove is really needed. I move my whole home dir around every day,
> > so I adding this operation will increase my productivity by 40%!
>
> so, you move it, and move it, and move it. So, one might say, at times
> you re-move it. To we need Tremove too :-)
>
> IT is april fools!
>
> ron
>

Re: [9fans] fuse bashing

[Ronald G Minnich]

Enache Adrian wrote:

> - doesn't have a rename/move operation.


well, it has rename. And, it should NOT have move. I don't see this as a 
problem.

ron

Re: [9fans] fuse bashing

[Enache Adrian]

On Wed, Jan 25, 2006 at 01:50:09PM -0700, Ronald G Minnich wrote:
> 
> >- doesn't have a rename/move operation.
> 
> well, it has rename.

Of course, I wasn't meaning something like wstat.

> And, it should NOT have move.

why ?

Thanks,
Adi

Re: [9fans] fuse bashing

[John Barham]

On 1/24/06, Ronald G Minnich <rminnich@lanl.gov> wrote:
> David Leimbach wrote:
>
> > I don't know too much about the limitations of 9P... and this choir
> > isn't likely to sing them [no offense but it is called 9fans after
> > all]
>
> The big complaints I know of so far on 9P are
>
> - there is no posix file locking (sorry, but people want it) although
> the 'only allow one open at a time' is a pretty damned good substitute
>
> - no ACLs (I'm convinced that the stat and wstat could be trivially
>         extended to support this --- 9p2000.acl)
>
> - doesn't fit linux vfs semantics too well (just a joke, son, but true
>     too -- sometimes you have to fit a good thing onto a broken thing)
>
> That's about all I've hit so far.

How about the potential performance hit of having to issue multiple
reads for a large file (vs. a single retr in ftp, for example), or has
fcp solved that problem adequately?

> I spent about 5 years hacking on nfs,
> and I have to say 9p is a way better protocol.

Ditto for WebDAV which is incredibly verbose for something as simple
as a directory listing.

  John

Re: [9fans] fuse bashing

[Dave Eckhardt]

> The big complaints I know of so far on 9P are [...]

If servers routinely dealt with thousands of clients and/or
clients with large persistent caches then some sort of
acceleration for cache validation might be nice--callbacks
(aka leases) or maybe something based on a log of
invalidations.

But given current usage patterns it's not clear there's
a need.

Dave Eckhardt

Re: [9fans] fuse bashing

[Ronald G Minnich]

Dave Eckhardt wrote:
>>The big complaints I know of so far on 9P are [...]
> 
> 
> If servers routinely dealt with thousands of clients and/or
> clients with large persistent caches then some sort of
> acceleration for cache validation might be nice--callbacks
> (aka leases) or maybe something based on a log of
> invalidations.
> 
> But given current usage patterns it's not clear there's
> a need.


except here and some other sites, where there is a burning need, and 
current parallel file systems are not quite working (yet)

having enough thousand nodes all try to stat the same file, at the same 
time, in a callback-based file system, can be trouble.


ron

Re: [9fans] fuse bashing

[Dave Eckhardt]

> having enough thousand nodes all try to stat the same file, at
> the same time, in a callback-based file system, can be trouble.

Also in a non-callback-based system.

But it makes sense to issue callbacks with varying lengths so
that mad rushes tend to get smeared out.

Dave Eckhardt

Re: [9fans] fuse bashing

[erik quanstrom]

one could write a fileserver that allows you to give it a file
to watch and either send a plumbmsg back or write a Dir
structure back on the original fd when it changes.

i'm not sure one needs to modify the underlying filesystem.

actually making the notifier live outside the filesystem would
allow many "notifiers" running against the same fs. say 1 fs
10 notifiers and 100 clients per notifier. 

- erik

Ronald G Minnich <rminnich@lanl.gov> writes

| 
| Dave Eckhardt wrote:
| >>The big complaints I know of so far on 9P are [...]
| > 
| > 
| > If servers routinely dealt with thousands of clients and/or
| > clients with large persistent caches then some sort of
| > acceleration for cache validation might be nice--callbacks
| > (aka leases) or maybe something based on a log of
| > invalidations.
| > 
| > But given current usage patterns it's not clear there's
| > a need.
| 
| 
| except here and some other sites, where there is a burning need, and 
| current parallel file systems are not quite working (yet)
| 
| having enough thousand nodes all try to stat the same file, at the same 
| time, in a callback-based file system, can be trouble.
| 
| 
| ron

Re: [9fans] fuse bashing

[Dave Eckhardt]

> no ACLs (I'm convinced that the stat and wstat could be trivially
> extended to support this --- 9p2000.acl)

I assume ACL semantics would need to be outside the protocol, but
have you found an ACL system you like?  AFS ACLs are usable but
not "obvious"; DFS (Son of AFS) ACLs are much more complicated
and, I think, hence less usable; Linux ACLs (at least the Red Hat
flavor) couldn't encode some natural-seeming thing my brother
wanted to do--I don't think it's a coincidence that the POSIX
committee came apart before completion.

The part of AFS I like is that every user can define new groups.
Once "owner" and "group" can be arbitrary sets of people, it's
not clear to me that you need more than "owner, group, world".
You can always come up with complicated scenarios, but in my
experience many complicated ACLs are wrong if you look carefully
at them.

Dave Eckhardt

Re: [9fans] fuse bashing

[Jack Johnson]

On 1/24/06, Dave Eckhardt <davide+p9@cs.cmu.edu> wrote:
> The part of AFS I like is that every user can define new groups.

Not ACLs, but I've thought it would be nice to reuse SSH keys and
maybe make another dot-directory, and anyone with a matching public
key in the dot-directory would have access to it, similar to
.ssh/authorized_keys but with a separate file per user.  Maybe have a
hierarchy inside to determine which operations were allowed by which
keysets.

That way the owner(s) could assign arbitrary permissions without any
third-party intervention, and potentially cross administrative realms.
 It could even be relatively cross-platform, something you could do
with a GUI on other OSes in a way semi-intuitive for newbies.  The key
processing would probably add too much overhead for general use, but
the tradeoff might be worth it for those instances where you're trying
to facilitate certain kinds of data sharing.

-Jack

Re: [9fans] fuse bashing

[Lyndon Nerenberg]

On Jan 24, 2006, at 11:46 PM, Jack Johnson wrote:

> Not ACLs, but I've thought it would be nice to reuse SSH keys and
> maybe make another dot-directory, and anyone with a matching public
> key in the dot-directory would have access to it, similar to
> .ssh/authorized_keys but with a separate file per user.

But these *are* ACLs, and by putting them into the filesystem you're  
trying to protect you create an intractable recursive problem: how to  
protect the ACLs with themselves.

To secure the filesystem you have to isolate this metadata. The  
problem to date is that nobody has figured out a clean way of doing  
this in a way that makes it simple to administer -- and maintain --  
the ACLs (and other crud).  E.g. FreeBSD supports ACLs in the  
filesystem, but all the tools that manipulate files need to be  
cognizant of them, and sadly, most aren't.  Thus an application that  
does it's work on a temporary file and then does the 'unlink(foo);  
rename(tmp,foo)' dance loses the ACL attributes (and other  
metadata).  On occasion I wonder if this asks for a replace(a,b)  
syscall that does the equivalent atomic unlink+rename while  
preserving the metadata (ACLs, MAC labels, etc) associated with the  
original file.

--lyndon

Re: [9fans] fuse bashing

[Lyndon Nerenberg]

> Thus an application that does it's work on a temporary file and  
> then does the 'unlink(foo); rename(tmp,foo)' dance

And before anyone flames: yes, I've run into UNIXen where the unlink  
was necessary :-P

--lyndon

RE: [9fans] fuse bashing

[Andriy G. Tereshchenko]

>  Thus an application that does it's work on a temporary file and then does
the 'unlink(foo); rename(tmp,foo)' dance loses the ACL attributes (and other
metadata). 
Even MS has issues with this - they try to address it with  "File System
Tunneling Effect" - http://support.microsoft.com/?kbid=172190

Re: [9fans] fuse bashing

[Ronald G Minnich]

Dave Eckhardt wrote:
>>no ACLs (I'm convinced that the stat and wstat could be trivially
>>extended to support this --- 9p2000.acl)
> 
> 
> I assume ACL semantics would need to be outside the protocol, but
> have you found an ACL system you like? 

I'm not so sure it has to be outside the protocol. We've already 
extended stat/wstat with .u, and it was not that big a deal. The *stat 
packets are a very good design, and I think could be stretched for ACLs.

> The part of AFS I like is that every user can define new groups.
> Once "owner" and "group" can be arbitrary sets of people, it's
> not clear to me that you need more than "owner, group, world".

Data General AOS/VS had a pretty usable ACL scheme. ACLs were simple 
sequences of tuples: (re, permission)+ -- IIRC, it's been 24 years, and 
I can't find that manual -- so, for example, you might have:
(rminnich,readwrite):(sys+, read):(myfriends, readwrite)

and so on.

In practice, everybody did user, group, world, just with re's instead of 
numbers; I think I agree with you. It's too hard to think about much 
else. I didn't say people NEED ACLs, they just WANT ACLs. As I used to 
say to my kids all the time, "You don't need that, you want that". After 
about 10 years, they got my point.

ACLs are the new fad, and they're going in, with silly bugs to boot 
(ACLs have been NOT fun on linux), and I really doubt in the end we'll 
go much beyond user, group, world.

ron

Re: [9fans] fuse bashing

[C H Forsyth]

> If you download a fuse distribution and look in
> kernel/fuse_kernel.h you can see the messages.

which i did, to do a limbo version.
then i found that as i expected, they'd screwed up the exchange
between kernel and user level so that the same component didn't
work even between 64-bit and 32-bit environments on the same host,
let alone between hosts.  ``right everybody, change protocols!''
then i found that they couldn't get the interrupt
stuff working properly, so they ifdef'd it out and hacked it.
then i found ...

after a time, one gets tired of reading badly-worked out
implementations of badly-thought out ideas.

i don't care who was first.
the question is: what is right?

they should have done a little `research' first, before jumping in there,
``first one to the keyboard wins!''

Re: [9fans] fuse bashing

[David Leimbach]

On 1/23/06, C H Forsyth <forsyth@vitanuova.com> wrote:
> > If you download a fuse distribution and look in
> > kernel/fuse_kernel.h you can see the messages.
>
> which i did, to do a limbo version.
> then i found that as i expected, they'd screwed up the exchange
> between kernel and user level so that the same component didn't
> work even between 64-bit and 32-bit environments on the same host,
> let alone between hosts.  ``right everybody, change protocols!''
> then i found that they couldn't get the interrupt
> stuff working properly, so they ifdef'd it out and hacked it.
> then i found ...
>
> after a time, one gets tired of reading badly-worked out
> implementations of badly-thought out ideas.
>
> i don't care who was first.
> the question is: what is right?
>
> they should have done a little `research' first, before jumping in there,
> ``first one to the keyboard wins!''
>
>

DragonFlyBSD also is supposed to get VFS messaging.... but I think
that will be the basis of the entire VFS, it's hard to say where they
are now.  I mentioned a few times on their list that they might want
to at least look at 9P... and that people may be interested in writing
DFly VFS->9P translation servers at some point.

Not sure how well-received it all was.

Dave

Re: [9fans] fuse bashing

[Ronald G Minnich]

David Leimbach wrote:

> Not sure how well-received it all was.

NIH, a powerful force in our "open" OS community.

ron

Re: [9fans] fuse bashing

[Skip Tavakkolian]

> let alone between hosts.  ``right everybody, change protocols!''

does it have a "version" message?

Re: [9fans] fuse bashing

[Latchesar Ionkov]

On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> It appears to match the kernel better, so the implementation
> should be simpler.  (As soon as you want to talk between

It is much simpler (and probably faster), and that's a big win for FUSE:

	$ cat fs/fuse/*.[ch] | wc -l
	4073

	$ cat 9p/*.[ch] | wc -l
	7271

And v9fs doesn't even have full 9P support (all users share single attach
for example).

V9fs's big plus is that it is network ready and has some form of
authentication.

I didn't see many interesting file systems on the FUSE page. I think that
v9fs can become very popular as long as we create enough useful file servers
(and even more important, write good documentation).

	Lucho

Re: [9fans] fuse bashing

[Charles Forsyth]

> It is much simpler (and probably faster), and that's a big win for FUSE:

given that it evades some problems (particularly in the kernel interface
wrt messed-up cache schemes, and being able to interrupt things, and
a few other things i noticed in my own survey last year, not to mention
portability), i'm not at all surprised fuse is currently smaller.

Re: [9fans] fuse bashing

[Charles Forsyth]

i probably ought to make clear that i have not been commenting on the
quality of the code of fuse--not at all--but rather various limitations of what it does,
and (to be fair to it even more) of the creaky system(s) beneath it.

as Tad used to say, though, about yet another attempt to
make anything interesting look as boring as everything else,
``come on [X], it's the 90s!!'' (and that was last century!)

at least to me, distribution is a big task of systems now, on
increasingly large scale and range.  space, the final frontier;
something like that.  it's hard enough as it is, but linux seems
currently no help whatsoever: each system is still stuck in its own
little box, fretting about those tlbs.  `distribution' occurs mainly
in discussing which new ones turned up this month.

i'm not surprised everything has fallen on the broad shoulders of apache.
no, that can't be right: that must be ajax.  interesting nomenclature:
apache, tomcat, jaguar, ..., all suggest potential for blood and pain to me.
or is it appealing to the warrior in the geek?

Re: [9fans] fuse bashing

[Charles Forsyth]

> i'm not surprised everything has fallen on the broad shoulders of apache.
> no, that can't be right: that must be ajax.

classically it would be atlas, of course, but that's something different.

Re: [9fans] fuse bashing

["Nils O. Selåsdal"]

Charles Forsyth wrote:
> 
> i'm not surprised everything has fallen on the broad shoulders of apache.
> no, that can't be right: that must be ajax.  interesting nomenclature:
> apache, tomcat, jaguar, ..., all suggest potential for blood and pain to me.
> or is it appealing to the warrior in the geek?

I guess even warriors vomits every so often.

Re: [9fans] fuse bashing

[quanstro]

On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > It appears to match the kernel better, so the implementation
> > should be simpler.  (As soon as you want to talk between
> 
> It is much simpler (and probably faster), and that's a big win for FUSE:
> 
> 	$ cat fs/fuse/*.[ch] | wc -l
> 	4073
> 
> 	$ cat 9p/*.[ch] | wc -l
> 	7271

why do you assume that line count is porportinal to
speed of execution or complexity?

Re: [9fans] fuse bashing

[Eric Van Hensbergen]

On 1/24/06, quanstro@quanstro.net <quanstro@quanstro.net> wrote:
> On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > It appears to match the kernel better, so the implementation
> > > should be simpler.  (As soon as you want to talk between
> >
> > It is much simpler (and probably faster), and that's a big win for FUSE:
> >
> >       $ cat fs/fuse/*.[ch] | wc -l
> >       4073
> >
> >       $ cat 9p/*.[ch] | wc -l
> >       7271
>

There's likely quite a bit more complexity in the 9P kernel modules. 
As was pointed out earlier, FUSE basically just gateways VFS calls to
user-space.  This is an oversimplificaiton, but its not doing any of
the mapping we are doing in v9fs.

I'm not willing to say FUSE sucks, it is what it is, and for some
folks its going to be the right solution.  However, for the file
systems I wanted to write, it wasn't a good match and for file systems
of any complexity the FUSE user-space code seems to get out of hand
quickly.

Which brings up a good point for folks who are wc -l 'ing -- anyone
look at the relative sizes of ramfs coded in 9P and FUSE (including
their library code)?

        -eric

Re: [9fans] fuse bashing

[Charles Forsyth]

> systems I wanted to write, it wasn't a good match and for file systems
> of any complexity the FUSE user-space code seems to get out of hand
> quickly.

that's interesting: why is that?

Re: [9fans] fuse bashing

[Latchesar Ionkov]

Is there ramfs for FUSE?

Thanks,
        Lucho

On Tue, Jan 24, 2006 at 03:37:54PM -0600, Eric Van Hensbergen said:
> On 1/24/06, quanstro@quanstro.net <quanstro@quanstro.net> wrote:
> > On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > > It appears to match the kernel better, so the implementation
> > > > should be simpler.  (As soon as you want to talk between
> > >
> > > It is much simpler (and probably faster), and that's a big win for FUSE:
> > >
> > >       $ cat fs/fuse/*.[ch] | wc -l
> > >       4073
> > >
> > >       $ cat 9p/*.[ch] | wc -l
> > >       7271
> >
> 
> There's likely quite a bit more complexity in the 9P kernel modules. 
> As was pointed out earlier, FUSE basically just gateways VFS calls to
> user-space.  This is an oversimplificaiton, but its not doing any of
> the mapping we are doing in v9fs.
> 
> I'm not willing to say FUSE sucks, it is what it is, and for some
> folks its going to be the right solution.  However, for the file
> systems I wanted to write, it wasn't a good match and for file systems
> of any complexity the FUSE user-space code seems to get out of hand
> quickly.
> 
> Which brings up a good point for folks who are wc -l 'ing -- anyone
> look at the relative sizes of ramfs coded in 9P and FUSE (including
> their library code)?
> 
>         -eric

Re: [9fans] fuse bashing

[Latchesar Ionkov]

On Tue, Jan 24, 2006 at 02:10:41PM -0600, quanstro@quanstro.net said:
> On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > It appears to match the kernel better, so the implementation
> > > should be simpler.  (As soon as you want to talk between
> > 
> > It is much simpler (and probably faster), and that's a big win for FUSE:
> > 
> > 	$ cat fs/fuse/*.[ch] | wc -l
> > 	4073
> > 
> > 	$ cat 9p/*.[ch] | wc -l
> > 	7271
> 
> why do you assume that line count is porportinal to
> speed of execution or complexity?

Because it doesn't need to convert an alien protocol to Linux VFS. It
doesn't need to care about converting error strings to error codes,
architecture independent format for the values etc. It doesn't need to care
about unix and tcp sockets. In fact I tested the speed of fuse+fusexmpl
compared with v9fs+u9fs, and fuse is noticeably faster.

If FUSE doesn't handle the interrupts well, that in part explains why is it
smaller. The proper signal handling is one of the most complex parts in v9fs.

Thanks,
        Lucho

Re: [9fans] fuse bashing

[erik quanstrom]

On Tue Jan 24 18:24:33 CST 2006, lucho@gmx.net wrote:
> On Tue, Jan 24, 2006 at 02:10:41PM -0600, quanstro@quanstro.net said:
> > On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > > It appears to match the kernel better, so the implementation
> > > > should be simpler.  (As soon as you want to talk between
> > > 
> > > It is much simpler (and probably faster), and that's a big win for FUSE:
> > > 
> > > 	$ cat fs/fuse/*.[ch] | wc -l
> > > 	4073
> > > 
> > > 	$ cat 9p/*.[ch] | wc -l
> > > 	7271
> > 
> > why do you assume that line count is porportinal to
> > speed of execution or complexity?
> 
> Because it doesn't need to convert an alien protocol to Linux VFS. It
> doesn't need to care about converting error strings to error codes,
> architecture independent format for the values etc.

even so, these sound like simple translations. they may produce more
code, but i'm not sure they add complexity.

> It doesn't need to care
> about unix and tcp sockets. In fact I tested the speed of fuse+fusexmpl
> compared with v9fs+u9fs, and fuse is noticeably faster.

why didn't you say so? ;-)

i don't use u9fs; was there any indication that v9fs in the kernel was slow?
have you tried any p9p fileservers? tapefs seemed speedy enough:

; 9p ls tapefs
--rw-r--r-- M 0 1000 100 6749246 Feb  5  1998 cyberbit.zip
; time 9p read tapefs/cyberbit.zip > /dev/null
0.18user 0.20system 0:01.07elapsed 36%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+404minor)pagefaults 0swaps

; time cat 9/tapefs/cyberbit.zip > /dev/null
0.00user 0.01system 0:01.15elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+146minor)pagefaults 0swaps

> 
> If FUSE doesn't handle the interrupts well, that in part explains why is it
> smaller. The proper signal handling is one of the most complex parts in v9fs.
> 
> Thanks,
>         Lucho

Re: [9fans] fuse bashing

[Latchesar Ionkov]

I didn't try tapefs, but ramfs in p9p is slower than u9fs (even after the
fid lookup is improved).

Thanks,
	Lucho

On Tue, Jan 24, 2006 at 07:31:44PM -0600, erik quanstrom said:
> 
> On Tue Jan 24 18:24:33 CST 2006, lucho@gmx.net wrote:
> > On Tue, Jan 24, 2006 at 02:10:41PM -0600, quanstro@quanstro.net said:
> > > On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > > > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > > > It appears to match the kernel better, so the implementation
> > > > > should be simpler.  (As soon as you want to talk between
> > > > 
> > > > It is much simpler (and probably faster), and that's a big win for FUSE:
> > > > 
> > > > 	$ cat fs/fuse/*.[ch] | wc -l
> > > > 	4073
> > > > 
> > > > 	$ cat 9p/*.[ch] | wc -l
> > > > 	7271
> > > 
> > > why do you assume that line count is porportinal to
> > > speed of execution or complexity?
> > 
> > Because it doesn't need to convert an alien protocol to Linux VFS. It
> > doesn't need to care about converting error strings to error codes,
> > architecture independent format for the values etc.
> 
> even so, these sound like simple translations. they may produce more
> code, but i'm not sure they add complexity.
> 
> > It doesn't need to care
> > about unix and tcp sockets. In fact I tested the speed of fuse+fusexmpl
> > compared with v9fs+u9fs, and fuse is noticeably faster.
> 
> why didn't you say so? ;-)
> 
> i don't use u9fs; was there any indication that v9fs in the kernel was slow?
> have you tried any p9p fileservers? tapefs seemed speedy enough:
> 
> ; 9p ls tapefs
> --rw-r--r-- M 0 1000 100 6749246 Feb  5  1998 cyberbit.zip
> ; time 9p read tapefs/cyberbit.zip > /dev/null
> 0.18user 0.20system 0:01.07elapsed 36%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+404minor)pagefaults 0swaps
> 
> ; time cat 9/tapefs/cyberbit.zip > /dev/null
> 0.00user 0.01system 0:01.15elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (0major+146minor)pagefaults 0swaps
> 
> > 
> > If FUSE doesn't handle the interrupts well, that in part explains why is it
> > smaller. The proper signal handling is one of the most complex parts in v9fs.
> > 
> > Thanks,
> >         Lucho

Re: [9fans] fuse bashing

[Eric Van Hensbergen]

Big reason is likely the page-cache.  FUSE can take full advantage
while we take overhead trying to circumvent it.  Perhaps its time to
look at putting cacheing back into v9fs.

We really need to do a performance drill-down again with a
cross-section of workloads.  I'd like to look at SDET (which is
essentially a software development based benchmark).  I'll see if I
can dig up source code.  Likely we'll need dcache and pagecache before
we'll have decent numbers, but I'd like to see how close we come
without them.

         -eric

On 1/24/06, Latchesar Ionkov <lucho@gmx.net> wrote:
> I didn't try tapefs, but ramfs in p9p is slower than u9fs (even after the
> fid lookup is improved).
>
> Thanks,
>         Lucho
>
> On Tue, Jan 24, 2006 at 07:31:44PM -0600, erik quanstrom said:
> >
> > On Tue Jan 24 18:24:33 CST 2006, lucho@gmx.net wrote:
> > > On Tue, Jan 24, 2006 at 02:10:41PM -0600, quanstro@quanstro.net said:
> > > > On Mon Jan 23 20:21:36 CST 2006, lucho@gmx.net wrote:
> > > > > On Mon, Jan 23, 2006 at 12:06:09PM -0500, Russ Cox said:
> > > > > > It appears to match the kernel better, so the implementation
> > > > > > should be simpler.  (As soon as you want to talk between
> > > > >
> > > > > It is much simpler (and probably faster), and that's a big win for FUSE:
> > > > >
> > > > >         $ cat fs/fuse/*.[ch] | wc -l
> > > > >         4073
> > > > >
> > > > >         $ cat 9p/*.[ch] | wc -l
> > > > >         7271
> > > >
> > > > why do you assume that line count is porportinal to
> > > > speed of execution or complexity?
> > >
> > > Because it doesn't need to convert an alien protocol to Linux VFS. It
> > > doesn't need to care about converting error strings to error codes,
> > > architecture independent format for the values etc.
> >
> > even so, these sound like simple translations. they may produce more
> > code, but i'm not sure they add complexity.
> >
> > > It doesn't need to care
> > > about unix and tcp sockets. In fact I tested the speed of fuse+fusexmpl
> > > compared with v9fs+u9fs, and fuse is noticeably faster.
> >
> > why didn't you say so? ;-)
> >
> > i don't use u9fs; was there any indication that v9fs in the kernel was slow?
> > have you tried any p9p fileservers? tapefs seemed speedy enough:
> >
> > ; 9p ls tapefs
> > --rw-r--r-- M 0 1000 100 6749246 Feb  5  1998 cyberbit.zip
> > ; time 9p read tapefs/cyberbit.zip > /dev/null
> > 0.18user 0.20system 0:01.07elapsed 36%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (0major+404minor)pagefaults 0swaps
> >
> > ; time cat 9/tapefs/cyberbit.zip > /dev/null
> > 0.00user 0.01system 0:01.15elapsed 1%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (0major+146minor)pagefaults 0swaps
> >
> > >
> > > If FUSE doesn't handle the interrupts well, that in part explains why is it
> > > smaller. The proper signal handling is one of the most complex parts in v9fs.
> > >
> > > Thanks,
> > >         Lucho
>