Re: [9fans] The Web9 Project

Re: [9fans] The Web9 Project

[Enrico Weigelt]

* Anant Narayanan <anant@kix.in> wrote:

> Following a request for more information on the project, here are a
> few details:

thx :)

> a) PHP bindings to libixp (client-only; server portion currently
> being worked on). I will probably try to get this included as a
> standard module in the PHP distribution.

Cool :)

Do you dock into the stream handling or use separate functions ?
Would be really cool if you simply can fopen() an 9p resource :)

BTW: perhaps you'd noticed I'm working on my own branch of libixp,
called libmixp. It will also contain some address/URI handling
functions, so an individual application doesn't need to care
about this anymore. Maybe you'd like to have a look at it.

> b) JS9P - Implementation of the 9P protocol in Javascript. You can
> create and parse auth messages, but authentication is not supported
> at a higher level. The library provides methods to only create and
> parse binary 9P messages, since JS doesn't have native networking
> functionality. The actual transport can be done by either using the
> Mozilla JS-XPCOM extensions (see below) - or using XMLHttpRequests -
> (a sample has implemented - visit the Web9 site for more info).

Huh, tunnel 9p messages through http ?!

> c) "Angled", a firefox extension to read files of a 9P serve right in
> the browser (uses JS9P). For example, typing in ninep://tcp!
> sources.cs.bell-labs.com!564/plan9/LICENSE into the address bar will
> return that file. Text and Images are shown in the browser itself,
> while other binary files trigger a download request. I'm still trying
> to figure out what is the best kind of interface to offer write/
> modify capabilities for the extension is.

IMHO, the approach is extending mozilla's stream handling (nsChannel stuff).
Some time ago a had some look at it. Doesn't seem very trivial.


cu
--
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [9fans] The Web9 Project

[Skip Tavakkolian]

nice!

for an eyecandy application, you might want to consider integrating
with something like eyeos (eyeos.org) or similar php based systems.

have you compared its performance to webdav?

> Hi,
>
> Following a request for more information on the project, here are a
> few details:
>
> a) PHP bindings to libixp (client-only; server portion currently
> being worked on). I will probably try to get this included as a
> standard module in the PHP distribution.
>
> b) JS9P - Implementation of the 9P protocol in Javascript. You can
> create and parse auth messages, but authentication is not supported
> at a higher level. The library provides methods to only create and
> parse binary 9P messages, since JS doesn't have native networking
> functionality. The actual transport can be done by either using the
> Mozilla JS-XPCOM extensions (see below) - or using XMLHttpRequests -
> (a sample has implemented - visit the Web9 site for more info).
>
> c) "Angled", a firefox extension to read files of a 9P serve right in
> the browser (uses JS9P). For example, typing in ninep://tcp!
> sources.cs.bell-labs.com!564/plan9/LICENSE into the address bar will
> return that file. Text and Images are shown in the browser itself,
> while other binary files trigger a download request. I'm still trying
> to figure out what is the best kind of interface to offer write/
> modify capabilities for the extension is.
>
> The work is a result of the Summer of Code 2007. You can clone the hg
> repository: http://code.kix.in/hg/web9 or view the code at http://
> code.kix.in/projects/web9/browser (there is also a mirror of the code
> at gsoc.cat-v.org)
> I'm also currently working on a pure PHP implementation of 9P,
> scheduled for inclusion at PEAR (pear.php.net) as the Net_9P package.
>
> Regards,
> --
> Anant

Re: [9fans] The Web9 Project

[Anant Narayanan]

> Do you dock into the stream handling or use separate functions ?
> Would be really cool if you simply can fopen() an 9p resource :)

Right now it's just two classes IxpClient and IxpServer; but thanks
for the suggestion. Making a stream handler for 9P would definitely
be worth it.

> Huh, tunnel 9p messages through http ?!

Yes. Using Mozilla's XPCOM extensions isn't a very portable solution,
and part of the reason web developers choose Javascript is because it
is browser-agnostic. The way this works is that the client encodes a
series of 9P T-messages into a XMLHttpRequest and the script on the
server forwards those messages in-order to the actual 9P service, and
responds with a series of R-messages (which the client can then decode).

> IMHO, the approach is extending mozilla's stream handling
> (nsChannel stuff).
> Some time ago a had some look at it. Doesn't seem very trivial.

Yes, an nsIChannel implementation was required for this to work.
Nothing in Mozilla is trivial - but I will admit that I took longer
to learn libdraw than Mozilla's netlib ;)

--
Anant

Re: [9fans] The Web9 Project

[Anant Narayanan]

> for an eyecandy application, you might want to consider integrating
> with something like eyeos (eyeos.org) or similar php based systems.

eyeos seems quite interesting, thanks for the pointer. Another
interesting application that is possible is a drawterm plugin for
firefox that uses Angled.

> have you compared its performance to webdav?

I don't have any numbers with me, but I would expect 9P to work
faster than WebDAV since 9P works one layer below HTTP.
Implementation details aside, header-overhead in itself makes WebDAV
a less of a competitor.

--
Anant

Re: [9fans] The Web9 Project

[Uriel]

> > have you compared its performance to webdav?
>
> I don't have any numbers with me, but I would expect 9P to work
> faster than WebDAV since 9P works one layer below HTTP.
> Implementation details aside, header-overhead in itself makes WebDAV
> a less of a competitor.

Maybe, if you have ridiculously low latency. Which is one of the
reasons I would like to see the latency issues addressed, so 9P
services can work well over non-LAN networks. Maybe we can finally
agree on a solution for this at this year's IWP9?

Best wishes

uriel

Re: [9fans] The Web9 Project

[Latchesar Ionkov]

We already agreed on a solution. Nobody is interested in implementing it.

On 9/8/07, Uriel <uriel99@gmail.com> wrote:
> > > have you compared its performance to webdav?
> >
> > I don't have any numbers with me, but I would expect 9P to work
> > faster than WebDAV since 9P works one layer below HTTP.
> > Implementation details aside, header-overhead in itself makes WebDAV
> > a less of a competitor.
>
> Maybe, if you have ridiculously low latency. Which is one of the
> reasons I would like to see the latency issues addressed, so 9P
> services can work well over non-LAN networks. Maybe we can finally
> agree on a solution for this at this year's IWP9?
>
> Best wishes
>
> uriel
>
>

Re: [9fans] The Web9 Project

[Francisco J Ballesteros]

On 9/9/07, Latchesar Ionkov <lucho@gmx.net> wrote:
> We already agreed on a solution. Nobody is interested in implementing it.
>

Well, I started indeed. But then noticed that I needed just a
different protocol and
went for it. I may try again in the future with what I learned from
our current attempt.

Re: [9fans] The Web9 Project

[erik quanstrom]

> We already agreed on a solution. Nobody is interested in implementing it.
> 
> On 9/8/07, Uriel <uriel99@gmail.com> wrote:
>> > > have you compared its performance to webdav?
>> >
>> > I don't have any numbers with me, but I would expect 9P to work
>> > faster than WebDAV since 9P works one layer below HTTP.
>> > Implementation details aside, header-overhead in itself makes WebDAV
>> > a less of a competitor.
>>
>> Maybe, if you have ridiculously low latency. Which is one of the
>> reasons I would like to see the latency issues addressed, so 9P
>> services can work well over non-LAN networks. Maybe we can finally
>> agree on a solution for this at this year's IWP9?

this topic has come up before.  i'm not sure i have a clear picture of the
problem.  would someone give a concrete example?

without really knowing what the problem is, there is one big thing that
9p clients traditionally don't do that would be enormously helpful for
larger files -- readahead.  there's nothing in 9p that prevents a client from
having R outstanding reads at the same time.  if l is the rtt latency and
s is the avg time it takes the fs to service a request, we can try to pick a
(reasonable) number of outstanding requests R s.t. Rs ≥ l.  even if we
can't, N outstanding should reduce the latency penalty for N packets
to l, not Nl.

if instead the problem is lots of little files the proposals i've seen
are something like bundles of 9p requests.  sent en mass to the fs.
how about the opposite?  why not bring the blocks en mass to the fs?
the remote fs could be treated as a block storage device (we know how
to do readahead on these things) and the "fs" could be run locally.
the "fs" a mkfs archive, mbox format, a fossil fs or whatever.

unfortunately, if the problem is fine-grained, highly concurrent access,
readahead just won't work.

- erik

Re: [9fans] The Web9 Project

[Francisco J Ballesteros]

The problem is that you need, in a way, to break 9p. You need readahead, you
need to bundle requests, and you need to cache in a very careful way.
This "caching"
is most effective if you can map several 9p fids to the same cache
entry (i.e, to the
same "fid" in the server). In the end, it was more simple to make two
9p-to-op and op-to-fs processes and keep the dialog between them
secret for the rest of the world.



On 9/9/07, erik quanstrom <quanstro@quanstro.net> wrote:
> > We already agreed on a solution. Nobody is interested in implementing it.
> >
> > On 9/8/07, Uriel <uriel99@gmail.com> wrote:
> >> > > have you compared its performance to webdav?
> >> >
> >> > I don't have any numbers with me, but I would expect 9P to work
> >> > faster than WebDAV since 9P works one layer below HTTP.
> >> > Implementation details aside, header-overhead in itself makes WebDAV
> >> > a less of a competitor.
> >>
> >> Maybe, if you have ridiculously low latency. Which is one of the
> >> reasons I would like to see the latency issues addressed, so 9P
> >> services can work well over non-LAN networks. Maybe we can finally
> >> agree on a solution for this at this year's IWP9?
>
> this topic has come up before.  i'm not sure i have a clear picture of the
> problem.  would someone give a concrete example?
>
> without really knowing what the problem is, there is one big thing that
> 9p clients traditionally don't do that would be enormously helpful for
> larger files -- readahead.  there's nothing in 9p that prevents a client from
> having R outstanding reads at the same time.  if l is the rtt latency and
> s is the avg time it takes the fs to service a request, we can try to pick a
> (reasonable) number of outstanding requests R s.t. Rs ≥ l.  even if we
> can't, N outstanding should reduce the latency penalty for N packets
> to l, not Nl.
>
> if instead the problem is lots of little files the proposals i've seen
> are something like bundles of 9p requests.  sent en mass to the fs.
> how about the opposite?  why not bring the blocks en mass to the fs?
> the remote fs could be treated as a block storage device (we know how
> to do readahead on these things) and the "fs" could be run locally.
> the "fs" a mkfs archive, mbox format, a fossil fs or whatever.
>
> unfortunately, if the problem is fine-grained, highly concurrent access,
> readahead just won't work.
>
> - erik
>
>

Re: [9fans] The Web9 Project

[erik quanstrom]

> The problem is that you need, in a way, to break 9p. You need readahead, you
> need to bundle requests, and you need to cache in a very careful way.

the way i read the protocol, 9p does support readahead.  Tread
takes tag, fid, offset, count.  i can have up to 64k tags, so i think
this means i can have up to 64k outstanding reads.  what am i missing?

why do we need to bundle requests?  that seems like the wrong level.
in effect, that is creating an execution environment on the fs.

if latency is that bad and readahead won't work because the files
are too small, why not treat the remote storage as a block device
and run the fs locally?  the "fs" could be a simple flat file.

this seems much simplier for most applications.  no fid cache mapping
problem. and it reuses


- erik

Re: [9fans] The Web9 Project

[Francisco J Ballesteros]

>
> the way i read the protocol, 9p does support readahead.  Tread
> takes tag, fid, offset, count.  i can have up to 64k tags, so i think
> this means i can have up to 64k outstanding reads.  what am i missing?
>
> why do we need to bundle requests?  that seems like the wrong level.
> in effect, that is creating an execution environment on the fs.

Latency. I want all requests on two different fids(!) for the same file
issued within a tiny window to coallesce on just one rpc. Yes, breaks coherency
a little bit but AFAIK there's no other way.

An example. If you stat a directory, it would take the same time to read all its
contents within the same rpc. if you later stat a file in the
directory, you could be done
just with what you got. Things get a lot worse for latency if you stat
the top-level dir in
this example before you read it.

Because caching issues, and rpc bundling, seemed to mix just to
achieve a particular
interaction pattern, it seemed more clear just to use a protocol using
that pattern than
forcing 9p into that.

You need a program in the server side anyway, because the client would
not know what
it is retrieving before retrieving it for the first time. Thus, if you
want to get an entire dir at a
time, you'd have to stat (9p) and once you know it's a dir you could
read it all ahead. By this
time, the client had stall waiting for the dir. On the other hand, if
the "protocol" specifies that
a "get" of a dir sends it all to the client, things stay simple and a
single rpc suffices.


> if latency is that bad and readahead won't work because the files
> are too small, why not treat the remote storage as a block device
> and run the fs locally?  the "fs" could be a simple flat file.

Because we wanted the protocol to work for things like o/mero, which
is not a real
fs at all. not to talk about other synthetic file servers.


In any case, I'd like to get this thing screened by others. It might
be a more simple way
that happens work and we'd like to learn which one. I'd be more than
happy to throw this
all away and implement a replacement provided it simplifies things.