[9fans] Speed of 9pfuse on Linux

[9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

Hi all,

I am trying 9pfuse (p9p) on my Linux (Fedora 10), and the access to
remote directories/files is extremly slow.
Do I make something wrong?

Thanks in advance.

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Gorka Guardiola]

Maybe yes, maybe no. What is the latency to your file server?.
http://lsub.org/ls/export/opiwp9.pdf
http://lsub.org/ls/export/opiwp9tlk.pdf
--
- curiosity sKilled the cat

Re: [9fans] Speed of 9pfuse on Linux

[Eric Van Hensbergen]

Also - which file server are you using?

Sent from my iPhone

On Feb 10, 2010, at 3:54 AM, Gorka Guardiola <paurea@gmail.com> wrote:

> Maybe yes, maybe no. What is the latency to your file server?.
> http://lsub.org/ls/export/opiwp9.pdf
> http://lsub.org/ls/export/opiwp9tlk.pdf
> --
> - curiosity sKilled the cat
>

Re: [9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

> Also - which file server are you using?
As I wrote above I tested it with 'sources.cs.bell-labs.com'.

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

> Maybe yes, maybe no. What is the latency to your file server?
I tested it with 'sources.cs.bell-labs.com'.

My tests are performed on the same HW.
If I boot into the native Plan9, the access is fast enough.
If I boot into the Fedora 10, the access is extremly slow...

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Eric Van Hensbergen]

File operation bandwidth should be roughly equivilent once the file is
open - directory reads will have a large penalty under Linux
complicated by the latency of the connection.

      -Eric

Sent from my iPhone

On Feb 10, 2010, at 11:57 AM, Pavel Klinkovsky <pavel.klinkovsky@gmail.com
 > wrote:

>> Maybe yes, maybe no. What is the latency to your file server?
> I tested it with 'sources.cs.bell-labs.com'.
>
> My tests are performed on the same HW.
> If I boot into the native Plan9, the access is fast enough.
> If I boot into the Fedora 10, the access is extremly slow...
>
> Pavel
>

Re: [9fans] Speed of 9pfuse on Linux

[maht]

> Hi all,
>
> I am trying 9pfuse (p9p) on my Linux (Fedora 10), and the access to
> remote directories/files is extremly slow.
> Do I make something wrong?
>
> Thanks in advance.
>
> Pavel
>
>
Two things you can test with :

1) real plan9 to the same place
2) qemu plan9 on Fedora to the same place

"It's slow, what's wrong" is perhaps a little vague.

Re: [9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

> 1) real plan9 to the same place
> 2) qemu plan9 on Fedora to the same place
As I wrote above, I made exactly the same test on exactly the same HW
(and internet connection).
1. Native Plan9.
2. Native Fedora 10 with p9p.

> "It's slow, what's wrong" is perhaps a little vague.
Not precisely measured (I can do it today).
My estimation of the time spent by 'ls' command in contrib directory:
- 9pfuse on Fedora was more than 10 times slower.

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Eric Van Hensbergen]

On Feb 10, 2010, at 7:32 AM, Pavel Klinkovsky wrote:

>> 1) real plan9 to the same place
>> 2) qemu plan9 on Fedora to the same place
> As I wrote above, I made exactly the same test on exactly the same HW
> (and internet connection).
> 1. Native Plan9.
> 2. Native Fedora 10 with p9p.
> 
>> "It's slow, what's wrong" is perhaps a little vague.
> Not precisely measured (I can do it today).
> My estimation of the time spent by 'ls' command in contrib directory:
> - 9pfuse on Fedora was more than 10 times slower.
> 

That's actually probably pretty good, if you look at slide  10 & 11 on my Linux 9P Trace and Walkthrough (top of http://www.graverobber.org at the moment) you'll see the bad protocol behavior caused by v9fs at the moment -- 9pfuse probably has around the same order increase.  The problem is that Linux doesn't know we got all the attribute information with the dirread, and then goes and individually queries each file in the directory with a stat -- this happens in a serial fashion, so the high latency to sources just makes the problem worse.

A potential solution is to cache metadata, which would speed things up dramatically for static data, but might mess things up for synthetic files.

      -eric

Re: [9fans] Speed of 9pfuse on Linux

[Steve Simon]

> The problem is that Linux doesn't know we got all the attribute
> information with the dirread, and then goes and individually queries
> each file in the directory with a stat -- this happens in a serial
> fashion, so the high latency to sources just makes the problem worse.

I have had similar problems in the past and opted to cache dirread
metadata for a second. This is short enough for most things to
continue to work, but long enough for ls(1) to feel responsive.

if you applications use  the old unix trick of using a directory as
an interprocess lock then your mileage may vary...

-Steve

Re: [9fans] Speed of 9pfuse on Linux

[Venkatesh Srinivas]

Perhaps the time to talk about QTDECENT is at hand?

-- vs

Re: [9fans] Speed of 9pfuse on Linux

[Gorka Guardiola]

On Wed, Feb 10, 2010 at 7:54 PM, Venkatesh Srinivas <me@acm.jhu.edu> wrote:
> Perhaps the time to talk about QTDECENT is at hand?
>

I feel like it is Groundhog Day lately when I read the list.


--
- curiosity sKilled the cat

Re: [9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

Results of my measurements ('ls' in sources repository)

Native Plan9:
1.05 s - very stable results

Native Linux:
40-75 s - very spread results

My "10x slower" estimation was too optimistic.

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Jacob Todd]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

On Fri, Feb 12, 2010 at 10:53:49AM +0000, Pavel Klinkovsky wrote:
> Results of my measurements ('ls' in sources repository)
> 
> Native Plan9:
> 1.05 s - very stable results
> 
> Native Linux:
> 40-75 s - very spread results
> 
> My "10x slower" estimation was too optimistic.
> 
> Pavel
> 
That's a really long time for ls. With 2.6.31.6, for me ls takes about 5s.

-- 
I am a man who does not exist for others.

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

Re: [9fans] Speed of 9pfuse on Linux

[roger peppe]

this is not surprising if you consider what's happening underneath:

native plan 9 to read a directory:
Twalk to directory
Topen directory
Tread directory
Tclunk

linux via fuse:
Twalk to directory
Topen directory
Tread directory
[
Twalk to directory item
Tstat item
Tclunk item
] * n-items in directory
Tclunk directory


On 12 February 2010 10:53, Pavel Klinkovsky <pavel.klinkovsky@gmail.com> wrote:
> Results of my measurements ('ls' in sources repository)
>
> Native Plan9:
> 1.05 s - very stable results
>
> Native Linux:
> 40-75 s - very spread results
>
> My "10x slower" estimation was too optimistic.
>
> Pavel
>
>

Re: [9fans] Speed of 9pfuse on Linux

[Pavel Klinkovsky]

> this is not surprising if you consider what's happening underneath:
Clear.

Understandable but very unpleasant.

Pavel

Re: [9fans] Speed of 9pfuse on Linux

[Eric Van Hensbergen]

A few "solutions" exist:

Use Op instead - of course there is no Op exporter from sources or
Linux client -- but these could be rectified with work.

If used v9fs run with cache=loose or cache=fscache both of which
enables caching meta-data.

Convince the Linux community to fix ls -  I think the kernel supports
Plan 9 style dirread now (there were some bug reports against v9fs
suggesting a plan 9 style dirread was possible) but tools haven't yet
been updated.

Submit a patch to 9pfuse and/or v9fs providing optional (mount time)
1s cache on metadata.

      -Eric

Sent from my iPhone

On Feb 12, 2010, at 12:15 PM, Pavel Klinkovsky <pavel.klinkovsky@gmail.com
 > wrote:

>> this is not surprising if you consider what's happening underneath:
> Clear.
>
> Understandable but very unpleasant.
>
> Pavel
>

Re: [9fans] Speed of 9pfuse on Linux

[Tim Newsham]

> Convince the Linux community to fix ls -  I think the kernel supports Plan 9
> style dirread now (there were some bug reports against v9fs suggesting a plan
> 9 style dirread was possible) but tools haven't yet been updated.

Any idea how "9 ls" does in p9p?

>    -Eric

Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com

Re: [9fans] Speed of 9pfuse on Linux

[Abhishek Kulkarni]

On Fri, Feb 12, 2010 at 12:51 PM, Tim Newsham <newsham@lava.net> wrote:
>> Convince the Linux community to fix ls -  I think the kernel supports Plan
>> 9 style dirread now (there were some bug reports against v9fs suggesting a
>> plan 9 style dirread was possible) but tools haven't yet been updated.
>
> Any idea how "9 ls" does in p9p?
>

9p ls sources
0.315s

9 ls /n/sources
4.012s

ls /n/sources
1.918s

averaged across 8 runs.

>>   -Eric
>
> Tim Newsham | www.thenewsh.com/~newsham | thenewsh.blogspot.com
>
>