[9fans] browse /sys/src

[9fans] browse /sys/src

[YAMANASHI Takeshi]

Hi, I setup a src browsing gateway on a tip9ug server.
	http://www.tip9ug.jp/magic/srcgw/sys/src

HISTORY link is not much useful yet because I don't pull
regulary and there's almost nothing in /n/dump.  Hope the
gateway finds sourcesdump soon.

Also, a mirror of Installation CD's iso image is also
available on the server.  Please read the License before
download the image.
	http://plan9.bell-labs.com/plan9dist/license.html
	http://www.tip9ug.jp/mirror/plan9.iso.bz2

The image is updated daily.

Hope these will help.
-- 
Sincerely, 
YAMANASHI Takeshi

Re: [9fans] browse /sys/src

[Kenji Okamoto]

> Hi, I setup a src browsing gateway on a tip9ug server.
> 	http://www.tip9ug.jp/magic/srcgw/sys/src
> 
> HISTORY link is not much useful yet because I don't pull
> regulary and there's almost nothing in /n/dump.  Hope the
> gateway finds sourcesdump soon.

I'm not offending your efforts basically.
It said first, however, I'd like to know the process of discussions 
where you and sources members and also this 9fans community 
members.   I don't want to discuss LICENSE frames, however,
I believe we have more basic moral to make refer the others
works...

Making source trees available to anyone who want it may help
the community or may not.   I think it's not so clear to me yet, 
only to me?

Kenji

Re: [9fans] browse /sys/src

[Michael H. Collins]

yes. i call bullshit

too

Kenji Okamoto wrote:
>>Hi, I setup a src browsing gateway on a tip9ug server.
>>	http://www.tip9ug.jp/magic/srcgw/sys/src
>>
>>HISTORY link is not much useful yet because I don't pull
>>regulary and there's almost nothing in /n/dump.  Hope the
>>gateway finds sourcesdump soon.
> 
> 
> I'm not offending your efforts basically.
> It said first, however, I'd like to know the process of discussions 
> where you and sources members and also this 9fans community 
> members.   I don't want to discuss LICENSE frames, however,
> I believe we have more basic moral to make refer the others
> works...
> 
> Making source trees available to anyone who want it may help
> the community or may not.   I think it's not so clear to me yet, 
> only to me?
> 
> Kenji
> 
> 
> 


-- 
Michael H. Collins  Admiral, Penguinista Navy

http://linuxlink.com

/"\    	ASCII Ribbon Campaign
\ /    	No HTML/RTF in email
  x     	No Word docs in email
/ \    	Respect for open standards

Take your laptop and yell out:
"Can a brother get a ip address?"

Re: [9fans] browse /sys/src

[boyd, rounin]

> yes. i call bullshit

the Repo code ...
--
MGRS 31U DQ 52572 12604

Re: [9fans] browse /sys/src

[Russ Cox]

No need to jump on Yamanashi.  He checked with us
at Bell Labs before announcing it.  The consensus here
is that making the LICENSE file obvious on the site and
in the downloaded tgzs is sufficient.

Lucent insists on the click-through for the official distribution,
but the terms of the Lucent Public License do not require
it of others.  (This is in contrast to the Plan 9 License,
which did.)  There is at least already one external site 
already that makes much of the Plan 9 source available
without a click-through: the plan9port site.

I believe the CMU 3rd ed. browser was shut down because
the person running it moved on and could no longer
maintain it.

I think the web interface to sources that Yamanashi set
up is quite nice.  

Russ

Re: [9fans] browse /sys/src

[Kenji Okamoto]

> Lucent insists on the click-through for the official distribution,
> but the terms of the Lucent Public License do not require
> it of others.  (This is in contrast to the Plan 9 License,
> which did.)  There is at least already one external site 
> already that makes much of the Plan 9 source available
> without a click-through: the plan9port site.

I believe, because I haven't got it, plan9port is only a part of Plan9,
I mean, by only those we cannot run Plan 9.

> I think the web interface to sources that Yamanashi set
> up is quite nice.  

I doubt this.
Web can be reached from anyone who wants, even he/she is
a terrorist etc, I don't mean the word same as Iqaqi etc.

We, academic community is 'open', anyone can get the resources
we developed.   However, if you want to enter the community,
you have to be evaluated before.   In this sense, academic community
is not completely 'open'.   However, we don't believe it's not open.
I'd like the Plan 9 community is as such.

I don't brame click-through procedure of Lucent, because it's just
like the above I mentioned.   I still believe moral term.

Kenji

Re: [9fans] browse /sys/src

[Kenji Okamoto]

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

I forgot one thing.

I don't say that the sources should be limited to Lucent alone.
However, this issue should be discussed before making decision.

Kenji

[-- Attachment #2: Type: message/rfc822, Size: 4358 bytes --]

From: Kenji Okamoto <okamoto@granite.cias.osakafu-u.ac.jp>
To: russcox@gmail.com, 9fans@cse.psu.edu
Cc: 
Subject: Re: [9fans] browse /sys/src
Date: Wed, 26 Jan 2005 13:28:54 +0900
Message-ID: <2aab0c267882f16e9ce77d5b9ff0f5a8@granite.cias.osakafu-u.ac.jp>

> Lucent insists on the click-through for the official distribution,
> but the terms of the Lucent Public License do not require
> it of others.  (This is in contrast to the Plan 9 License,
> which did.)  There is at least already one external site 
> already that makes much of the Plan 9 source available
> without a click-through: the plan9port site.

I believe, because I haven't got it, plan9port is only a part of Plan9,
I mean, by only those we cannot run Plan 9.

> I think the web interface to sources that Yamanashi set
> up is quite nice.  

I doubt this.
Web can be reached from anyone who wants, even he/she is
a terrorist etc, I don't mean the word same as Iqaqi etc.

We, academic community is 'open', anyone can get the resources
we developed.   However, if you want to enter the community,
you have to be evaluated before.   In this sense, academic community
is not completely 'open'.   However, we don't believe it's not open.
I'd like the Plan 9 community is as such.

I don't brame click-through procedure of Lucent, because it's just
like the above I mentioned.   I still believe moral term.

Kenji

Re: [9fans] browse /sys/src

[Skip Tavakkolian]

> We, academic community is 'open', anyone can get the resources
> we developed.   However, if you want to enter the community,
> you have to be evaluated before.   In this sense, academic community
> is not completely 'open'.   However, we don't believe it's not open.
> I'd like the Plan 9 community is as such.

I don't.

BTW, Plan9 is the most practical system I have. I can access all my data
all the time from anywhere.  It's secure, reliable and resilient. It's wrong
to reinforce the impression that Plan9 is some academic exercise.

Re: [9fans] browse /sys/src

[Matthias Teege]

> all the time from anywhere.  It's secure, reliable and resilient. It's wrong
> to reinforce the impression that Plan9 is some academic exercise.

I don't know if I understand Kenji Okamotos point but I agree with
Skip in this point.  Plan 9 isn't a toy.  It is one of it strengths
that it has left behind the academic exercise stage.  I don't know any
other "alternative" system which is usefull (today) like Plan 9 is.
Ok, maybe Minix works but it's yust another UNIX.  ;-)

Matthias

[9fans] fossil speed

[rog]

has anyone else found fossil to be very slow?  even when it's not
using venti at all (e.g.  extracting a large tar archive).

crudely comparing the speed of my local fossil on a T22 laptop to a
local linux box, it seems to perform operations anywhere from 100 to
200 times as slow.  (again, this is totally crude; i just extracted
the firefox source code, 300MB uncompressed, and then removed it).

even allowing for a factor of 10 in drive/driver speed, this
still seems like a lot.

is there something that can be done about this?  maybe i've just got
things set up wrong.

day to day i don't really use big files so this hadn't impacted me
until now.

  cheers,
    rog.

Re: [9fans] fossil speed

[andrey mirtchovski]

> has anyone else found fossil to be very slow?  even when it's not
> using venti at all (e.g.  extracting a large tar archive).
> 

"almost" identical machines using ide disks (2.0 vs 2.6ghz cpu's):

on plan9:

	plan9% time tar cv xen | gzip > xen.tgz
	a xen/9pccpu 3189 blocks
	a xen/etc_xen_plan9 7 blocks
	a xen/kfs_root_image.gz 298211 blocks
	0.42u 6.48s 56.15r 	 tar cv xen
	plan9%

then this very same file was copied and untarred on the linux box:

	$ time tar cvf - xen | gzip > xen.tgz
	xen/
	xen/9pccpu
	xen/etc_xen_plan9
	xen/kfs_root_image.gz
	
	real    0m29.123s
	user    0m22.805s
	sys     0m2.635s
	$ 

i have both rwm and dma turned on.  the disks are IDE in both cases.
for Plan 9, fossil is mounted through a devfs mirror of two disks,
which should slow it down even more.

andrey

Re: [9fans] fossil speed

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 32 bytes --]

what happens when you extract?

[-- Attachment #2: Type: message/rfc822, Size: 2932 bytes --]

From: andrey mirtchovski <mirtchov@cpsc.ucalgary.ca>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] fossil speed
Date: Wed, 26 Jan 2005 14:36:29 -0700
Message-ID: <bcad6afcd37a3dc9f5ec58a336592223@plan9.ucalgary.ca>

> has anyone else found fossil to be very slow?  even when it's not
> using venti at all (e.g.  extracting a large tar archive).
> 

"almost" identical machines using ide disks (2.0 vs 2.6ghz cpu's):

on plan9:

	plan9% time tar cv xen | gzip > xen.tgz
	a xen/9pccpu 3189 blocks
	a xen/etc_xen_plan9 7 blocks
	a xen/kfs_root_image.gz 298211 blocks
	0.42u 6.48s 56.15r 	 tar cv xen
	plan9%

then this very same file was copied and untarred on the linux box:

	$ time tar cvf - xen | gzip > xen.tgz
	xen/
	xen/9pccpu
	xen/etc_xen_plan9
	xen/kfs_root_image.gz
	
	real    0m29.123s
	user    0m22.805s
	sys     0m2.635s
	$ 

i have both rwm and dma turned on.  the disks are IDE in both cases.
for Plan 9, fossil is mounted through a devfs mirror of two disks,
which should slow it down even more.

andrey

Re: [9fans] fossil speed

[andrey mirtchovski]

i managed to find a way of slowing down fossil even more (while discussing
this and how to include anything past the pipe in the timing on irc) by
working with many small files in a single directory:

plan9% ls b24 | wc -l; du b24
    3601
11827	b24
plan9% time tar cf b24.tar b24
0.05u 0.75s 7.31r 	 tar cf b24.tar b24
plan9% time tar xf b24.tar
0.09u 0.94s 4.99r 	 tar xf b24.tar
plan9%

$ time tar cf b24.tar b24

real    0m0.251s
user    0m0.060s
sys     0m0.175s
$ time tar xf b24.tar

real    0m1.291s
user    0m0.062s
sys     0m1.207s
$


repeated timings get the factor of difference up to 7, but still way below
the one you're observing..

Re: [9fans] fossil speed

[andrey mirtchovski]

here's the extract:

plan9% time tar xf xen.tar
0.22u 5.12s 39.09r 	 tar xf xen.tar
plan9%

$ time tar xf xen.tar

real    0m15.771s
user    0m0.048s
sys     0m1.388s
$

Re: [9fans] fossil speed

[rog]

% time rc -c 'bunzip2 < firefox.tar.bz2 | dd -bs 512k | tar x'
63.41u 44.98s 1888.55r 	 rc -c bunzip2 < firefox.tar.bz2 | dd -bs 512k | tar x
% time rm -r mozilla
0.72u 5.21s 691.54r 	 rm -r mozilla
% 

the directory contained 41166 files.
dma and rwm were on.

Re: [9fans] fossil speed

[rog]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

that's not too bad.

i fancy that the big difference i was seeing might have
been to do with the fact that i was creating (or destroying)
tens of thousands of tiny files, whereas your example contains
only three. also, isn't gzip somewhat more compute-intensive
than gunzip, in which case maybe what you were doing
was more cpu-bound?

one example:

	linux% time rm -r mozilla
	
	real	0m3.621s
	user	0m0.090s
	sys	0m1.180s
	linux% 

an equivalent under fossil, removing only half that
number of files took more than 5 minutes.

another:

	linux% ls -l *.bz2
	-rw-r--r--    1 rog      rog      32686249 Jan 26 20:54 firefox.tar.bz2
	linux% time sh -c 'bunzip2 < firefox.tar.bz2 | dd 'bs=512k' | tar xf -'
	
	real	1m23.266s
	user	0m36.600s
	sys	0m5.050s
	linux% 

(the big blocksize dd is there to try and reduce disk contention
between the bunzip process and the tar write).

when i did this using fossil i killed it after an hour had passed.
thanks for pointing out the dma/rwm - i hadn't in fact enabled these
when network booting.  however, even after enabling them, it still
going to take 20-40 minutes (estimate varies a bit)

it might very well be my slow disk, but i'd like to hear how other
people's systems compare.

i wonder if perhaps the rather complex fossil/venti directory structure
might be slowing things down?  also, my fossil cache size is 2000
blocks.  i would imagine this'd be sufficient to cache
parent-directory meta data, but perhaps i'm wrong.

[-- Attachment #2: Type: message/rfc822, Size: 2922 bytes --]

From: andrey mirtchovski <mirtchov@cpsc.ucalgary.ca>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] fossil speed
Date: Wed, 26 Jan 2005 14:36:29 -0700
Message-ID: <bcad6afcd37a3dc9f5ec58a336592223@plan9.ucalgary.ca>

> has anyone else found fossil to be very slow?  even when it's not
> using venti at all (e.g.  extracting a large tar archive).
> 

"almost" identical machines using ide disks (2.0 vs 2.6ghz cpu's):

on plan9:

	plan9% time tar cv xen | gzip > xen.tgz
	a xen/9pccpu 3189 blocks
	a xen/etc_xen_plan9 7 blocks
	a xen/kfs_root_image.gz 298211 blocks
	0.42u 6.48s 56.15r 	 tar cv xen
	plan9%

then this very same file was copied and untarred on the linux box:

	$ time tar cvf - xen | gzip > xen.tgz
	xen/
	xen/9pccpu
	xen/etc_xen_plan9
	xen/kfs_root_image.gz
	
	real    0m29.123s
	user    0m22.805s
	sys     0m2.635s
	$ 

i have both rwm and dma turned on.  the disks are IDE in both cases.
for Plan 9, fossil is mounted through a devfs mirror of two disks,
which should slow it down even more.

andrey

Re: [9fans] fossil speed

[Charles Forsyth]

ah. which linux system is being used in the office?

Re: [9fans] fossil speed

[Charles Forsyth]

>>ah. which linux system is being used in the office?

sorry: i meant: which machine (host name)

Re: [9fans] fossil speed

[andrey mirtchovski]

On Wed, 26 Jan 2005 rog@vitanuova.com wrote:

> 	linux% time rm -r mozilla
>
> 	real	0m3.621s
> 	user	0m0.090s
> 	sys	0m1.180s
> 	linux%
>
> an equivalent under fossil, removing only half that
> number of files took more than 5 minutes.

here's what i got here:

$ time rm -rf mozilla
real    0m6.685s
user    0m0.166s
sys     0m1.637s
$

plan9% time rm -rf mozilla
0.12u 3.01s 152.24r 	 rm -rf mozilla
plan9%

Re: [9fans] fossil speed

[Russ Cox]

fossil is not smart about its handling of directory
entries, so removing n files is an O(n^2) operation.
a directory with 40,000 files is going to take a little
while to remove.  200 directories of 200 files each
should be noticeably faster.

russ

Re: [9fans] fossil speed

[rog]

> fossil is not smart about its handling of directory
> entries, so removing n files is an O(n^2) operation.
> a directory with 40,000 files is going to take a little
> while to remove.  200 directories of 200 files each
> should be noticeably faster.

i don't know whether that's an issue here.  for the directory in
question, 65% of directories have 4 or less entries, and 95% have less
than 30.

Re: [9fans] fossil speed

[Fco. J. Ballesteros]

> thanks for pointing out the dma/rwm - i hadn't in fact enabled these
> when network booting.  however, even after enabling them, it still

Any of you using rwm with ≥2 ide disks in the same machine?
thanks

Re: [9fans] fossil speed

[geoff]

I have a ken fs (not kfs) with 4 IDE disks using dma and rwm
on a Promise controller.  It works, but this is using the new
64-bit fs kernel with entirely new IDE code.  jmk's got the code.

Re: [9fans] fossil speed

[andrey mirtchovski]

> I have a ken fs (not kfs) with 4 IDE disks using dma and rwm
> on a Promise controller.  It works, but this is using the new
> 64-bit fs kernel with entirely new IDE code.  jmk's got the code.

great! what's the maximum length of a file name in the new ken-fs?

Re: [9fans] fossil speed

[Russ Cox]

whatever you #define NAMELEN to.


On Thu, 27 Jan 2005 14:23:10 -0700, andrey mirtchovski
<mirtchov@cpsc.ucalgary.ca> wrote:
> > I have a ken fs (not kfs) with 4 IDE disks using dma and rwm
> > on a Promise controller.  It works, but this is using the new
> > 64-bit fs kernel with entirely new IDE code.  jmk's got the code.
> 
> great! what's the maximum length of a file name in the new ken-fs?
> 
>

Re: [9fans] fossil speed

[geoff]

File names are unbounded, as always in Plan 9 (and originally in Unix,
before Unix was ``fixed'').

File name *components* are limited to NAMELEN-1 bytes (not characters).
This was 27 ASCII characters or 9 Japanese characters.  In the new ken fs,
I've made it 55 ASCII characters or 18 Japanese characters, which turns
out to be enough to handle all but one of the file names that lnfs on
my machines recorded (and that was one was some obscene XML file name).
When you install the new ken fs, you can crank NAMELEN up higher if you want,
though you'll then get few directory entries per block.

Re: [9fans] fossil speed

[Kenji Okamoto]

Hi geoff.

> In the new ken fs,
> I've made it 55 ASCII characters or 18 Japanese characters, 

I have some long filename by a Windows user like:
履修指針(専門基礎ー化学)12ー1
yeah, it's safe, however, more long filename can be found, I suppose.

> When you install the new ken fs, you can crank NAMELEN up higher if you want,
> though you'll then get few directory entries per block.

So, the file trees already there on now ken's file server must be reinstalled
for your new fileserver?

Kenji

Re: [9fans] fossil speed

[geoff]

It is possible to generate file server kernels exactly compatible with
existing ken fs on-disk file systems, but you get none of the benefits
of the new file server if you do that.  It does however let you build
old and new file server kernels from the same sources.

If you want to get the benefits of the new file server (64-bit file
sizes, offsets and block numbers; smarter IDE code ported from the cpu
kernel; larger file name components; indirect blocks beyond
double-indirect; etc.), you have to build a new file server kernel
that will not be compatible on disk with existing file systems (it
will however speak 9p2000 compatibly, though not 9p1, because 9p1
depends crucially on NAMELEN being 28; 9p1 is disabled in the new file
server kernels, but it turns out that nothing needs 9p1 any more [old
drawterm uses it, but it never connects to file servers directly]).

I decided that I could live without the history on my old file server,
so just copied its main file system to a new file server.  It's nice
to not need lnfs any more.

Re: [9fans] fossil speed

[Kenji Okamoto]

Thanks geoff for your kind reply.
Yes, 9p2000 is good for all of us, I agree.

> I decided that I could live without the history on my old file server,
> so just copied its main file system to a new file server.  It's nice
> to not need lnfs any more.

Ok, I got it.

How about il protocol?
Does the new ken fileserver use TCP?

Kenji  -- still a user of ken's file server and fossil

Re: [9fans] fossil speed

[geoff]

It still requires IL.  I would like to make it use TCP,
at least as an option, but that's a bigger project.
(I think it requires lifting up the `upper layers' of
the fs kernel and dropping them onto a cpu kernel.)

I wanted to get my work to date out to the rest of you
in its current condition, since progress has been slow
since November.

Re: [9fans] fossil speed

[boyd, rounin]

try comparing that to an 11/780 with 4Mb of ram and some rm-0[35]s.
--
MGRS 31U DQ 52572 12604

Re: [9fans] browse /sys/src

[Fco. J. Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

The interface for browsing is really nice. It's also nice
to be able to use the dump from there :-)

Thanks a lot.

[-- Attachment #2: Type: message/rfc822, Size: 3067 bytes --]

From: YAMANASHI Takeshi <9.nashi@gmail.com>
To: 9fans@cse.psu.edu
Subject: [9fans] browse /sys/src
Date: Wed, 26 Jan 2005 11:00:32 +0900
Message-ID: <6abdda19759c2c9e08a6a15e7e871166@orthanc.cc.titech.ac.jp>

Hi, I setup a src browsing gateway on a tip9ug server.
	http://www.tip9ug.jp/magic/srcgw/sys/src

HISTORY link is not much useful yet because I don't pull
regulary and there's almost nothing in /n/dump.  Hope the
gateway finds sourcesdump soon.

Also, a mirror of Installation CD's iso image is also
available on the server.  Please read the License before
download the image.
	http://plan9.bell-labs.com/plan9dist/license.html
	http://www.tip9ug.jp/mirror/plan9.iso.bz2

The image is updated daily.

Hope these will help.
-- 
Sincerely, 
YAMANASHI Takeshi