From mboxrd@z Thu Jan 1 00:00:00 1970 MIME-Version: 1.0 In-Reply-To: References: <9D4F9D2A-6522-4223-807B-0B8F0F6AF68D@lsub.org> Date: Tue, 19 Feb 2013 01:36:06 +0000 Message-ID: From: Charles Forsyth To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Content-Type: multipart/alternative; boundary=f46d0444e97f8da60304d609dd50 Subject: Re: [9fans] going too far? Topicbox-Message-UUID: 1aa2d1c2-ead8-11e9-9d60-3106f5b1d025 --f46d0444e97f8da60304d609dd50 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 18 February 2013 19:23, andrey mirtchovski wrote= : > sorry, not all source files, just the 'import' section. In nemo's example, it seemed to have wandered off into go/src/cmd, reading 4k from everything there each time, which began to add up, if I've read the iostats correctly. As to caching, the kernel does cache (/sys/src/9/port/cache.c), but not as much as needed for that example. Streaming probably isn't interesting if you're anyway caching, unless the files are huge. The directory structure and metadata also matters. With floren's simpler example, I got a 0.30 elapsed time when I tried it: subito% time go clean github.com/floren/ellipsoid 0.03u 0.01s 0.30r go clean github.com/floren/ellipsoid With a caching system in the way (fscfs), the first time, the time goes up from that: subito% time go clean github.com/floren/ellipsoid 0.02u 0.04s 0.44r go clean github.com/floren/ellipsoid subito% cat /n/cached/cfsctl Client Server #calls =CE=94 ms/call =CE=94 #calls =CE=94 ms/call = =CE=94 1 1 0.670 0.670 1 1 0.660 0.660 Tversion 1 1 0.365 0.365 1 1 0.343 0.343 Tauth 1 1 0.495 0.495 1 1 0.484 0.484 Tattach 433 433 0.188 0.188 352 352 0.219 0.219 Twalk 333 333 0.223 0.223 332 332 0.115 0.115 Topen 359 359 0.202 0.202 359 359 0.189 0.189 Tread 3 3 0.242 0.242 3 3 0.236 0.236 Twrite 427 427 0.009 0.009 0 0 Tclunk 93 93 0.122 0.122 93 93 0.116 0.116 Tstat 36 36 ndirread 323 323 ndelegateread 0 0 ninsert 0 0 ndelete 0 0 nupdate 833411 833411 bytesread 113 113 byteswritten 777224 777224 bytesfromserver 56187 56187 bytesfromdirs 0 0 bytesfromcache 777050 777050 bytestocache The cache doesn't save any reading, because there is no redundancy, but it cuts the number of walk requests (because it's caching directory structure and there is overlap there). The next time subito% time go clean github.com/floren/ellipsoid 0.01u 0.01s 0.25r go clean github.com/floren/ellipsoid It has cut the calls through to the server dramatically, at the risk of getting out-of-date information, which could be a nuisance on a shared project. (Linux vfs doesn't have to worry about that because it's all local, and dangerous anyway.) The only opens and reads that do go through are for directory contents, because fscfs doesn't cache them (I didn't need that on Blue Gene). subito% cat /n/cached/cfsctl Client Server #calls =CE=94 ms/call =CE=94 #calls =CE=94 ms/call = =CE=94 1 0 0.670 1 0 0.660 Tversion 1 0 0.365 1 0 0.343 Tauth 1 0 0.495 1 0 0.484 Tattach 864 431 0.098 0.007 352 0 0.219 Twalk 666 333 0.119 0.016 349 17 0.115 0.112 Topen 717 358 0.153 0.105 603 244 0.172 0.147 Tread 3 0 0.242 3 0 0.236 Twrite 854 427 0.008 0.008 0 0 Tclunk 186 93 0.122 0.122 186 93 0.116 0.116 Tstat 72 36 ndirread 531 208 ndelegateread 0 0 ninsert 0 0 ndelete 0 0 nupdate 1666648 833237 bytesread 113 0 byteswritten 777224 0 bytesfromserver 112374 56187 bytesfromdirs 777050 777050 bytesfromcache 777050 0 bytestocache This is with IL/IP which is ever-so-slightly faster than TCP/IP (I tried them both), because there's less overhead, especially on the reads (that get through). I don't think streaming will help you, really; the main difference is not fetching it at all, and only if you're working with the same contents a lot; and you need to cache directory contents and other metadata, with all that entails. The iostats wasn't particularly strange for that one, so I wonder if there's something about nemo's example that gets {go clean ...} wandering through all the src/cmd stuff so often. --f46d0444e97f8da60304d609dd50 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

= On 18 February 2013 19:23, andrey mirtchovski <mirtchovski@gmail.com> wrote:
sorry, not all source files, just the 'import' sec= tion.

In nemo's example, it seemed to have wandered off into go/src= /cmd, reading 4k from everything there each time, which began to add up,
if I've read the iostats correctly.

As to= caching, the kernel does cache (/sys/src/9/port/cache.c), but not as much = as needed for that example.
Streaming= probably isn't interesting if you're anyway caching, unless the fi= les are huge.
The directory structure and metadata also = matters. With floren's simpler example, I got a 0.30 elapsed time when = I tried it:

subito% time go clean github= .com/floren/ellipsoid
0.03u 0.01s 0.30r= go clean github.com/floren/ellipsoid

With = a caching system in the way (fscfs), the first time, the time goes up from = that:

subito% time go clean github.com/floren/ellipsoid
0.02u 0.04s 0.44r go clean github.com/flo= ren/ellipsoid

subito% cat= /n/cached/cfsctl
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 Client =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0Server
=C2=A0 = =C2=A0#calls =C2=A0 =C2=A0 =CE=94 =C2=A0ms/call =C2=A0 =C2=A0=CE=94 =C2=A0 = =C2=A0 =C2=A0#calls =C2=A0 =C2=A0 =CE=94 =C2=A0ms/call =C2=A0 =C2=A0=CE=94<= /div>
=C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 = =C2=A0 0.670 =C2=A0 0.670 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 =C2= =A0 0.660 =C2=A0 0.660 Tversion
=C2=A0 =C2= =A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 0.365 =C2=A0 0.365 =C2=A0 =C2=A0= =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 0.343 =C2=A0 0.343 Tauth
=C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 0.495 =C2=A0 0.495 =C2= =A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 0.484 =C2=A0 0.484 Tattac= h
=C2=A0 =C2=A0 433 =C2=A0 =C2=A0 433 =C2= =A0 0.188 =C2=A0 0.188 =C2=A0 =C2=A0 352 =C2=A0 =C2=A0 352 =C2=A0 0.219 =C2= =A0 0.219 Twalk
=C2=A0 =C2=A0 333 =C2=A0 = =C2=A0 333 =C2=A0 0.223 =C2=A0 0.223 =C2=A0 =C2=A0 332 =C2=A0 =C2=A0 332 = =C2=A0 0.115 =C2=A0 0.115 Topen
=C2=A0 =C2=A0 359 =C2=A0 =C2=A0 359 =C2=A0 0.202= =C2=A0 0.202 =C2=A0 =C2=A0 359 =C2=A0 =C2=A0 359 =C2=A0 0.189 =C2=A0 0.189= Tread
=C2=A0 =C2=A0 =C2=A0 3 =C2=A0 =C2=A0= =C2=A0 3 =C2=A0 0.242 =C2=A0 0.242 =C2=A0 =C2=A0 =C2=A0 3 =C2=A0 =C2=A0 = =C2=A0 3 =C2=A0 0.236 =C2=A0 0.236 Twrite
=C2=A0 =C2=A0 427 =C2=A0 =C2=A0 427 =C2=A0 0.009 =C2=A0 0.009 =C2=A0 =C2=A0= =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 Tclunk
=C2=A0 =C2=A0 =C2=A093= =C2=A0 =C2=A0 =C2=A093 =C2=A0 0.122 =C2=A0 0.122 =C2=A0 =C2=A0 =C2=A093 = =C2=A0 =C2=A0 =C2=A093 =C2=A0 0.116 =C2=A0 0.116 Tstat
=C2=A0 =C2=A0 =C2=A036 =C2=A0 =C2=A0 =C2=A036 ndirread
=C2=A0 =C2=A0 323 =C2=A0 =C2=A0 323 ndelegaterea= d
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2= =A0 0 ninsert
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0= =C2=A0 =C2=A0 0 ndelete
=C2=A0 =C2=A0 =C2= =A0 0 =C2=A0 =C2=A0 =C2=A0 0 nupdate
=C2=A0833411 =C2=A0833411 bytesread
=C2=A0 =C2=A0 113 =C2=A0 =C2=A0 113 byteswritten
<= div class=3D"gmail_extra">=C2=A0777224 =C2=A0777224 bytesfromserver
=C2=A0 56187 =C2=A0 56187 bytesfromdirs
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 0 by= tesfromcache
=C2=A0777050 =C2=A0777050 byte= stocache

The cache doesn't save any reading, because there is no redund= ancy, but it cuts
the number of walk requests (because it= 9;s caching directory structure and there is
overlap there).

The next time

subito% time go clean github.com/floren/ellipsoid
0.0= 1u 0.01s 0.25r go clean= github.com/floren/ellipsoid=

It has cut the calls through to the server dramatically, a= t the risk
of getting out-of-date informati= on, which could be a nuisance on a shared project.
(Linux vfs doesn't have to worry about that = because it's all local, and dangerous anyway.)
The only opens and reads that do go through are for directory= contents,
because fscfs doesn't cache them (I di= dn't need that on Blue Gene).
subito% cat /n/cached/cfsctl
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Client =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Server
=C2=A0 =C2=A0#calls =C2=A0 =C2=A0 =CE=94 =C2=A0ms/call =C2= =A0 =C2=A0=CE=94 =C2=A0 =C2=A0 =C2=A0#calls =C2=A0 =C2=A0 =CE=94 =C2=A0ms/c= all =C2=A0 =C2=A0=CE=94
=C2=A0 =C2=A0 =C2= =A0 1 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 0.670 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 0.660 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Tversion
=C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 0 = =C2=A0 0.365 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2= =A0 =C2=A0 0 =C2=A0 0.343 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Tauth
=C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 0.495= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A0 =C2=A0 =C2=A0 0 = =C2=A0 0.484 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Tattach
=C2=A0 =C2=A0 864 =C2=A0 =C2=A0 431 =C2=A0 0.098 =C2=A0 0.007 =C2=A0 =C2=A0= 352 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 0.219 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Twalk<= /div>
=C2=A0 =C2=A0 666 =C2=A0 =C2=A0 333 =C2=A0 = 0.119 =C2=A0 0.016 =C2=A0 =C2=A0 349 =C2=A0 =C2=A0 =C2=A017 =C2=A0 0.115 = =C2=A0 0.112 Topen
=C2=A0 =C2=A0 717 =C2=A0= =C2=A0 358 =C2=A0 0.153 =C2=A0 0.105 =C2=A0 =C2=A0 603 =C2=A0 =C2=A0 244 = =C2=A0 0.172 =C2=A0 0.147 Tread
=C2=A0 =C2=A0 =C2=A0 3 =C2=A0 =C2=A0 =C2=A0 0 = =C2=A0 0.242 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 3 =C2=A0 =C2= =A0 =C2=A0 0 =C2=A0 0.236 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Twrite
=C2=A0 =C2=A0 854 =C2=A0 =C2=A0 427 =C2=A0 0.008 =C2=A0 0= .008 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Tclunk
=C2=A0 =C2=A0 186 =C2=A0 =C2=A0 =C2=A093 =C2=A0 0.122 =C2=A0 0.122 =C2=A0 = =C2=A0 186 =C2=A0 =C2=A0 =C2=A093 =C2=A0 0.116 =C2=A0 0.116 Tstat
=C2=A0 =C2=A0 =C2=A072 =C2=A0 =C2=A0 =C2=A036 ndirre= ad
=C2=A0 =C2=A0 531 =C2=A0 =C2=A0 208 ndel= egateread
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2= =A0 =C2=A0 0 ninsert
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 =C2=A0 0 nd= elete
=C2=A0 =C2=A0 =C2=A0 0 =C2=A0 =C2=A0 = =C2=A0 0 nupdate
1666648 =C2=A0833237 bytes= read
=C2=A0 =C2=A0 113 =C2=A0 =C2=A0 =C2=A0= 0 byteswritten
=C2=A0777224 =C2=A0 =C2=A0 =C2=A0 0 bytesfromser= ver
=C2=A0112374 =C2=A0 56187 bytesfromdirs=
=C2=A0777050 =C2=A0777050 bytesfromcache
=C2=A0777050 =C2=A0 =C2=A0 =C2=A0 0 bytestoc= ache

This is with IL/IP which is ever-so-slightly fast= er than TCP/IP (I tried them both), because there's less overhead,
especially on the reads (that get through).

I don't think streaming will help you, really; the= main difference is not fetching it at all,
and only if you= 're working with the same contents a lot;
and you need = to cache directory contents and other metadata, with all that entails.

The iostats wasn't particularly s= trange for that one, so I wonder if there's something about nemo's = example
that gets {go clean ...} wandering through all the = src/cmd stuff so often.
--f46d0444e97f8da60304d609dd50--