[9fans] ports performance

[9fans] ports performance

[Scott Schwartz]

I've noticed that running (the ported version of) ls under OS X (10.2),
it is very very slow.  Profiling tools (is gprof+"recompile the world"
all there is?  why doesn't everyone have pixie!) didn't reveal much,
but top shows that lookupd is burning lots of cycles.  Apparently we're
doing lots of lookups (of passwd entries I guess) that the native version
doesn't do?

Under linux the difference is very much less, but it's there.  At first
I attributed it to something minor like bio doing less buffering than
stdio, but maybe that's not the whole story.

Re: [9fans] ports performance

[rog]

[-- Attachment #1: Type: text/plain, Size: 51 bytes --]

it probably doesn't cache user-id->name mappings.

[-- Attachment #2: Type: message/rfc822, Size: 2171 bytes --]

From: Scott Schwartz <schwartz@bx.psu.edu>
To: 9fans@cse.psu.edu
Subject: [9fans] ports performance
Date: Thu, 08 Apr 2004 14:51:25 -0400
Message-ID: <E1BBecX-0000je-00@ella.bx.psu.edu>

I've noticed that running (the ported version of) ls under OS X (10.2),
it is very very slow.  Profiling tools (is gprof+"recompile the world"
all there is?  why doesn't everyone have pixie!) didn't reveal much,
but top shows that lookupd is burning lots of cycles.  Apparently we're
doing lots of lookups (of passwd entries I guess) that the native version
doesn't do?

Under linux the difference is very much less, but it's there.  At first
I attributed it to something minor like bio doing less buffering than
stdio, but maybe that's not the whole story.

Re: [9fans] ports performance

[Rob Pike]

because of the way ls works on plan 9, a simple port to unix makes
every ls as expensive as ls -l.

-rob

Re: [9fans] ports performance

[Scott Schwartz]

| because of the way ls works on plan 9, a simple port to unix makes
| every ls as expensive as ls -l.

Of course, and I expected that, but I'm seeing something much slower, like
several minutes for "ls -l /usr/bin /usr/bin /usr/bin /usr/bin /usr/bin".

Re: [9fans] ports performance

[Geoff Collyer]

On OS X, getpwuid() of a uid that isn't in the netinfo/lookupd magic
databases is excruciatingly slow.  I turn off as much of that goo as I
can and make it just read /etc/passwd, etc.  netinfo is just another
yellow pages (nis for the young ones).

Re: [9fans] ports performance

[Russ Cox]

It is in fact getpwuid and getgrgid that are taking
the time, not the extra stat calls.  (Quite stupid
given that I'm asking the same questions over
and over again -- you'd think the routines would
not need to work so hard the second time!)

I just checked into CVS a version of _p9dir that
caches the last uid and gid it looked up.  This
should speed up the overwhelming majority of
cases.  I built a new tarball too, since there have
been a handful of other minor changes.

In response to the utility comment, if you don't
want Plan 9's ls, you can put /bin in your path
before /usr/local/plan9/bin, and never worry
again.  Personally, I like the fact that Plan 9's
ls -l is more compact on the screen, that it does
quoting of special names for me, and that I don't
have to learn the differences between all the 
different ls commands on the various Unixes
I have to use.  Ditto (even more so) for ps.

Russ

Re: [9fans] ports performance

[Scott Schwartz]

| I just checked into CVS a version of _p9dir that

stat2dir.c and _p9dir.c are very similar...  should they be refactored
do you think?

Re: [9fans] ports performance

[Russ Cox]

stat2dir.c was a dreg.  it's gone now.

Re: [9fans] ports performance

[Russ Cox]

> it is very very slow.  Profiling tools (is gprof+"recompile the world"
> all there is?  why doesn't everyone have pixie!) didn't reveal much,
> but top shows that lookupd is burning lots of cycles.  Apparently we're

Slightly off-topic, but pct (see http://pdos.lcs.mit.edu/cb/pct/)
provides the functionality of Plan 9's tprof and, like tprof, doesn't
require recompiling.  It probably wouldn't have found your problem
(ls wasn't the one chewing cycles), but I've found it much more
useful than gprof.  The invocation is a little odd.  It really needs a 
better wrapper.

Russ

Re: [9fans] ports performance

[Ben Huntsman]

It's nice and fun to be able to run Plan 9's ls on other systems, but really quite useless, when there's already an ls on every unix.

>>> Rob Pike<robpike@gmail.com> 4/8/2004 11:57:43 AM >>>
because of the way ls works on plan 9, a simple port to unix makes
every ls as expensive as ls -l.

-rob

Re: [9fans] ports performance

[rog]

> It's nice and fun to be able to run Plan 9's ls on other systems, but
> really quite useless, when there's already an ls on every unix.

plan 9's is more useful.

not to mention i can remember how the options work...

Re: [9fans] ports performance

[a]

// It's nice and fun to be able to run Plan 9's ls on other
// systems, but really quite useless, when there's already
// an ls on every unix.

as rog already pointed out, plan9's is often better than the
vendor-supplied one. and almost always has a smaller set of
options. and, most importantly, those options are the same
across every platform, as is their output.
ア

Re: [9fans] ports performance

[Scott Schwartz]

| It's nice and fun to be able to run Plan 9's ls on other systems, but
| really quite useless, when there's already an ls on every unix.

The point it to understand what's happening, so we know if something
in a port library needs fixing.