Re: [9fans] Performance

Re: [9fans] Performance

[Richard Miller]

> Of course, on a multiprocessor, something can be put on the run
> queue by another processor without an external event occurring on
> this processor, so the HLT instruction isn't used in that case.

Isn't there a race condition here even on a uniprocessor?  The scan
of the run queue and call to HLT are done with interrupts enabled:

 - scheduler scans run queue, finds no ready process
 - scheduler calls halt()
 - interrupt occurs, placing process P on the run queue
 - halt() executes HLT instruction
 - dispatch of process P is delayed until (worst case) the next
   clock interrupt

-- Richard

Re: [9fans] Performance

[Douglas A. Gwyn]

Richard Miller wrote:
> Isn't there a race condition here even on a uniprocessor?  The scan
> of the run queue and call to HLT are done with interrupts enabled:

Yes, the canonical solution, which addresses HLT on MP systems
as well, is that whatever modifies the run queue always as a side
effect wakes up all the dispatchers.  Some platforms support use
of a software interrupt just as though it were a hardware
interrupt; on others a kludge would be required, or else just
leave it up to the clock interrupt.

Re: [9fans] Performance

[jmk]

On Fri May 11 04:42:36 EDT 2001, DAGwyn@null.net wrote:
> jmk@plan9.bell-labs.com wrote:
> > I think the correct kludge is to wait for the clock interrupt.
> 
> Also the most portable.

You learn something new every day. Charles Forsyth pointed out to me
that had I read the x86 description of the STI command as well as HLT
I would see there is no possibility of an interrupt occurring between
the sequence
	STI
	HLT
as the STI allows interrupts to be responded to after the completion
of the following instruction, and HLT is deemed to have completed when
it enters its halt state. Therefore, I think a possible way to close
the window on the x86 is

TEXT halt(SB), $0
	CLI
	CMPL	nrdy(SB), $0
	JEQ	_nothingready
	STI
	RET

_nothingready:
	STI
	HLT
	RET

Whether this is coded correctly or makes a difference is left as an
exercise for the student.

--jim

Re: [9fans] Performance

[Douglas A. Gwyn]

jmk@plan9.bell-labs.com wrote:
> I think the correct kludge is to wait for the clock interrupt.

Also the most portable.

Re: [9fans] Performance

[jmk]

On Thu May 10 11:31:55 EDT 2001, DAGwyn@null.net wrote:
> Richard Miller wrote:
> > Isn't there a race condition here even on a uniprocessor?  The scan
> > of the run queue and call to HLT are done with interrupts enabled:
> 
> Yes, the canonical solution, which addresses HLT on MP systems
> as well, is that whatever modifies the run queue always as a side
> effect wakes up all the dispatchers.  Some platforms support use
> of a software interrupt just as though it were a hardware
> interrupt; on others a kludge would be required, or else just
> leave it up to the clock interrupt.

Waking up other processors would require interprocessor interrupts
and that way lies madness.

A software interrupt on a uniprocessor wouldn't fix the race.

I think the correct kludge is to wait for the clock interrupt.

Re: [9fans] Performance

[Douglas A. Gwyn]

Russ Cox wrote:
> On the other hand, lmbench might be more suited to
> Plan 9 than to Unix.  At least on Plan 9 you can
> be more certain that you're measuring something real,
> as opposed to code that gcc has optimized into the
> empty instruction sequence.

That's a standard problem whenever benchmarks become a
de facto standard that affects acquisition decisions.
A compiler *might* detect that a benchmark source is
being compiled and turn it into the equivalent of
#include <stdio.h>
int main(void) { printf("0.0001 usec.\n); return 0; }

Re: [9fans] Performance

[Juhani Forsman]

i'm also working with this issue by comparing cluster of cpu 
servers with beowulf cluster and real smp/mpp boxes and i'll 
send the url of results to 9fans after the work is completed. 


On Wed, 9 May 2001, Franklin Robert Araujo França wrote:

> In fact, I had studied about Plan 9 performance against Linux and Sun OS. I
> have a paper with all the results. That paper is one of the results of my
> Master Thesis. The advisor for this Thesis was Professor Ph.D Celio
> Guimaraes. I'll send the results for this list as soon as possible.
> 
> 
> presotto@plan9.bell-labs.com wrote:
> 
> > Not that i know of
> >
> >   ------------------------------------------------------------------------
> >
> > Subject: [9fans] Performance
> > Date: Tue, 08 May 2001 10:42:15 -0700
> > From: "Fariborz 'Skip' Tavakkolian" <fst@real.com>
> > Reply-To: 9fans@cse.psu.edu
> > To: 9fans@cse.psu.edu
> >
> > Are there any published performance numbers more recent than those
> > mentioned in 9 and net papers? Any for typical PC platforms compared
> > to Linux or FreeBSD?
> 
> 
> 
> 

Re: [9fans] Performance

[presotto]

Or to put it more simply, if there was nothing to run, the problem
is waiting for something to happen outside of the cpu, usually a
remote file system.  Were you running with a local file system or
a file server.  If the former, you were probably waiting for disk,
if the latter, for the file server.

Re: [9fans] Performance

[jmk]

On Wed May  9 04:44:05 EDT 2001, pschay@pobox.com wrote:
> Hi,
> 
> There are many things besides kernel compilation that
> are also really fast compared to other systems: booting,
> compiling the window system, running the window system,
> ....
> 
> Anyway, as fast as kernel compilation is, I do have some
> questions about the performance.  I tried a few ways of
> profiling the kernel during kernel compilation.  At first
> I found that a lot of the kernel time was spent in halt()
> so I changed HLT to NOP in l.s.  Now almost a third of the
> time is apparently spent in runproc().
> 
> Is this a fixable problem?  Or is it actually the desired
> behavior?  (All I can imagine is that maybe the scheduler
> sacrifices the good of the few for the good of the many and
> me + my measly kernel compilation processes are not the
> common case?  Or perhaps scheduling typically consumes this
> kind of overhead?)  I know zilch about schedulers so
> I'd greatly appreciate any insight, or pointers to
> literature.  I think I recall reading a brief posting a year
> or two ago by one of the experts who said something to the
> effect that the scheduler has lots of room for improvement.
> Why? How? Could anybody elaborate?
> 
> Thanks.
> -Pete

The system looks for processes to run by scanning the run queues
in runproc(). If it doesn't find anything it just keeps looking until
something is put in the queue. Nothing can be put in the queue
except by an external event happening, i.e. some interrupt causes
a process to change state. The HLT instruction waits until an
interrupt occurs then continues. As you observed, you can replace
HLT with a NOP (or take the call to the routine out entirely) to
no ill effect, and this is what the system used to do. However,
HLT gives the processor opportunities to conserve power and thereby
run cooler and lengthen battery life, etc.

Of course, on a multiprocessor, something can be put on the run
queue by another processor without an external event occurring on
this processor, so the HLT instruction isn't used in that case.

--jim

Re: [9fans] Performance

[forsyth]

>>Anyway, as fast as kernel compilation is, I do have some
>>questions about the performance.  I tried a few ways of
>>profiling the kernel during kernel compilation.  At first
>>I found that a lot of the kernel time was spent in halt()
>>so I changed HLT to NOP in l.s.  Now almost a third of the
>>time is apparently spent in runproc().

it's in runproc waiting for something to do (hence the HLT in later versions).
perhaps your action is i/o bound.

Re: [9fans] Performance

[Peter Schay]

Hi,

There are many things besides kernel compilation that
are also really fast compared to other systems: booting,
compiling the window system, running the window system,
....

Anyway, as fast as kernel compilation is, I do have some
questions about the performance.  I tried a few ways of
profiling the kernel during kernel compilation.  At first
I found that a lot of the kernel time was spent in halt()
so I changed HLT to NOP in l.s.  Now almost a third of the
time is apparently spent in runproc().

Is this a fixable problem?  Or is it actually the desired
behavior?  (All I can imagine is that maybe the scheduler
sacrifices the good of the few for the good of the many and
me + my measly kernel compilation processes are not the
common case?  Or perhaps scheduling typically consumes this
kind of overhead?)  I know zilch about schedulers so
I'd greatly appreciate any insight, or pointers to
literature.  I think I recall reading a brief posting a year
or two ago by one of the experts who said something to the
effect that the scheduler has lots of room for improvement.
Why? How? Could anybody elaborate?

Thanks.
-Pete

PS In case anybody is curious, here are some data from
my experiments today:

There was not much difference between a compilation with 1
process and a compilation with 32.  Here is the output of
time and kprof (fyi I'm running an AMD K6-300 with 128MB
of SIMMs.  I did a mk; mk clean before each of these:)

26.36u 21.46s 67.63r 	 mk CONF=pcdisk
total: 66804	in kernel text: 39312	outside kernel text: 27492
KTZERO 80100000
ms	  %	sym
11160	 28.3	halt
6108	 15.5	i8259isr
3852	  9.7	memmove
3276	  8.3	memset
1596	  4.0	lock
1200	  3.0	envgen

26.46u 24.36s 66.20r 	 mk NPROC=32 CONF=pcdisk
total: 65184	in kernel text: 37644	outside kernel text: 27540
KTZERO 80100000
ms	  %	sym
7008	 18.6	halt
6312	 16.7	i8259isr
4404	 11.6	memmove
3804	 10.1	memset
1680	  4.4	lock
1308	  3.4	envgen

# Next 2 compilations were done with the
# kernel in which I substituted NOP for HLT in halt() in l.s:

25.84u 21.89s 67.30r 	 mk CONF=pcdisk
total: 66276	in kernel text: 39264	outside kernel text: 27012
KTZERO 80100000
ms	  %	sym
11484	 29.2	runproc
6240	 15.8	i8259isr
3816	  9.7	memmove
3564	  9.0	memset
1368	  3.4	lock
1104	  2.8	envgen

653809t + 489696d + 87400b = 1230905	9pcdisk
26.15u 24.60s 66.90r 	 mk NPROC=32 CONF=pcdisk
total: 65880	in kernel text: 38460	outside kernel text: 27420
KTZERO 80100000
ms	  %	sym
7188	 18.6	runproc
6972	 18.1	i8259isr
4668	 12.1	memmove
4236	 11.0	memset
1848	  4.8	lock
1440	  3.7	envgen




jmk@plan9.bell-labs.com wrote:
> 
> Almost any speed comparisons will show Plan 9 as coming in 2nd,
> except for the time it takes to compile the kernel.

Re: [9fans] Performance

[nemo]

[-- Attachment #1: Type: text/plain, Size: 148 bytes --]

Even so. Take into account that I was talking about the
total memory usage (system+apps). Linux apps tend to be 
of the memory consuming kind.


[-- Attachment #2: Type: message/rfc822, Size: 1821 bytes --]

From: Scott Schwartz <schwartz@bio.cse.psu.edu>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Performance
Date: Tue, 08 May 2001 14:46:39 -0400
Message-ID: <20010508184639.17683.qmail@f.bio.cse.psu.edu>

| that the memory usage for me is 1/10th or so w/ Plan9 wrt Linux
| (I include X and gnome here).

Even counting wired down memory for /dev/draw in the kernel?

Re: [9fans] Performance

[Franklin Robert Araujo Fran€a]

In fact, I had studied about Plan 9 performance against Linux and Sun OS. I
have a paper with all the results. That paper is one of the results of my
Master Thesis. The advisor for this Thesis was Professor Ph.D Celio
Guimaraes. I'll send the results for this list as soon as possible.

Thanks.

Franklin.


presotto@plan9.bell-labs.com wrote:

> Not that i know of
>
>   ------------------------------------------------------------------------
>
> Subject: [9fans] Performance
> Date: Tue, 08 May 2001 10:42:15 -0700
> From: "Fariborz 'Skip' Tavakkolian" <fst@real.com>
> Reply-To: 9fans@cse.psu.edu
> To: 9fans@cse.psu.edu
>
> Are there any published performance numbers more recent than those
> mentioned in 9 and net papers? Any for typical PC platforms compared
> to Linux or FreeBSD?

Re: [9fans] Performance

[Ronald G Minnich]

On Tue, 8 May 2001, Russ Cox wrote:

> On the other hand, lmbench might be more suited to
> Plan 9 than to Unix.  At least on Plan 9 you can
> be more certain that you're measuring something real,
> as opposed to code that gcc has optimized into the
> empty instruction sequence.

Don't be too hard on it. Some tests are useful. I have always had good
luck with the memory stuff, to the point that a good digital scope and
lmbench agree on memory cycle time to within a few ns +-. Handy if you
don't want to rip the machine open and measure it.

ron

Re: [9fans] Performance

[Boyd Roberts]

performance?

think about all those old VAXen, blits, whatever ...

Re: [9fans] Performance

[Russ Cox]

> it's just not worth the trouble. last time i looked it was so unix-centric
> the results would be meaningless. how many line of code does it take to
> determine the clock-rate of the cpu, doesn't the firmware print it out when
> the machine boots?

On the other hand, lmbench might be more suited to 
Plan 9 than to Unix.  At least on Plan 9 you can
be more certain that you're measuring something real,
as opposed to code that gcc has optimized into the
empty instruction sequence.

</lmbench flame>

Russ

Re: [9fans] Performance

[jmk]

On Tue May  8 14:46:27 EDT 2001, schwartz@bio.cse.psu.edu wrote:
> lmbench would be interesting to run once in a while.  The overheads
> introduced by APE might be problematic, though.

it's just not worth the trouble. last time i looked it was so unix-centric
the results would be meaningless. how many line of code does it take to
determine the clock-rate of the cpu, doesn't the firmware print it out when
the machine boots?

Re: [9fans] Performance

[Scott Schwartz]

| that the memory usage for me is 1/10th or so w/ Plan9 wrt Linux
| (I include X and gnome here).

Even counting wired down memory for /dev/draw in the kernel?

Re: [9fans] Performance

[Scott Schwartz]

lmbench would be interesting to run once in a while.  The overheads
introduced by APE might be problematic, though.

Re: [9fans] Performance

[jmk]

Almost any speed comparisons will show Plan 9 as coming in 2nd,
except for the time it takes to compile the kernel.

Re: [9fans] Performance

[aam396]

i have gathered a bit more data since i posted my PovRay comparison between plan9 and linux (also available from  
http://homepage.usask.ca/aam396/P9BL.html)... not exact numbers, but the breakdown is pretty much like:

network -- linux much faster
io -- linux somewhat faster, much faster when io involves screen/graphics
raw cpu instruction execution -- linux and p9 are comparable, with linux being slightly ahead...

now, the following i am reluctant to post, since i don't think it is very good (at least for the eyes of most of the 
people reading this list), but nonetheless, here is one of my milestones for the project discussed above:

http://homepage.usask.ca/~aam396/milestone.txt

this one is not really p9 vs Linux, but i hope it's useful

andrey



Quoting Fariborz 'Skip' Tavakkolian <fst@real.com>:

> Are there any published performance numbers more recent than those
> mentioned in 9 and net papers? Any for typical PC platforms compared
> to Linux or FreeBSD?
> 
> 

Re: [9fans] Performance

[nemo]

I've been a daily linux user until  3rd edition came out
(when I became a daily plan9 user). I have noticed
that the memory usage for me is 1/10th or so w/ Plan9 wrt Linux
(I include X and gnome here).

Regarding speed, the system `looks' (again for me as a user,
no measures taken) more or less like linux but for a couple
of things I noticed:

 - network transfers are slower in my case using Plan 9.
   (but fast enough).
 - mail processing is a lot slower in my case using Plan 9,
   perhaps due to the combination of acme Mail, upas/fs and the
   network; But again, for me it's fast enough. 

[9fans] Performance

[Fariborz 'Skip' Tavakkolian]

Are there any published performance numbers more recent than those
mentioned in 9 and net papers? Any for typical PC platforms compared
to Linux or FreeBSD?

Re: [9fans] Performance

[presotto]

[-- Attachment #1: Type: text/plain, Size: 20 bytes --]

Not that i know of

[-- Attachment #2: Type: message/rfc822, Size: 1522 bytes --]

From: "Fariborz 'Skip' Tavakkolian" <fst@real.com>
To: 9fans@cse.psu.edu
Subject: [9fans] Performance
Date: Tue, 08 May 2001 10:42:15 -0700
Message-ID: <3.0.5.32.20010508104215.01407b50@mail.real.com>

Are there any published performance numbers more recent than those
mentioned in 9 and net papers? Any for typical PC platforms compared
to Linux or FreeBSD?