Re: [9fans] Plan 9 and multicores/parallelism/concurrency?

["Roman V. Shaposhnik" <rvs@sun.com>]

On Mon, 2008-07-14 at 12:35 -0400, erik quanstrom wrote:
> > >  Plan 9 makes it easy via 9p, its file system/resource sharing
> > > protocol. In plan 9, things like graphics and network drivers export a
> > > 9p interface (a filetree). Furthermore, 9p is network transparent
> > > which means accesses to remote resources look exactly like accesses to
> > > local resources, and this is the main trick - processes do not care
> > > whether the file they are interested in is being served by the kernel,
> > > a userspace process, or a machine half way across the world.
> >
> > All very true. And it sure does provide enormous benefits on distributed
> > memory architectures. But do you know of any part that would be
> > beneficial for highly-SMP systems?
>
> do you have some reason to believe that 9p (or just read and write)
> is not effective on such a machine?

I have some (not a whole lot, since I haven't looked at source code
for a while) reason to believe that the current 9P implementation
doesn't seem to exploit the opportunity when both ends happen to run
on the same shared memory. I would love to be proved wrong. Although,
the higher level issue that I have with 9P on a shared memory
architectures is the fact that file and communication abstractions
might not be the best way to represent the shared memory resources
to begin with. IOW, mmap()-like things might be a closer match.

> since scheduling would be the main shared resource, do you think
> it would be the limiting factor?

Yes. And that's where the comment in my first email came from:
scheduling is a tricky thing on a shared memory NUMA-like systems.
Solaris's scheduler is not shy when it comes to big iron (100+ CPU SMP
boxes) but even it had to be heavily tuned when a Batoka box first
came to the labs. When you have physcical threads (CPUs), virtual
threads and a non trivial memory hierarchy -- the decision of what
is the best place (hardware-wise) for a give thread to run becomes
a non-trivial one. Kernels that can track affinity properly rule
the day. I don't think that Plan9 scheduler has had an
opportunity to be tuned for such an environment. Same goes for
virtual memory page related algorithms.

Here's a decent (albeit brief) overview of what kernel has to
face these days in order to be reasonably savvy on shared memory,
multicore architectures with NUMA-like memory hierarchy:
   http://www.redhat.com/promo/summit/2008/downloads/pdf/Wednesday_1015am_Rik_Van_Riel_Hot_Topics.pdf
Start from slide #13.

Thanks,
Roman.