Re: [9fans] Plan 9 and multicores/parallelism/concurrency?

[Bakul Shah <bakul+plan9@bitblocks.com>]

On Tue, 15 Jul 2008 10:50:46 PDT Paul Lalonde <plalonde@telus.net>  wrote:
>
> On 15-Jul-08, at 1:01 AM, Bakul Shah wrote:
> >
> > I suspect a lot of this complexity will end up being dropped
> > when you don't have to worry about efficiently using the last
> > N% of cpu cycles.
>
> Would that I weren't working on a multi-core graphics part...  That N%
> is what the game is all about.

I was really wondering about what might happen when there are
100s of cores per die. My reasoning was that more and more
cores can be (and will be) put on a die but a corresponding
increase in off chip memory bandwidth will not be possible so
at some point memory bottleneck will prevent 100% use of
cores even if you assume ideal placement of threads and no
thread movement to a different core.

> > When your bottleneck is memory bandwidth
> > using core 100% is not going to happen in general.
>
> But in most cases, that memory movement has to share the bus with
> increasingly remote cache accesses, which in turn take bandwidth.
> Affinity is a serious win for reducing on-chip bandwidth usage in
> cache-coherent many-core systems.

I was certainly not suggesting moving threads around.  I was
speculating that as the number of cores goes up perhaps the
kernel is not the right place to do affinity scheduling or
much any sophisticated scheduling.

> >  And I am
> > not sure thread placement belongs in the kernel.  Why not let
> > an application manage its allocation of h/w thread x cycle
> > resources?  I am not even sure a full kernel belongs on every
> > core.
>
> I'm still looking for the right scheduler, in kernel or user space,
> that lets me deal with affinitizing 3 resources that run at different
> granularities: per-core cache, hardware-thread-to-core, and cross-chip
> caches.  There's a rough hierarchy implied by these three resources,
> and perfect scheduling might be possible in a purely cooperative
> world, but reality imposes pre-emption and resource virtualization.

Some friends of mine are able to sqeeze a lot of parallelism
out supposedly hard to parallelize code.  But this is in a
purely cooperative worlds where they assume threads don't
move and where machines are dedicated to specific tasks.

> > Unlike you I think the kernel should do even less as more and
> > more cores are added.  It should basically stay out of the
> > way.  Less government, more privatization :-)  So may be
> > the plan9 kernel would a better starting point than a Unix
> > kernel.
>
> Agreed, less and less in the kernel, but *enough*.  I like resource
> virtualization, and as long as it gets affinity right, I win.