From mboxrd@z Thu Jan  1 00:00:00 1970
To: 9fans@cse.psu.edu
Subject: Re: [9fans] Concurrency and message passing with Newsqueak
From: "Russ Cox" <rsc@swtch.com>
Date: Sat, 19 May 2007 12:12:41 -0400
In-Reply-To: <20070519063834.GC96831@kris.home>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Message-Id: <20070519161104.7C8161E8C1F@holo.morphisms.net>
Topicbox-Message-UUID: 6c1421c4-ead2-11e9-9d60-3106f5b1d025

> If we're to compare channels to any 
> language feature, though, why not closures?

And why not for loops?

Prime sieves and power series are undeniably clean
message-passing programs.  They are good examples
of *how* channels can be used, but not *why*
channels are useful in systems programs.

Channels provide a mechanism for implementing the
interface between two different pieces of code and
also a way to synchronize the two.  Both are very
important reasons why people use channels for
systems programming.  Closures alone don't provide
either.

Closures are useful too, just for different things.

Acme uses a trick where closures get sent from one
proc to another along a channel in order to start
the closure running as a new thread in the target
proc.  The acme paper, p. 12 (Concurrency in the
implementation) discusses this.

When using libthread (or Alef), one reason you
care which proc each thread (task) runs in is that
when a proc blocks on a system call (not a channel
op), all the other threads in that proc block too.
So you might have one proc whose only job is to do
blocking I/O, using a channel to get I/O requests
from threads in other procs -- see ioproc(2) for a
library that wraps this up (ioproc(3) on p9p).

I reimplemented venti's buildindex a couple years
ago to run a lot faster.  In the new buildindex,
the first step is to read each data block on the
arena disks, make an index entry for that data
block, and write it to the correct index disk, in
any order.  (Future passes sort the individual
index disks into the right format, in parallel.
This step is just "spraying" each index entry onto
the right disk, like a multiway quicksort pivot.)
Buildindex starts a proc reading each arena and a
proc writing each index disk.  The arena procs
just read a data block at a time and write an
index entry to the channel for the appropriate
disk.  Each index proc reads a buffer worth of
index entries from its disk's channel, writes the
buffer to disk, and repeats.  It just worked the
first time, and channels made the I/O trivial.

Russ