From mboxrd@z Thu Jan  1 00:00:00 1970
To: 9fans@9fans.net
Subject: Re: [9fans] channel_lock
From: "Russ Cox" <rsc@swtch.com>
Date: Mon, 14 Jul 2008 16:58:03 -0400
In-Reply-To: <071420081847.25641.487B9F36000AB7F2000064292200761064040196960E040E9F@comcast.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Message-Id: <20080714205638.BD12F1E8C1F@holo.morphisms.net>
Topicbox-Message-UUID: e6c74ada-ead3-11e9-9d60-3106f5b1d025

> I have been looking and the implementation of channels/alt in Plan9,
> and I have a question.  Does lock()/unlock() work across procs (not just
> threads)?  For example, in channel.c there is a static Lock *channel_lock.
> Does this provide exclusive to channel data across procs?

Yes: Locks are shared-memory spin locks.  They don't do any scheduling
(in contrast to QLocks).

> I assume yes, just trying to make sure I'm understanding it correctly.
> I was expecting to see code which handles channel access "across procs"
> differently than channel access between threads "in the same proc",
> but I didn't see anything like that.

This is a good question.  Most code doesn't need to keep that distinction
in mind.  There are simply threads, and they are running or ready
or asleep.

Plan 9's synchronization primitive is called rendezvous.  Processes in
a rendezvous group (usually also processes that share memory) can call

	rendezvous(tag, value)

and once two processes in the same group have called rendezvous with
the same tag, the two calls each return the other's value.

Libthread provides a thread-aware version of rendezvous called
_threadrendezvous.  The libc QLock, which normally uses rendezvous, uses
_threadrendezvous inside libthread programs but is otherwise unchanged.

No matter what the interface, at a low level things always look like this:
a running thread decides for whatever reason that it will stop running.
It records a pointer to its Thread* structure t somewhere, and then it
calls a function to enter back into its proc's scheduler.  (In Plan 9
libthread this function is called _sched.)  That proc has a run queue,
and t is not on it.  So either the proc runs some other thread that has
been waiting to run, or, if the run queue is empty, the proc itself goes
to sleep, using the OS-provided rendezvous.

At some point in the future, some thread, perhaps running in a different
proc, comes along and decides it is time for t to wake up.  It calls
_threadready(t), which puts t on t's proc's run queue (with appropriate
locking).  Then, if t's proc is asleep waiting for something to be placed
on the run queue, _threadready needs to wake that proc:

	q = &t->proc->ready;
	lock(&t->proc->readylock);
	... add t to q ...
	if(q->asleep){
		q->asleep = 0;
		/* lock passes to other proc */
		_threaddebug(DBGSCHED, "waking process %d", t->proc->pid);
		while(rendezvous(q, 0) == (void*)~0){
			if(_threadexitsallstatus)
				exits(_threadexitsallstatus);
		}
	}else
		unlock(&t->proc->readylock);

Here, q->asleep can only be true if t->proc is not the current thread's
proc.

Almost every operation in the thread library operates on the current
thread and the current proc.  Readying a thread is the only fundamental
operation that can interact with foreign procs.

Russ