[9fans] channel_lock

[9fans] channel_lock

[palazzol]

Hello,

I have been looking and the implementation of channels/alt in Plan9, and I have a question.  Does lock()/unlock() work across procs (not just threads)?  For example, in channel.c there is a static Lock *channel_lock.  Does this provide exclusive to channel data across procs?

I assume yes, just trying to make sure I'm understanding it correctly.  I was expecting to see code which handles channel access "across procs" differently than channel access between threads "in the same proc", but I didn't see anything like that.

Thanks,
Frank

Re: [9fans] channel_lock

[erik quanstrom]

> I have been looking and the implementation of channels/alt in Plan9,
> and I have a question.  Does lock()/unlock() work across procs (not
> just threads)?  For example, in channel.c there is a static Lock
> *channel_lock.  Does this provide exclusive to channel data across
> procs?
>
>  I assume yes, just trying to make sure I'm understanding it
>  correctly.  I was expecting to see code which handles channel access
>  "across procs" differently than channel access between threads "in
>  the same proc", but I didn't see anything like that.

locks are not necessary between threads in the same proc
because only one of them can run at a time and they are
cooperatively scheduled.

the only case that needs to be handled is locking between
procs.  thus there is no special case for locking between
threads regardless of implementation

on the other hand, if you look at the implementation of lock
(/sys/src/libc/port/lock.c), it does not depend on the the
implentation of threads or procs.  it relies on an atomic
tas instruction (or simuation thereof for mips) only.

- erik

Re: [9fans] channel_lock

[Russ Cox]

> I have been looking and the implementation of channels/alt in Plan9,
> and I have a question.  Does lock()/unlock() work across procs (not just
> threads)?  For example, in channel.c there is a static Lock *channel_lock.
> Does this provide exclusive to channel data across procs?

Yes: Locks are shared-memory spin locks.  They don't do any scheduling
(in contrast to QLocks).

> I assume yes, just trying to make sure I'm understanding it correctly.
> I was expecting to see code which handles channel access "across procs"
> differently than channel access between threads "in the same proc",
> but I didn't see anything like that.

This is a good question.  Most code doesn't need to keep that distinction
in mind.  There are simply threads, and they are running or ready
or asleep.

Plan 9's synchronization primitive is called rendezvous.  Processes in
a rendezvous group (usually also processes that share memory) can call

	rendezvous(tag, value)

and once two processes in the same group have called rendezvous with
the same tag, the two calls each return the other's value.

Libthread provides a thread-aware version of rendezvous called
_threadrendezvous.  The libc QLock, which normally uses rendezvous, uses
_threadrendezvous inside libthread programs but is otherwise unchanged.

No matter what the interface, at a low level things always look like this:
a running thread decides for whatever reason that it will stop running.
It records a pointer to its Thread* structure t somewhere, and then it
calls a function to enter back into its proc's scheduler.  (In Plan 9
libthread this function is called _sched.)  That proc has a run queue,
and t is not on it.  So either the proc runs some other thread that has
been waiting to run, or, if the run queue is empty, the proc itself goes
to sleep, using the OS-provided rendezvous.

At some point in the future, some thread, perhaps running in a different
proc, comes along and decides it is time for t to wake up.  It calls
_threadready(t), which puts t on t's proc's run queue (with appropriate
locking).  Then, if t's proc is asleep waiting for something to be placed
on the run queue, _threadready needs to wake that proc:

	q = &t->proc->ready;
	lock(&t->proc->readylock);
	... add t to q ...
	if(q->asleep){
		q->asleep = 0;
		/* lock passes to other proc */
		_threaddebug(DBGSCHED, "waking process %d", t->proc->pid);
		while(rendezvous(q, 0) == (void*)~0){
			if(_threadexitsallstatus)
				exits(_threadexitsallstatus);
		}
	}else
		unlock(&t->proc->readylock);

Here, q->asleep can only be true if t->proc is not the current thread's
proc.

Almost every operation in the thread library operates on the current
thread and the current proc.  Readying a thread is the only fundamental
operation that can interact with foreign procs.

Russ

Re: [9fans] channel_lock

[Russ Cox]

> locks are not necessary between threads in the same proc
> because only one of them can run at a time and they are
> cooperatively scheduled.

It is important to note that using cooperatively scheduled threads
doesn't automatically mean you don't need locking.  If your
threads can go to sleep with some resources in intermediate states
that other threads should not observe, then you need locking.
It's just that instead of using Locks, which do not yield the current
thread, you should use QLocks, which do.

The precise version of Erik's statement is this:

    For resources shared only by threads in a single proc, Locks are either
    unnecessary (because there is no true concurrent access due to the
    cooperative scheduling) or incorrect (because spinning won't
    let the other thread run, so the lock will never be unlocked).

If the Lock would always protect a tiny critical section that
obviously doesn't yield or call any function that might yield
(qlock, alt, send, recv, etc.), then it's unnecessary.  Otherwise,
it's incorrect, and you should be using QLocks.

Russ