Re: [9fans] cache lines, and 60000 cycles of doom

[erik quanstrom <quanstro@quanstro.net>]

> It disables out-of-order execution by the processor, so there's a
> pipeline stall.

we know there's going to be a stall already, since we can't get
the cacheline we're looking for.

> There's overhead to calling the tsc instructions, but not that much.
>
> Does `srb->wmach != m->machno` imply that t0 and t1 could be run on
> different CPUs? TSC is synchronized between cores (unless someone does
> wrmsr), but if you bounce to another processor, there's no guarantee.
> Perhaps the difference between when the CPUs came online was on the
> order of 60k cycles. No clue how cheap sched() is these days.

srb->wmach is the wakeup mach, m->machno is us.  and i just realized
that that deadlock prevention is not necessary.  if srb->wmach == m->machno,
then wakeup has cleared the hazard.

i've rewritten the function as so
	static void
	srbfree(Srb *srb)
	{
		while(monmwait(&srb->state, Alloc) == Alloc)
			{}
		mbfree(srb->msgbuf);
	}

by the way, to head off the next speculation, i tried wrapping the
wakeup in splhi(), but that made no difference.  the waker is not being
scheduled.

by the way, this is ken's kernel on amd64: /n/atom/plan9/sys/src/fs
the development version has drifted a bit, but i can give you access
if you're interested.

- erik