From mboxrd@z Thu Jan  1 00:00:00 1970
Message-ID: <89a44511b98911d8e26633895373dab7@hamnavoe.com>
To: blstuart@bellsouth.net, 9fans@9fans.net
From: Richard Miller <9fans@hamnavoe.com>
Date: Tue,  3 Apr 2018 22:05:54 +0100
In-Reply-To: <1092190499.1253166.1522781548163@mail.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Subject: Re: [9fans] Plan9 on Pi 3B+
Topicbox-Message-UUID: d379324e-ead9-11e9-9d60-3106f5b1d025

> But what is the
> rationale for the lock in the second loop?

Sorry for the confusing code.  The startlocks are locked by
cpu0 before booting the secondary cores.  Each core unlocks
its own startlock when it starts.  Cpu0 blocks on each startlock
waiting for the corresponding core to unlock it, as a way of
serialising the booting of the secondary cores (and detecting
that they have started).

If for some reason the secondary cpus don't start up, the
result is a lock loop.  I really should have used canlock()
instead, to expose a more helpful error message.  But it's
a "should never happen" condition.

It's happening with the new start_cd.elf because the Pi
Foundation made a little change in the firmware which makes
secondary cores wait with a WFE when booting, so the OS
now has to do a SEV to wake them up.

I'm just about to release an update to the bcm port for
the 3B+, including drivers for the new "gigabit" ethernet
(not really, because it's squeezed through a slow usb2 adapter)
and the updated wifi chip.  But in the meantime the following
should let you boot any pi2 or pi3 with the latest start_cd.elf:

--- /sys/src/9/bcm/archbcm2.c	Fri Mar 16 17:08:21 2018
***************
*** 167,172 ****
--- 167,174 ----
  	if(mb->clr[cpu].startcpu)
  		return -1;
  	mb->set[cpu].startcpu = PADDR(cpureset);
+ 	coherence();
+ 	sev();
  	for(i = 0; i < 1000; i++)
  		if(mb->clr[cpu].startcpu == 0)
  			return 0;
***************
*** 196,202 ****
  int
  startcpus(uint ncpu)
  {
! 	int i;

  	for(i = 0; i < ncpu; i++)
  		lock(&startlock[i]);
--- 198,204 ----
  int
  startcpus(uint ncpu)
  {
! 	int i, timeout;

  	for(i = 0; i < ncpu; i++)
  		lock(&startlock[i]);
***************
*** 204,210 ****
  	for(i = 1; i < ncpu; i++){
  		if(startcpu(i) < 0)
  			return i;
! 		lock(&startlock[i]);
  		unlock(&startlock[i]);
  	}
  	return ncpu;
--- 206,215 ----
  	for(i = 1; i < ncpu; i++){
  		if(startcpu(i) < 0)
  			return i;
! 		timeout = 10000000;
! 		while(!canlock(&startlock[i]))
! 			if(--timeout == 0)
! 				return i;
  		unlock(&startlock[i]);
  	}
  	return ncpu;

--- /sys/src/9/bcm/armv7.s	Fri Mar 16 16:47:18 2018
***************
*** 17,22 ****
--- 17,23 ----
  #define WFI	WORD	$0xe320f003	/* wait for interrupt */
  #define WFI_EQ	WORD	$0x0320f003	/* wait for interrupt if eq */
  #define ERET	WORD	$0xe160006e	/* exception return from HYP */
+ #define SEV	WORD	$0xe320f004	/* send event */

  /* tas/cas strex debugging limits; started at 10000 */
  #define MAXSC 1000000
***************
*** 386,391 ****
--- 387,396 ----

  TEXT coherence(SB), $-4
  	BARRIERS
+ 	RET
+
+ TEXT sev(SB), $-4
+ 	SEV
  	RET

  /*