9front and raspberry pi 4 8gb model

9front and raspberry pi 4 8gb model

[JimErickson]

i have a raspberry pi 4 4gb model and a raspberry pi 4 8gb model. 9front 
runs just fine on the 4gb model but on the 8gb model the keyboard and 
mouse are not detected. both models have the same firmware and eeprom, 
the latest release dated april 23, 2020. am i doing something wrong or 
has something changed on the usb bus on the 8gb model?

Re: [9front] 9front and raspberry pi 4 8gb model

[Eli Cohen]

I was lurking in the channel and saw cinap complaining a later rpi
requires a firmware upload for the xhci to work. He may have meant
your model rpi4. We have to be patient or code it ourselves not as
well as he would, he is only one man

On Tue, Jun 2, 2020 at 4:12 PM JimErickson <jimerickso@gmail.com> wrote:
>
> i have a raspberry pi 4 4gb model and a raspberry pi 4 8gb model. 9front
> runs just fine on the 4gb model but on the 8gb model the keyboard and
> mouse are not detected. both models have the same firmware and eeprom,
> the latest release dated april 23, 2020. am i doing something wrong or
> has something changed on the usb bus on the 8gb model?
>


-- 
http://echoline.org

Re: [9front] 9front and raspberry pi 4 8gb model

[hiro]

yes, the rpi people left away some flash and now the software has to
provide a certain additional firmware. somebody (i didn't recognize
his name) fixed it already according to the rpi github pages, but i
don't know if they provided any patches for the rest of us :(
cinap doesn't have an 8gb model, so it would be a waste of time for
him to remote-guess all these little undocumented changes the rpi
foundation is making.

On 6/2/20, Eli Cohen <echoline@gmail.com> wrote:
> I was lurking in the channel and saw cinap complaining a later rpi
> requires a firmware upload for the xhci to work. He may have meant
> your model rpi4. We have to be patient or code it ourselves not as
> well as he would, he is only one man
>
> On Tue, Jun 2, 2020 at 4:12 PM JimErickson <jimerickso@gmail.com> wrote:
>>
>> i have a raspberry pi 4 4gb model and a raspberry pi 4 8gb model. 9front
>> runs just fine on the 4gb model but on the 8gb model the keyboard and
>> mouse are not detected. both models have the same firmware and eeprom,
>> the latest release dated april 23, 2020. am i doing something wrong or
>> has something changed on the usb bus on the 8gb model?
>>
>
>
> --
> http://echoline.org
>

Re: [9front] 9front and raspberry pi 4 8gb model

[JimErickson]

would it help to supply cinap with one of my extra 8gb models? though i 
realize he is very busy. maybe cinap will see this and reply.

On 6/3/20 2:34 AM, hiro wrote:
> yes, the rpi people left away some flash and now the software has to
> provide a certain additional firmware. somebody (i didn't recognize
> his name) fixed it already according to the rpi github pages, but i
> don't know if they provided any patches for the rest of us :(
> cinap doesn't have an 8gb model, so it would be a waste of time for
> him to remote-guess all these little undocumented changes the rpi
> foundation is making.
>
> On 6/2/20, Eli Cohen <echoline@gmail.com> wrote:
>> I was lurking in the channel and saw cinap complaining a later rpi
>> requires a firmware upload for the xhci to work. He may have meant
>> your model rpi4. We have to be patient or code it ourselves not as
>> well as he would, he is only one man
>>
>> On Tue, Jun 2, 2020 at 4:12 PM JimErickson <jimerickso@gmail.com> wrote:
>>> i have a raspberry pi 4 4gb model and a raspberry pi 4 8gb model. 9front
>>> runs just fine on the 4gb model but on the 8gb model the keyboard and
>>> mouse are not detected. both models have the same firmware and eeprom,
>>> the latest release dated april 23, 2020. am i doing something wrong or
>>> has something changed on the usb bus on the 8gb model?
>>>
>>
>> --
>> http://echoline.org
>>

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

the firmware is loaded by the gpu. it is just that you need to issue a
firmware command to the gpu firmware after mapping the pcie devices.

richard miller sent me a patch, his toubleshooting is documented here:

https://github.com/raspberrypi/firmware/issues/1402

the adapted and untested patch for 9front is below.

if you have a pi4 with 8GB, it would be great if you can try it.

just apply the patch below and build a kernel in /sys/src/9/bcm64
with: 'mk CONF=pi4'. or you can try out the kernels i'v just build:

http://felloff.net/usr/cinap_lenrek/9pi4 (raw kernel)
http://felloff.net/usr/cinap_lenrek/s9pi4 (a.out with debug symbols)

the resulting 9pi4 raw kernel can be put into the sdcard image
from the latest release.

diff -r ff6b294d07a9 sys/src/9/bcm/vcore.c
--- a/sys/src/9/bcm/vcore.c	Mon Jun 01 00:07:01 2020 +0200
+++ b/sys/src/9/bcm/vcore.c	Wed Jun 03 19:17:09 2020 +0200
@@ -53,6 +53,7 @@
 	TagSetEgpioConf	= 0x00038043,
 
 	TagGettemp	= 0x00030006,
+	TagXhciReset	= 0x00030058,
 	TagFballoc	= 0x00040001,
 	TagFbfree	= 0x00048001,
 	TagFbblank	= 0x00040002,
@@ -413,3 +414,19 @@
 	buf[1] = on;
 	vcreq(TagSetEgpioState, buf, sizeof(buf), sizeof(buf));
 }
+
+/*
+ * Notify gpu that xhci firmware might need loading. This is for some
+ * pi4 board versions which are missing the eeprom chip for the vl805,
+ * requiring its firmware to come from the boot eeprom instead.
+ */
+int
+xhcireset(int devaddr)
+{
+	u32int buf[1];
+
+	buf[0] = devaddr;
+	if(vcreq(TagXhciReset, buf, sizeof(buf), sizeof(buf[0])) == sizeof(buf[0]))
+		return buf[0];
+	return -1;
+}
diff -r ff6b294d07a9 sys/src/9/bcm64/archbcm4.c
--- a/sys/src/9/bcm64/archbcm4.c	Mon Jun 01 00:07:01 2020 +0200
+++ b/sys/src/9/bcm64/archbcm4.c	Wed Jun 03 19:55:12 2020 +0200
@@ -172,5 +172,20 @@
 void
 archbcm4link(void)
 {
+	Pcidev *p;
+
+	/*
+	 * The firmware resets PCI before starting the host OS because
+	 * without SDRAM the VL805 makes inbound requests to page-in firmware
+	 * from SDRAM. If the OS has a different PCI mapping that would all break.
+	 * There's no way to pause and move the mappings and it's not really desirable
+	 * for the firmware to dictate the PCI configuration. Consequently, the mailbox
+	 * is required so that the OS can reset the VLI after asserting PCI chip reset.
+	 */
+	if((p = pcimatch(nil, 0x1106, 0x3483)) != nil){
+		iprint("VL805 reset...\n");
+		xhcireset(BUSBNO(p->tbdf)<<20 | BUSDNO(p->tbdf)<<15 | BUSFNO(p->tbdf)<<12);
+	}
+
 	// addclock0link(wdogfeed, HZ);
 }
diff -r ff6b294d07a9 sys/src/9/bcm64/fns.h
--- a/sys/src/9/bcm64/fns.h	Mon Jun 01 00:07:01 2020 +0200
+++ b/sys/src/9/bcm64/fns.h	Wed Jun 03 19:17:09 2020 +0200
@@ -170,6 +170,7 @@
 extern void vgpinit(void);
 extern void vgpset(uint port, int on);
 extern void egpset(uint port, int on);
+extern int xhcireset(int devaddr);
 
 /* bootargs */
 extern void bootargsinit(uintptr);
diff -r ff6b294d07a9 sys/src/9/bcm64/pi4
--- a/sys/src/9/bcm64/pi4	Mon Jun 01 00:07:01 2020 +0200
+++ b/sys/src/9/bcm64/pi4	Wed Jun 03 19:17:09 2020 +0200
@@ -27,11 +27,11 @@
 link
 	gisb
 	pci
-	archbcm4
+	archbcm4	pci
+	usbxhci		pci archbcm4
 	ethergenet	ethermii
 	ethermedium
 	loopbackmedium
-	usbxhci		pci
 
 ip
 	tcp

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[hiro]

oh. that "9pi" guy is rmiller? that explains it :)

Re: [9front] 9front and raspberry pi 4 8gb model

[JimErickson]

sorry cinap but the pre-compiled kernels did not work with the latest 
release and neither did the patch and resulting kernel. the end results 
are the same, mainly that the keyboard and mouse are not detected. i 
know that you are very busy, but is there anything else i can do to help 
out?

On 6/3/20 12:58 PM, cinap_lenrek@felloff.net wrote:
> the firmware is loaded by the gpu. it is just that you need to issue a
> firmware command to the gpu firmware after mapping the pcie devices.
>
> richard miller sent me a patch, his toubleshooting is documented here:
>
> https://github.com/raspberrypi/firmware/issues/1402
>
> the adapted and untested patch for 9front is below.
>
> if you have a pi4 with 8GB, it would be great if you can try it.
>
> just apply the patch below and build a kernel in /sys/src/9/bcm64
> with: 'mk CONF=pi4'. or you can try out the kernels i'v just build:
>
> http://felloff.net/usr/cinap_lenrek/9pi4 (raw kernel)
> http://felloff.net/usr/cinap_lenrek/s9pi4 (a.out with debug symbols)
>
> the resulting 9pi4 raw kernel can be put into the sdcard image
> from the latest release.
>
> diff -r ff6b294d07a9 sys/src/9/bcm/vcore.c
> --- a/sys/src/9/bcm/vcore.c	Mon Jun 01 00:07:01 2020 +0200
> +++ b/sys/src/9/bcm/vcore.c	Wed Jun 03 19:17:09 2020 +0200
> @@ -53,6 +53,7 @@
>   	TagSetEgpioConf	= 0x00038043,
>   
>   	TagGettemp	= 0x00030006,
> +	TagXhciReset	= 0x00030058,
>   	TagFballoc	= 0x00040001,
>   	TagFbfree	= 0x00048001,
>   	TagFbblank	= 0x00040002,
> @@ -413,3 +414,19 @@
>   	buf[1] = on;
>   	vcreq(TagSetEgpioState, buf, sizeof(buf), sizeof(buf));
>   }
> +
> +/*
> + * Notify gpu that xhci firmware might need loading. This is for some
> + * pi4 board versions which are missing the eeprom chip for the vl805,
> + * requiring its firmware to come from the boot eeprom instead.
> + */
> +int
> +xhcireset(int devaddr)
> +{
> +	u32int buf[1];
> +
> +	buf[0] = devaddr;
> +	if(vcreq(TagXhciReset, buf, sizeof(buf), sizeof(buf[0])) == sizeof(buf[0]))
> +		return buf[0];
> +	return -1;
> +}
> diff -r ff6b294d07a9 sys/src/9/bcm64/archbcm4.c
> --- a/sys/src/9/bcm64/archbcm4.c	Mon Jun 01 00:07:01 2020 +0200
> +++ b/sys/src/9/bcm64/archbcm4.c	Wed Jun 03 19:55:12 2020 +0200
> @@ -172,5 +172,20 @@
>   void
>   archbcm4link(void)
>   {
> +	Pcidev *p;
> +
> +	/*
> +	 * The firmware resets PCI before starting the host OS because
> +	 * without SDRAM the VL805 makes inbound requests to page-in firmware
> +	 * from SDRAM. If the OS has a different PCI mapping that would all break.
> +	 * There's no way to pause and move the mappings and it's not really desirable
> +	 * for the firmware to dictate the PCI configuration. Consequently, the mailbox
> +	 * is required so that the OS can reset the VLI after asserting PCI chip reset.
> +	 */
> +	if((p = pcimatch(nil, 0x1106, 0x3483)) != nil){
> +		iprint("VL805 reset...\n");
> +		xhcireset(BUSBNO(p->tbdf)<<20 | BUSDNO(p->tbdf)<<15 | BUSFNO(p->tbdf)<<12);
> +	}
> +
>   	// addclock0link(wdogfeed, HZ);
>   }
> diff -r ff6b294d07a9 sys/src/9/bcm64/fns.h
> --- a/sys/src/9/bcm64/fns.h	Mon Jun 01 00:07:01 2020 +0200
> +++ b/sys/src/9/bcm64/fns.h	Wed Jun 03 19:17:09 2020 +0200
> @@ -170,6 +170,7 @@
>   extern void vgpinit(void);
>   extern void vgpset(uint port, int on);
>   extern void egpset(uint port, int on);
> +extern int xhcireset(int devaddr);
>   
>   /* bootargs */
>   extern void bootargsinit(uintptr);
> diff -r ff6b294d07a9 sys/src/9/bcm64/pi4
> --- a/sys/src/9/bcm64/pi4	Mon Jun 01 00:07:01 2020 +0200
> +++ b/sys/src/9/bcm64/pi4	Wed Jun 03 19:17:09 2020 +0200
> @@ -27,11 +27,11 @@
>   link
>   	gisb
>   	pci
> -	archbcm4
> +	archbcm4	pci
> +	usbxhci		pci archbcm4
>   	ethergenet	ethermii
>   	ethermedium
>   	loopbackmedium
> -	usbxhci		pci
>   
>   ip
>   	tcp
>
> --
> cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[ori]

> sorry cinap but the pre-compiled kernels did not work with the latest 
> release and neither did the patch and resulting kernel. the end results 
> are the same, mainly that the keyboard and mouse are not detected. i 
> know that you are very busy, but is there anything else i can do to help 
> out?

Just as a sanity check: Are you seeing the message the patch
adds?

> +		iprint("VL805 reset...\n");

Can you add a print in the xhci reset function, and make sure
that it's acutally getting called?

> +xhcireset(int devaddr)

Re: [9front] 9front and raspberry pi 4 8gb model

[JimErickson]

ori yes i am seeing the "VL805 reset..." message so i think it is 
getting called.

On 6/3/20 8:57 PM, ori@eigenstate.org wrote:
>> sorry cinap but the pre-compiled kernels did not work with the latest
>> release and neither did the patch and resulting kernel. the end results
>> are the same, mainly that the keyboard and mouse are not detected. i
>> know that you are very busy, but is there anything else i can do to help
>> out?
> Just as a sanity check: Are you seeing the message the patch
> adds?
>
>> +		iprint("VL805 reset...\n");
> Can you add a print in the xhci reset function, and make sure
> that it's acutally getting called?
>
>> +xhcireset(int devaddr)

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

i do know as much as you about this issue.

maybe you also need to update bootcode.bin in the image?
or it needs another device tree file? or some new file we dont
know about?

the next steps would be to print the return value of xhcireset()...
maybe that can give us a hint if the command was supported by the
firmware or not.

you can also ask richard miller about this... maybe download his
image and see if this works. and see what firmware and device tree
files he has there.

we could also to take a look at raspbian image. and in see what
they do in the linux kernel.

the diff i send is based on richard millers patch.
i only tested that it doesnt break my raspberry pi 4 with 1gb ram.
i had to move the xhcireset() call to archinit() tho which is
basically just after pci controller reset and after we mapped
the bus and programmed the membars.

originally, the xhcireset() call was after firmware handoff in
the usbxhci driver itself, but i asked richard if he could try
to do the reset earlier because our xhci driver is in port/,
and this is very raspberry pi specific quirk that doesnt belong
into the portable driver.

richard confirmed that it still works when doing the firmware
call before the pcienable(), but maybe theres some other side
effect. could even be just timing.

good luck.

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

> would it help to supply cinap with one of my extra 8gb models? though i 
> realize he is very busy. maybe cinap will see this and reply.

sure, i'll take it. tho it might be easier and cheaper for me to buy one
myself and have it shipped to spain.

it will take some time.

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[JimErickson]

well i got the may 20,2020 image of plan9 from 
fromhttps://9p.io/sources/contrib/miller/9pi.img.gz 
<https://9p.io/sources/contrib/miller/9pi.img.gz>but it does not boot in 
the 8gb model. i suppose i need a newer image. guess i will have to wait 
till he updates it. unless anyone knows how to get the newest image. 
thanks for all you responses. how would i get in touch with richard miller?

On 6/4/20 12:21 PM, cinap_lenrek@felloff.net wrote:
> i do know as much as you about this issue.
>
> maybe you also need to update bootcode.bin in the image?
> or it needs another device tree file? or some new file we dont
> know about?
>
> the next steps would be to print the return value of xhcireset()...
> maybe that can give us a hint if the command was supported by the
> firmware or not.
>
> you can also ask richard miller about this... maybe download his
> image and see if this works. and see what firmware and device tree
> files he has there.
>
> we could also to take a look at raspbian image. and in see what
> they do in the linux kernel.
>
> the diff i send is based on richard millers patch.
> i only tested that it doesnt break my raspberry pi 4 with 1gb ram.
> i had to move the xhcireset() call to archinit() tho which is
> basically just after pci controller reset and after we mapped
> the bus and programmed the membars.
>
> originally, the xhcireset() call was after firmware handoff in
> the usbxhci driver itself, but i asked richard if he could try
> to do the reset earlier because our xhci driver is in port/,
> and this is very raspberry pi specific quirk that doesnt belong
> into the portable driver.
>
> richard confirmed that it still works when doing the firmware
> call before the pcienable(), but maybe theres some other side
> effect. could even be just timing.
>
> good luck.
>
> --
> cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

ok, i ordered a pi4 8GB model now. it will probably arrive
at the end of next week.

we'll get to the bottom of this, don't worry :)

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[Roman Shaposhnik]

On Thu, Jun 11, 2020 at 10:46 AM <cinap_lenrek@felloff.net> wrote:
>
> ok, i ordered a pi4 8GB model now. it will probably arrive
> at the end of next week.
>
> we'll get to the bottom of this, don't worry :)

Btw, the following thread may be helpful:
https://lists.denx.de/pipermail/u-boot/2020-April/405216.html

Thanks,
Roman.

P.S. Apparently 8Gb model is actually somewhat different -- I've just
gotten mine and starting to poke

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

> Btw, the following thread may be helpful:
> https://lists.denx.de/pipermail/u-boot/2020-April/405216.html

this is probably not the issue. the xhci works fine with the 1gb and 2gb
variants. and the issue they had was that they missed flushing the
dcache for the scratchpad memory they hand to the xhci controller.

we do flush the dcache for these regions here:

		dmaflush(1, ctlr->sbp, ctlr->nscratch*ctlr->pagesize);
		dmaflush(1, ctlr->sba, ctlr->nscratch*8);

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[Roman Shaposhnik]

On Sat, Jun 13, 2020 at 6:56 PM <cinap_lenrek@felloff.net> wrote:
>
> > Btw, the following thread may be helpful:
> > https://lists.denx.de/pipermail/u-boot/2020-April/405216.html
>
> this is probably not the issue. the xhci works fine with the 1gb and 2gb
> variants. and the issue they had was that they missed flushing the
> dcache for the scratchpad memory they hand to the xhci controller.
>
> we do flush the dcache for these regions here:
>
>                 dmaflush(1, ctlr->sbp, ctlr->nscratch*ctlr->pagesize);
>                 dmaflush(1, ctlr->sba, ctlr->nscratch*8);

Upon closer examination you may be right.

That said -- in general I feel that retracing u-boot steps in
supporting USB boot on RPi4 8Gb may be easier than retracing steps of
the Linux kernel.

Thanks,
Roman.

Re: [9front] 9front and raspberry pi 4 8gb model

[cinap_lenrek]

ok, a update.

i received my raspberry pi 4 8gb variant this monday and today
had some chance trying to reproduce the problem.

the patch for the xhci reset i sent just worked out of the box.

(tested with a usb mouse)

i also disabled the patch to verify that the new xhcireset()
mailbox command is actually required, and indeed, it is.
xhci just hangs otherwise.

it is unclear to me why this did not work for you. maybe you
have another problem with your usb device? can you verify that
it works at all with the 9front usb stack on a pc with a xhci
controller?

everything appears to be working fine now, the commits are:

changeset:   7862:cd897da2cce1
tag:         tip
user:        cinap_lenrek@felloff.net
date:        Thu Jul 02 21:12:40 2020 +0200
summary:     bcm64: handle 8GB of physical memory for raspberry pi4

changeset:   7861:887585744716
user:        cinap_lenrek@felloff.net
date:        Thu Jul 02 21:04:01 2020 +0200
summary:     bcm64: fix usb xhci controller on pi4 8GB variant (thanks richard miller)

--
cinap

Re: [9front] 9front and raspberry pi 4 8gb model

[Roman Shaposhnik]

On Thu, Jul 2, 2020 at 12:27 PM <cinap_lenrek@felloff.net> wrote:
>
> ok, a update.
>
> i received my raspberry pi 4 8gb variant this monday and today
> had some chance trying to reproduce the problem.
>
> the patch for the xhci reset i sent just worked out of the box.
>
> (tested with a usb mouse)
>
> i also disabled the patch to verify that the new xhcireset()
> mailbox command is actually required, and indeed, it is.
> xhci just hangs otherwise.
>
> it is unclear to me why this did not work for you. maybe you
> have another problem with your usb device? can you verify that
> it works at all with the 9front usb stack on a pc with a xhci
> controller?
>
> everything appears to be working fine now, the commits are:
>
> changeset:   7862:cd897da2cce1
> tag:         tip
> user:        cinap_lenrek@felloff.net
> date:        Thu Jul 02 21:12:40 2020 +0200
> summary:     bcm64: handle 8GB of physical memory for raspberry pi4
>
> changeset:   7861:887585744716
> user:        cinap_lenrek@felloff.net
> date:        Thu Jul 02 21:04:01 2020 +0200
> summary:     bcm64: fix usb xhci controller on pi4 8GB variant (thanks richard miller)

I can offer to test on my unit if you can make a kernel binary
available, cinap -- I'm actually also curious what the difference is
here.

Thanks,
Roman.

9front and raspberry pi 4 8gb model

[JimErickson]

thank you cinap!!
i guess i must have applied the patch wrong the first time because a 
fresh build of the kernel after sysupdate and it booted fine on my 8 gb 
raspberry pi 4. thanks so much and sorry you had to go to all this trouble.