Re: [9fans] Raspberry Pi 4

Re: [9fans] Raspberry Pi 4

[Richard Miller]

In contrib/miller on 9p.io there are new kernel binaries 9pi4 and 9pi4cpu,
now with USB3 fully supported (thanks cinap).  If you build from the source
in contrib/miller/9/bcm, you'll need an updated usb/usbd command to initialise
the more complicated root hub topology of the pi4; for convenience the updated
source files are in cmd-usb.tar in that directory.  Eventually I'll supply a
corresponding patch for the 9legacy collection.

I still need to implement kmap, to gain access to the upper 2GiB of a 4GiB pi4.
For the next couple of weeks I have other distractions, but I'll get there ...

Re: [9fans] Raspberry Pi 4

[David du Colombier]

I've generated a patch from your changes in cmd-usb.tar:

http://9legacy.org/9legacy/patch/usb-usbd-usb3.diff

-- 
David du Colombier

Re: [9fans] Raspberry Pi 4

[Richard Miller]

The pi4 kernel on 9p.io is now more complete, with support for
SD card (which was pleasantly standards-conforming), and ethernet
(thanks to cinap).  USB3 still missing, so useful only as a cpu
server.

On a pi4 with 4GB RAM, only 2GB will be accessible until we have
a proper implementation of kmap().

For convenience there's a bootable 9pi4cpu binary in /contrib/miller.

Needs firmware from 5 July 2019 or later.
config.txt for pi4 should 'core_freq=250' and 'device_tree='

Re: [9fans] Raspberry Pi 4

[cinap_lenrek]

arrg! and i forgot the clean BEFORE the issuing dma.

--
cinap

Re: [9fans] Raspberry Pi 4

[cinap_lenrek]

> Your dmaflush code doesn't look to me like it will do the right
> thing if the dma buffer isn't cache aligned.  (Could this ever be
> the case in the 9front kernel?) Suppose dma starts, cpu prefetches
> some old data into the cache line overlapping the start of the dma
> buffer, then dma completes.

> Your dmaflush then calls cachedwbse which writes back the pre-fetched
> data, overwriting the new data which dma wrote to memory.

what? where? it should call dmaflush with the clean parameter set
to zero after the hardware has completed the dma:

dma.c:228: 		dmaflush(0, ctlr->flush, ctlr->len);

dmaflush() then calls cachedwbinvse() on the partial cache lines
which should just invalidate IFF the lines are clean. the write back should
only happen when the lines are dirty. in that case the damage has already
been done and the wb at least makes sure we wont loose the dirty data
from the cache.

so the underlying assumption here is that a cache clean operation
writes back only when the line is dirty. correct me if i'm wrong
here.

so yes, it will not work correctly for an invalidate when the buffer
is not cache line size aligned and the partial start or end gets
modified while the dma operation is in flight.

--
cinap

Re: [9fans] Raspberry Pi 4

[Richard Miller]

> Your dmaflush then calls cachedwbse

Sorry I meant cachdwbinvse !

Re: [9fans] Raspberry Pi 4

[Richard Miller]

> no, i wish. only reading linux source and deciphering device tree
> mazes. :(

Supposedly a h/w manual is being edited now.

> i think theres a bug with the dma code tho. you actually need to
> invalidate caches *AFTER* the hw completed because the
> core can speculatively prefetch memory and bring it into the
> cache before the hardware is done writing memory. iirc the
> code just cleans/invalidates BEFORE issuing the dma operation.

I fixed this a long time ago, but you might have missed it because
I call cachedinvse in the device drivers after return from dmawait,
instead of doing it in the dma code itself.

Your dmaflush code doesn't look to me like it will do the right
thing if the dma buffer isn't cache aligned.  (Could this ever be
the case in the 9front kernel?)  Suppose dma starts, cpu prefetches
some old data into the cache line overlapping the start of the dma
buffer, then dma completes.  Your dmaflush then calls cachedwbse
which writes back the pre-fetched data, overwriting the new data
which dma wrote to memory.

Anyway this doesn't seem relevant for ether4330 because I get the
same trouble using pio instead of dma.

Re: [9fans] Raspberry Pi 4

[cinap_lenrek]

> Thanks, I'll take a look at this.  Have you seen any pi4 h/w docs
> or have you been working just from linux drivers?

no, i wish. only reading linux source and deciphering device tree
mazes. :(

> I've been stalled on getting ether4330 to initialise.  Putting
> some coherence() calls into emmc.c helped (I don't like
> out-of-order cpus) but I keep getting spurious interrupt 1023
> followed by watchdog reboot.

i dont know anything about the 4330.

i think theres a bug with the dma code tho. you actually need to
invalidate caches *AFTER* the hw completed because the
core can speculatively prefetch memory and bring it into the
cache before the hardware is done writing memory. iirc the
code just cleans/invalidates BEFORE issuing the dma operation.

> Do you have the sdcard working?  According to a pi engineer,
> the new emmc controller is sdhci compliant so it shouldn't
> be hard.

i have not tried sdcard yet.

--
cinap

Re: [9fans] Raspberry Pi 4

[Richard Miller]

Thanks, I'll take a look at this.  Have you seen any pi4 h/w docs
or have you been working just from linux drivers?

I've been stalled on getting ether4330 to initialise.  Putting
some coherence() calls into emmc.c helped (I don't like
out-of-order cpus) but I keep getting spurious interrupt 1023
followed by watchdog reboot.

Do you have the sdcard working?  According to a pi engineer,
the new emmc controller is sdhci compliant so it shouldn't
be hard.

Re: [9fans] Raspberry Pi 4

[cinap_lenrek]

i just commited everything i got for the raspberry pi 4.

sorry that this took so long. i was trying to get
the ethernet reliable but this has some bizzare issue
that i was unable to resolve.

http://code.9front.org/hg/plan9front/rev/889b634159e5
http://code.9front.org/hg/plan9front/rev/55d93e47a2de
http://code.9front.org/hg/plan9front/rev/686cdda01118
http://code.9front.org/hg/plan9front/rev/4dbf2522f668
http://code.9front.org/hg/plan9front/rev/db3e8d004b27
http://code.9front.org/hg/plan9front/rev/5afd2605d0f3
http://code.9front.org/hg/plan9front/rev/6e8b95bccec4
http://code.9front.org/hg/plan9front/rev/234f51ace8cd
http://code.9front.org/hg/plan9front/rev/406bef7018a8
http://code.9front.org/hg/plan9front/rev/386fe43162b4
http://code.9front.org/hg/plan9front/rev/f3966b67adc9

note, this is a arm64 kernel, but the drivers should be portable.

- interrupts, uart, multicore, graphics and /dev/reboot works
- pcie and xhci works
- genet works somewhat (good enougth for /dev/reboot netbooting) but
  causes bizzare 42 second bus stalls with bursty bidirectional
  ethernet traffic which hangs all gisb devices (pcie/xhci)

i'll write you the details and what i tried about that ethernet
issue offlist, maybe we can figure this out together.

--
cinap

Re: [9fans] Raspberry Pi 4

[Richard Miller]

>> Do you mean usbxhci or something else?
>
> yeah. as i heard he will commit soon the pcie implementation also for
> rpi4 so it can used there, too.

That will be very useful, thanks.

Re: [9fans] Raspberry Pi 4

[hiro]

yeah. as i heard he will commit soon the pcie implementation also for
rpi4 so it can used there, too.

On 7/19/19, Richard Miller <9fans@hamnavoe.com> wrote:
>> do check cinap's latest 9front commits!
>
> Do you mean usbxhci or something else?
>
>
>

Re: [9fans] Raspberry Pi 4

[Richard Miller]

> do check cinap's latest 9front commits!

Do you mean usbxhci or something else?

Re: [9fans] Raspberry Pi 4

[hiro]

do check cinap's latest 9front commits!

[9fans] Raspberry Pi 4

[Richard Miller]

The 9pi kernel source in /n/sources/contrib/miller/9/bcm on 9p.io has been
updated with (very limited) support for the new pi4.

Reminder: 'srv -nq tcp!9p.io sources /n/sources' to get access to this mirror.
Or point a browser at https://9p.io/sources/contrib/miller/9/bcm

I'm not suggesting anyone should rush out to buy a pi4 just yet.  Unlike
previous releases of the board, this incarnation has some very big changes,
so a lot of new driver writing will be needed before it's at all useful
under Plan 9.  It's quite daunting actually.

Quoting 9/bcm/words.pi4:

raspberry pi 4 -- work in progress!

broadcom 2838 SoC (based on 2711 cpu)
4 x cortex-a72 (1500Mhz, out-of-order pipeline)

Boots as far as the root prompt, but there are too many new
peripherals without driver support (or documentation so far)
to do anything useful.

- usb host sockets are all connected to new usb3 controller
- sdcard is connected to new emmc controller (can't use sdhost)
- ethernet is connected to new native gbe controller

Wifi is unchanged from 3B+, but currently fails intermittently
with emmc IO_RW_EXTENDED data corruption.

Needs firmware from 5 July 2019 or later.

config.txt for pi4 should include 'core_freq=250' for
the mini-uart console, and 'device_tree=' to ensure that
the loader passes an ATAG list to the kernel instead of
a device tree.