From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 29644 invoked from network); 22 Nov 2021 22:19:11 -0000 Received: from 4ess.inri.net (216.126.196.42) by inbox.vuxu.org with ESMTPUTF8; 22 Nov 2021 22:19:11 -0000 Received: from a8-21.smtp-out.amazonses.com ([54.240.8.21]) by 4ess; Mon Nov 22 17:09:06 -0500 2021 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=a4tryept6ew4kp6pdupzhtikhmwphypy; d=mapinternet.com; t=1637618483; h=From:Content-Type:Content-Transfer-Encoding:Mime-Version:Subject:Date:References:To:In-Reply-To:Message-Id; bh=90l2H9CeD8pb9S7jD7u9swbfoaS8QHzcHeQbHIU41jA=; b=AjU31rgf97M8ZMzAPFAmTEIj1jnQ4z8IIcpdNskprsDoEqPwAa3Ew3K3UdRDNK5v AjfFVkpNHTY7ogvuDz+33w6cZhvTR2JBO+ZNJbqsk9GSpf7TnNMX022TurBl43gW4xb MtK4jo4xB64VA9haO45Mgw+M5MkjFh8C+l3MBA9c= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ug7nbtf4gccmlpwj322ax3p6ow6yfsug; d=amazonses.com; t=1637618483; h=From:Content-Type:Content-Transfer-Encoding:Mime-Version:Subject:Date:References:To:In-Reply-To:Message-Id:Feedback-ID; bh=90l2H9CeD8pb9S7jD7u9swbfoaS8QHzcHeQbHIU41jA=; b=PO8x7Cju9SBzHwE2liF6raF0OZ/RTDBk6Pem7sEOluJ+0jlGNiC9MX5fymtz4zlw v+jfaI7U9rXugUdKzpg23li6CKPgQpzS1XC21q9Avw5HpgTOzml5qCiDIoRcg3/VDfC J0YdjBCAZzP8IrXehkQeyjZr8yU63n8hXgtyTLa0= X-Default-Received-SPF: pass (skip=loggedin (res=PASS)) x-ip-name=50.202.122.66; envelope-from=; From: Mack Wallace Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\)) Date: Mon, 22 Nov 2021 22:01:23 +0000 References: <151E7A4FFB5C3C650D2D9F7090B0AFD2@felloff.net> To: 9front@9front.org In-Reply-To: <151E7A4FFB5C3C650D2D9F7090B0AFD2@felloff.net> Message-ID: <0100017d49aafed3-57681c2d-2725-4b6b-af16-b0cfd55b85e0-000000@email.amazonses.com> X-Mailer: Apple Mail (2.3608.120.23.2.1) X-Qnum: 1320975210 X-Authenticated-User: mackbw@mapinternet.com Feedback-ID: 1.us-east-1.X+xhoL9JiEQ8K0gzGjV36WZnSewOzOs8YCWuakKsLBY=:AmazonSES X-SES-Outgoing: 2021.11.22-54.240.8.21 List-ID: <9front.9front.org> List-Help: X-Glyph: ➈ X-Bullshit: progressive service-scale hypervisor-based database Subject: Re: [9front] Porting RTL8111 (ether8169) driver to bcm64 Reply-To: 9front@9front.org Precedence: bulk So, I am trying to understand porting of the driver. I=E2=80=99ve made = some progress in understanding things - but not enough to get anything = functional. I know there is a bit of DMA work that needs to be done = which I have not even begun to broach, right now I am just focusing on = getting the adapter to be recognized. I did make changes to the = ether8169 driver from the /pc lineage and got that compiled. After some = discussion in a chat room (thanks Grobe0ba) I was directed to the Tegra2 = version of the driver. That didn=E2=80=99t solve anything, but between = that and looking at the dmesg in linux led me to believe I needed to be = looking at BAR2 rather than stupidly looking at BAR0 (sorry cinap should = have read your last message in better detail). =E2=80=94 However, the = only thing I=E2=80=99ve managed to fool into thinking I=E2=80=99m a = competent programmer is the compiler - which isn=E2=80=99t too hard. So = I am looking for a bit of inspiration here, and some basic knowledge.=20 In the pc driver, I first started changing all instances of =E2=80=98port=E2= =80=99 to =E2=80=98mmio=E2=80=99 as I would start to set things up as in = the usbxhci.c and vmap an area for io. Of course I then ran across = edev->port in rtl8169pnp and realized that this needs to be set. Of = course between type mismatches and not being to figure out how port is = defined in the Ether struct, that has served as an obstacle. After doing some fudging I was able to get it to compile so I could boot = the kernel and use some print statements in the code to get a better = understanding of the order calls were being made and whether the RTL8111 = was being recognized, which it is.=20 bus dev type vid did intl memory 0 0/0 06 04 00 14e4 2711 0 ioa:00000000-00001000 4096 = mema:600000000-600100000 1048576 ->1 1 0/0 02 00 00 10ec 8168 255 0:00000001 256 2:600000004 4096 = 4:600004004 16384=20 =E2=80=A6 Checking ether8169 We passed the first test PCIe is: 1 Mem0 is 256 bytes, and BAR: 1 rtl8169: 0x10ec 0x8168: port 0 size 256 irq 255 =E2=80=A6 #l1: rtl8169: reset failed Obviously, the first two lines would be from any 9front kernel, = recognizing PCI devices. The rest of the lines are from the driver itself - including some of my = debugging prints.=20 Being confused (not unusual) and wanting more information, the Linux = dmesg=E2=80=A6 [ 0.099846] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 = ranges: [ 0.099869] brcm-pcie fd500000.pcie: No bus range found for = /scb/pcie@7d500000, using [bus 00-ff] [ 0.099905] brcm-pcie fd500000.pcie: MEM = 0x0600000000..0x0603ffffff -> 0x00f8000000 [ 0.099943] brcm-pcie fd500000.pcie: IB MEM = 0x0000000000..0x003fffffff -> 0x0400000000 [ 0.134142] brcm-pcie fd500000.pcie: link up, 2.5 GT/s x1 (SSC) [ 0.134309] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00 [ 0.134327] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.134344] pci_bus 0000:00: root bus resource [mem = 0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff]) [ 0.134389] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400 [ 0.134479] pci 0000:00:00.0: PME# supported from D0 D3hot [ 0.137849] pci 0000:00:00.0: bridge configuration invalid ([bus = 00-00]), reconfiguring [ 0.137972] pci 0000:01:00.0: [10ec:8168] type 00 class 0x020000 [ 0.138022] pci 0000:01:00.0: reg 0x10: [io 0x0000-0x00ff] [ 0.138059] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x00000fff = 64bit] [ 0.138088] pci 0000:01:00.0: reg 0x20: [mem 0x00000000-0x00003fff = 64bit] [ 0.138217] pci 0000:01:00.0: supports D1 D2 [ 0.138230] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot = D3cold [ 0.141500] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to = 01 [ 0.141537] pci 0000:00:00.0: BAR 8: assigned [mem = 0x600000000-0x6000fffff] [ 0.141557] pci 0000:01:00.0: BAR 4: assigned [mem = 0x600000000-0x600003fff 64bit] [ 0.141587] pci 0000:01:00.0: BAR 2: assigned [mem = 0x600004000-0x600004fff 64bit] [ 0.141614] pci 0000:01:00.0: BAR 0: no space for [io size 0x0100] [ 0.141629] pci 0000:01:00.0: BAR 0: failed to assign [io size = 0x0100] [ 0.141643] pci 0000:00:00.0: PCI bridge to [bus 01] [ 0.141659] pci 0000:00:00.0: bridge window [mem = 0x600000000-0x6000fffff] [ 0.141806] pcieport 0000:00:00.0: enabling device (0000 -> 0002) [ 0.141928] pcieport 0000:00:00.0: PME: Signaling with IRQ 38 [ 0.142148] pcieport 0000:00:00.0: AER: enabled with IRQ 38 ... [ 6.777869] r8169 0000:01:00.0: enabling device (0000 -> 0002) [ 6.792249] r8169 0000:01:00.0: can't read MAC address, setting = random one [ 6.803140] libphy: r8169: probed [ 6.806902] r8169 0000:01:00.0 eth1: RTL8168h/8111h, = d6:3a:e5:18:0b:68, XID 541, IRQ 39 [ 6.814951] r8169 0000:01:00.0 eth1: jumbo features [frames: 9200 = bytes, tx checksumming: ko] I have been confused by whether I need to assign an address to the = rtl8169 and if so, what address to use. Confused by the two PCI devices = recognized as being in the same address space (0x600000000) though on = different busses. Do I need to include devpnp.c? After looking at ethergenet.c, and what the data sheet says on page 37 = of the registers data sheet, I believe (correct me if I am wrong) that = the IOBAR and MEMBAR need to have their addresses set. And looking at = the BCM2711 ARM Peripherals datasheet = (https://datasheets.raspberrypi.com/bcm2711/bcm2711-peripherals.pdf), it = seems that the peripherals are usually mapped to either between 0x7C00 = 0000-0x8000 0000. Those addresses may also be represented as 0x4 7C00 = 0000 - 0x4 8000 0000 or 0x0FC00 0000 - 0x0 FF80 0000 depending on which = view of the address map one is using? The VIRTIO seems to maps into that = 0x7c00 0000 region through comments in mem.h: mem.h:#define VIRTIO2 (0xFFFFFFFFBC000000ULL) /* 0x7C000000 = - 0xFC000000 */ and there are the other VIRTIO definitions=E2=80=A6 mem.h:#define VIRTIO1 (0xFFFFFFFFBD000000ULL) /* 0x7D000000 = - 0xFD000000 */ mem.h:#define VIRTIO (0xFFFFFFFFBE000000ULL) /* 0x7E000000 = 0x3F000000 0xFE000000 */ I see that ethergenet.c seems to assign an address through ctlr->regs=E2=80= =A6 static int pnp(Ether *edev) { static Ctlr ctlr[1]; if(ctlr->regs !=3D nil) return -1; ctlr->regs =3D (u32int*)(VIRTIO1 + 0x580000); ctlr->rx->regs =3D &ctlr->regs[RdmaOffset + nelem(ctlr->rd)*3 + = 16*RingCfg]; ctlr->tx->regs =3D &ctlr->regs[TdmaOffset + nelem(ctlr->td)*3 + = 16*RingCfg]; edev->port =3D (uintptr)ctlr->regs; But I am confused by the pc version of the rtl8169 device - first = calling rtl8169pnp which fairly soon after calls on rtl8169pci. I = originally thought that I would set up my vmap under the pci routine. = Looking at the PCIDev there, i am getting the correct information about = the vendor and device id, but then there are some calls to vet (get) the = mac address, but before a base address would be established under pnp? I = suppose an address could be established in either pnp or pci, but I = presume it makes more sense to do it in the pnp routine. I guess my = question is how do I establish the address? I see in ethergenet = addresses written to the ctlr->regs, but that driver doesn=E2=80=99t = seem to have any mention of pci, so I am guessing that is not it. I=E2=80=99= m guessing this has to be done through the Pcidev structure and some = call to write to that address? pcicfgw maybe? - Then what address do I = use? VIRTIO1 + 0x680000 (I have no idea what the memory window is for = ethergenet.c is, I just added the window of the other pci device, which = of course is not located at VIRTIO1+0x580000, so 0x680000 is a bullshit = number - so I need to figure out a free window above ethergenet? Is = there a general rule of thumb regarding how and where such spaces should = be allocated (e.g. every PCI device gets 16k or something like that? Below is some of my initial (ignorant) coding in the rtl8169pci routine. = I need to better understand the purpose for Ctlr->port, vs = Etherdev->port, vs Ctlr->regs. I/O? DMA? Memory?=20 At the bottom are the results of lspci.=20 Hope that someone can help with some pieces of the puzzle. Thanks, mackbw print("Mem2 is %d bytes, and BAR: %llux\n", p->mem[2].size, = p->mem[2].bar); if(p->mem[0].size =3D=3D 0 || (p->mem[0].bar & 1) =3D=3D = 0) continue; if(p->mem[2].bar & 1) //checks to see if this is = "memory" or "io" continue; io =3D p->mem[2].bar & ~0x0f; if(io =3D=3D 0) continue; print("rtl8169: %#x %#x: port %llux size %d irq %d\n", p->vid, p->did, io, p->mem[2].size, p->intl); mmio =3D vmap(io, p->mem[2].size); if(mmio =3D=3D nil){ print("rtl8169: cannot map registers\n"); continue; } ctlr =3D malloc(sizeof(Ctlr)); if(ctlr =3D=3D nil){ print("rtl8169: can't allocate memory\n"); vunmap(mmio, p->mem[2].size); continue; } ctlr->port =3D io; ctlr->mmio =3D mmio; 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. = RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15) 00: ec 10 68 81 06 04 10 00 15 00 00 02 10 00 00 00 10: 01 00 00 00 00 00 00 00 04 40 00 f8 00 00 00 00 20: 04 00 00 f8 00 00 00 00 00 00 00 00 ec 10 68 81 30: 00 00 00 00 40 00 00 00 00 00 00 00 26 01 00 00 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. = RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15) Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI = Express Gigabit Ethernet controller Flags: bus master, fast devsel, latency 0, IRQ 39 I/O ports at [disabled] Memory at 600004000 (64-bit, non-prefetchable) [size=3D4K] Memory at 600000000 (64-bit, non-prefetchable) [size=3D16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable+ Count=3D1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable- Count=3D4 Masked+ Capabilities: [100] Advanced Error Reporting Capabilities: [140] Virtual Channel Capabilities: [160] Device Serial Number 00-00-00-00-00-00-00-00 Capabilities: [170] Latency Tolerance Reporting Capabilities: [178] L1 PM Substates Kernel driver in use: r8169 lspci: Unable to load libkmod resources: error -12 > On Nov 14, 2021, at 12:57 AM, cinap_lenrek@felloff.net wrote: >=20 > = https://recomb-omsk.ru/published/SC/html/scripts/doc/RTL8111B_8168B_Regist= ers_DataSheet_1.0.pdf >=20 > or just search for "rtl8111 registers datasheet" on the internet. >=20 > page 28 descibes that it has a 64-bit MEMBAR (bar2/bar3) for the > registers as well as a IOBAR (bar0). >=20 > -- > cinap >=20