Re: [9front] Porting RTL8111 (ether8169) driver to bcm64

[Mack Wallace <mackbw@mapinternet.com>]

What was outlined was generally what I was doing. A notable difference was when pcienable occurred - I don’t know whether it is that important to execute that before or after the vmap.

Previously, the controller did not appear to be initializing, and so my question was whether or not the PCI setup was occurring at all. Whether 9front was set up to properly configure extra PCIe devices, or if more work was needed.  

I’ve moved around my code - tried a few different things, just trying to see if I can get the device to reset or read whether I have a link - and I am failing at those basic tasks… and there are a few symptoms that concern me, but I am not sure whether I should be concerned.

Symptom 1: the irq is 255. Maybe it is a non-issue - maybe expected the way interrupts are allocated, I don’t know.

Symptom 2: I get an odd MAC address: c6:3d:40:80:0:0. I say this is odd for a few reasons. The last two octets being zero are unusual, but also knowing that this board does not have a rom to load a MAC address from and that the linux driver, upon failing to read a MAC address from the unit, assigns its own. I would presume that without a ROM, that such an unassigned address would be all zeros… but that’s a complete guess. 

The data sheet seems to contradict itself regarding the MAC address field: "ID registers 0-5 are only permitted to read/write via 4-byte access. Read access can be byte, word, or double word access. …” I’ve attempted reading by bytes to check if the latter statement is true, but I get something completely different than when I read via 4-byte access. So maybe only 4-byte access is viable. However, if I read the first four bytes of the MAC address (by either mmio[0], or by using the drivers csr32r(ctlr, 0)), and then the last four bytes (by either mmio[2], or by using the drivers csr32r(ctlr, 2)), the latter value does not seem to contain the two middle bytes shifted - i.e. for the first read I get 0x8678a1c6, and the second read with a two byte offset I get 0x00020040 when I expect 0x0000c6a1. 

Symptom 3: The lower nibble of Config1 is read only and should always be 0xF. - But when I read it, it is 0.

There are some other items - but it seems that I am missing something in the vmap.

I’ll include my current code - debugging acrobatics and print statements and all. I know, ‘io’ is a misnomer as I am trying to look at the PCI memory. Although, looking at the data sheet. the MAC registers are available in both I/O and PCI Memory space. I’d try the I/O space, but mem[0].bar = 0x01 - which makes the address 0x0 - though this seems consistent with the linux driver, and then linux just says that it can not allocate the memory for the i/o space.

— thanks, 
mackbw


Current Code: 

static void
rtl8169pci(void)
{
	Pcidev *p;
	Ctlr *ctlr;
	int i, pcie;
	uvlong io;
	u32int *mmio, a, b;
	uint macv, r;

	p = nil;
	print("Checking ether8169\n");
	while(p = pcimatch(p, 0, 0)){
		if(p->ccrb != 0x02 || p->ccru != 0)
			continue;
		pcie = 0;
		switch(i = ((p->did<<16)|p->vid)){
		default:
			continue;
		case Rtl8100e:			/* RTL810[01]E ? */
		case Rtl8168b:			/* RTL8168B */
			pcie = 1;
			break;
		case Rtl8169c:			/* RTL8169C */
		case Rtl8169sc:			/* RTL8169SC */
		case Rtl8169:			/* RTL8169 */
			break;
		case (0xC107<<16)|0x1259:	/* Corega CG-LAPCIGT */
			i = Rtl8169;
			break;
		}

		print("Mem0 is %d bytes, and BAR: %llux\n", p->mem[0].size, p->mem[0].bar);
		print("Mem2 is %d bytes, and BAR: %llux\n", p->mem[2].size, p->mem[2].bar);

		if(p->mem[2].bar & 1) //checks to see if this is "memory" or "io"
			continue;

		pcienable(p);

		io = p->mem[2].bar & ~0xF;
		mmio = vmap(io, p->mem[2].size);

		if(mmio == nil){
			print("rtl8169: cannot map registers\n");
			continue;
		}

		ctlr = malloc(sizeof(Ctlr));
		if(ctlr == nil){
			print("rtl8169: can't allocate memory\n");
			vunmap(mmio, p->mem[2].size);
			continue;
		}
		ctlr->port = (uint)mmio;
		ctlr->mmio = mmio;
		ctlr->pcidev = p;
		ctlr->pciv = i;
		ctlr->pcie = pcie;
		print("rtl8169: %#x %#x: port %ux size %d irq %d\n",
			p->vid, p->did, *mmio, p->mem[2].size, p->intl);

		if(vetmacv(ctlr, &macv) == -1){
			print("rtl8169: %T: unknown mac %.4ux %.8ux\n", p->tbdf, p->did, macv);
			pcidisable(p);
			vunmap(mmio, p->mem[2].size);
			free(ctlr);
			continue;
		}


/*		if(rtl8169reset(ctlr)){
			print("Failed to reset!\n");
			pcidisable(p);
			vunmap(mmio, p->mem[2].size);
			free(ctlr);
			continue;
		} */

		print("bytes of MAC address %ux:%ux:%ux:%ux:%ux:%ux\n", mmio[0], mmio[1], mmio[2], mmio[3], mmio[4], mmio[5]);
		a = mmio[0];
		b = mmio[2];
		print("bytes of MAC address %ux:%ux\n", a, b);

		print("Configuration byte: %ux\n", mmio[Config1]);

	r = csr8r(ctlr, Phystatus);

	/*
	 * Maybe the link changed - do we care very much?
	 * Could stall transmits if no link, maybe?
	 */
	print ("Phystatus: %ux\n", r);
	if (r & Linksts)
		print("pre: We have link!\n");
	else
		print("pre: We don't have a link!\n");

		rtl8169halt(ctlr);

		/*
		 * Extract the chip hardware version,
		 * needed to configure each properly.
		 */

		ctlr->macv = macv;
		if(rtl8169ctlrhead != nil)
			rtl8169ctlrtail->next = ctlr;
		else
			rtl8169ctlrhead = ctlr;
		rtl8169ctlrtail = ctlr;
	}
}



Suggested Code from cinap:
> 
> // find my device.
> Pcidev *pcidev = pcimatch(....);
> 
> // bring device out of suspend
> pcienable(pcidev);
> 
> // make sure it has the membar
> if(pcidev->mem[2].bar&1){
> 	print("this is not a membar\n");
> 	return -1;
> }
> 
> // make sure it has the expected size
> if(pcidev->mem[2].size <= HOWMANBYTESAREMYREGISTERBLOCK){
> 	print("the membar is too small, does not cover all the registers\n");
> 	return -1;
> }
> 
> // map it into kernel virtual memory
> uint32 *regs = vmap(pcidev->mem[2].bar&~0xF, pcidev->mem[2].size);
> 
> if(regs == nil){
> 	print("can't map it\");
> 	return -1;
> }
> 
> // then you can poke at the register
> regs[CONTROL123] = 1;
> 
> thats it.
> 
> —
> cinap

Code from my prior message:
> 	print("Mem2 is %d bytes, and BAR: %llux\n", p->mem[2].size, p->mem[2].bar);
> 		if(p->mem[0].size == 0 || (p->mem[0].bar & 1) == 0)
> 			continue;
> 
> 		if(p->mem[2].bar & 1) //checks to see if this is "memory" or "io"
> 			continue;
> 		io = p->mem[2].bar & ~0x0f;
> 		if(io == 0)
> 			continue;
> 		print("rtl8169: %#x %#x: port %llux size %d irq %d\n",
> 			p->vid, p->did, io, p->mem[2].size, p->intl);
> 		mmio = vmap(io, p->mem[2].size);
> 		if(mmio == nil){
> 			print("rtl8169: cannot map registers\n");
> 			continue;
> 		}
> 		ctlr = malloc(sizeof(Ctlr));
> 		if(ctlr == nil){
> 			print("rtl8169: can't allocate memory\n");
> 			vunmap(mmio, p->mem[2].size);
> 			continue;
> 		}
> 		ctlr->port = io;
> 		ctlr->mmio = mmio;