Re: [9fans] plan9: how to get irq to mach routing

[erik quanstrom <quanstro@quanstro.net>]

On Thu Jan 24 05:09:23 EST 2013, rgandhasri@gmail.com wrote:
> Hi,
> 
> I would like to retrieve what IRQs are routed to what Machs (or
> lapics).  I dumped IOAPIC's Redirection Table entries, but Destination
> bits [63:48] are always '0xFF'.  Is there way to determine the routing
> between IRQs and Machs/LAPICs serving them?.  One rudimentary way I
> tried is that, in the interrupt service routine I accessed the 'm'
> (Mach *) external variable to figure the routing between the IRQ and
> and the Mach..  Wondering if there is a better way.  (for all legacy,
> msi and msi-x interrupt types)

leaving 8259 interrupts to the side, in a system using apics, there are
four potential sources of interrupts for a processor
- processor traps,
- the lapic.  1 lapic attached to each processor
- i/o apics
- device sending an msi/msi-x interrupt.

obviously lapic interrupts (such as the clock timer) and traps
(such as #PF) are handled by the processor that generated them.
in flat or cluster mode, external interrupts are usually routed to
the "lowest priority" processor.  in physical mode, a particular
processor is targeted.  msi reuses the same logic.

the distribution uses physical addressing.  this means that interrupt
ι always targets processor p.  so the high bits should
match the processor picked.  see
	/n/sources/plan9/sys/src/9/pc/mp.c:793
	/n/sources/plan9/sys/src/9/pc/mp.c:824
i have not run this code to check, but it doesn't look like it sets
the redirection to broadcast.  (iirc, it's illegal to target a non-existent
processor from an i/o apic.)

i was having trouble with apics a few years ago due to the hardware
and thus did the apic implementation described in the man page.
this implementation supports flat, cluster & physical modes:

	http://www.quanstro.net/magic/man2html/3/apic

there are a number of files in '#P' that should help you poke
around such as '#P/mpirq' which prints out the vectors as
programmed.

nix uses the same implementation with msi, but only
uses physical mode in round-robin fashion.  i need to
merge these.  there's no reason for them to be different.

here's an example from a machine with physical-mode
vectors.  the i/o apic targets lapic 00 and 04.  
minooka;  cat '#P/mpvec' | grep -v '10000$'
           8           1 0000000000000141
           8           4 0400000000000161

here are the processors so we can find them.
(this format keeps changing.  don't depend on it.  :))

minooka; cat '#P/mpapic'
proc            0 00000000 00000000 00000000 be            0 0xfee00000
proc            1 01000000 01000000 01000000  e            8 0xfee00000
proc            2 02000000 02000000 02000000  e            1 0xfee00000
proc            3 03000000 03000000 03000000  e            9 0xfee00000
proc            4 04000000 04000000 04000000  e            2 0xfee00000
proc            5 05000000 05000000 05000000  e           10 0xfee00000
proc            6 06000000 06000000 06000000  e            3 0xfee00000
proc            7 07000000 07000000 07000000  e           11 0xfee00000
ioapic          8 00000000 00000000 00000000  e            0 0xfec00000
ioapic          9 00000000 00000000 00000000  e            0 0xfec8a000
proc           10 10000000 10000000 10000000  e            4 0xfee00000
proc           11 11000000 11000000 11000000  e           12 0xfee00000
proc           12 12000000 12000000 12000000  e            5 0xfee00000
proc           13 13000000 13000000 13000000  e           13 0xfee00000
proc           14 14000000 14000000 14000000  e            6 0xfee00000
proc           15 15000000 15000000 15000000  e           14 0xfee00000
proc           16 16000000 16000000 16000000  e            7 0xfee00000
proc           17 17000000 17000000 17000000  e           15 0xfee00000

so this is processor 0 and 4.  (that's not normally the case.  this
machine is pretty easy.)

for msi,

minooka;  /mnt/term/386/bin/pci -m | grep 00000000fee
#	flags	target addr	data		next ptr
0.31.2	0009	00000000fee05000	00004069	80
1.0.0	0181	00000000fee02000	00004051	50
1.0.1	0181	00000000fee03000	00004059	50
3.0.0	0081	00000000fee01000	00004049	44

bits 16:12 are the target.  so we see that we've targeted lapic 1, 3, 2, 5.
(which are fortunately for this example, identity mapped.)

- erik