Re: [developer] Re: illumos/uefi-edk2 update 2023-11

[Stephan Althaus <Stephan.Althaus@Duedinghausen.eu>]

Hi All!

I thought i'd try out a GPU passthrough on OI to a Windows vm with bhyve 
again.

This time it is an AMD RX 6400, and i verified the System settings with 
FreeBSD 4.1, there the GPU is working, so the BIOS settings and the 
GPU-ROM are ok.

The VM startscripts and data are within a NVME zpool, so that i can 
switch to OI easily and have the same with FrreBSD and OI.

When i enable immu-intrmap-enable="true"; in 
/platform/i86pc/kernel/drv/rootnex.conf
this leads to error messages when i start the bhyve test script with 
messages
APIC ERROR Interrupt on CPU 0. Status 0 = 0 Status 1 = 40.
on the console and the system is not responsive anymore..

When i leave it toimmu-intrmap-enable=" false", the Windows vm starts 
and the GPU has error 43 within the device-manager.

What can i do to see where the APIC error comes from ? some dtrace 
script maybe ?

Any help is appreciated.

Regards,
Stephan

P.S. The test command:

/usr/sbin/bhyve -U bd60bf59-0ed6-61a1-f0ed-b17e99ada165 -A -H \
-B 
1,manufacturer=OpenIndiana,product=OpenIndianaHVM,version=1.0,serial=69a18a85-c1df-c266-f620-a99a632d3c11,sku=001,family=VirtualMachine 
\
-c 4 -m 16G -w \
-s 31:0,lpc -l bootrom,/home2/vm/BHYVE_UEFI.fd \
-s 0,hostbridge,model=i440fx  \
-s 4:0,nvme,/dev/zvol/rdsk/home2/vm/win10vol \
-s 6:0,virtio-net-viona,vnic0 \
-s 7:0,ahci-cd,/home2/vm/virtio-win-0.1.240.iso \
-s 30:0,fbuf,tcp=0.0.0.0:5905,w=1920,h=1080 \
-s 30:1,xhci,tablet \
-S \
-s 8:0,passthru,/dev/ppt0,rom=/home2/vm/Navi24.rom \
-s 8:1,passthru,/dev/ppt1 \
win10

$ pptadm
DEV        VENDOR DEVICE PATH
/dev/ppt0  1002   743f 
/pci@7a,0/pci8086,6f08@3/pci1002,1478@0/pci1002,1479@0/display@0
/dev/ppt1  1002   ab28 
/pci@7a,0/pci8086,6f08@3/pci1002,1478@0/pci1002,1479@0/pci1002,ab28@0,1


On 2/16/24 10:29, Stephan Althaus wrote:
> Hello!
>
> Small update on this..
>
> The goal i want to achieve is to have a GPU passthrough on OI to a 
> Windows vm with bhyve...
>
> I now tested linux/proxmox (kvm)  and FreeBSD 14 (bhyve) with success, 
> so i can assume the Hardware and the BIOS settings are ok.
>
> FreeBSD has this version of the edk2 image:
>
> $ pkg search edk2
> edk2-bhyve-g202308_4           EDK2 Firmware for bhyve
>
> .. so version 202311 seems not to be needed (?)
>
> What i see if i try it on OI, is that
>
> immu-intrmap-enable="true";
>
> in
>
> /platform/i86pc/kernel/drv/rootnex.conf
>
> leads to error messages when i start the bhyve test script with messages
> APIC ERROR Interrupt on CPU 0. Status 0 = 0 Status 1 = 40.
> on the console and the system is not responsive anymore..
>
>
> Do i _need_ immu-intrmap-enable="true"; ?
>
>
> I currently have these values set:
>
> immu-enable="true";
> #immu-intrmap-enable="true";
> immu-blacklist=
>                 "SMBIOS","Supermicro","X10DRI-T",""
>                 ;
> immu-dvma-mapping="xlate";
> immu-dvma-unity-drivers="igb","ahci","ixgbe","ehci","xhci","nvme";
> immu-dvma-xlate-drivers="mpt";
>
> Is anyone around with a success story who has been there and done that ?
>
>
> i want to passtrough this:
>
> $ pptadm list -o all
> DEV        VENDOR DEVICE SUBVENDOR SUBDEVICE REV 
> PATH                                              LABEL
> /dev/ppt0  1002   743f   1da2      e458      c7 
> /pci@7a,0/pci8086,6f08@3/pci1002,1478@0/pci1002,1479@0/display@0 
> Advanced Micro Devices, Inc. [AMD/ATI] Navi 24 [Radeon RX 6400/6500 
> XT/6500M]
> /dev/ppt1  1002   ab28   1002      ab28      0 
> /pci@7a,0/pci8086,6f08@3/pci1002,1478@0/pci1002,1479@0/pci1002,ab28@0,1 
> Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio 
> Controller
>
> ... using this test script:
>
> $ /usr/sbin/bhyve -U bd60bf59-0ed6-61a1-f0ed-b17e99ada165 -A -H \
> -B 
> 1,manufacturer=OpenIndiana,product=OpenIndianaHVM,version=1.0,serial=69a18a85-c1df-c266-f620-a99a632d3c11,sku=001,family=VirtualMachine 
> \
> -c 4 -m 16G -w \
> -s 31:0,lpc -l bootrom,/usr/share/bhyve/firmware/BHYVE.fd,fwcfg=qemu \
> -s 0,hostbridge,model=i440fx  \
> -s 4:0,nvme,/dev/zvol/rdsk/home2/vm/win10vol \
> -s 6:0,virtio-net-viona,vnic0 \
> -s 30:0,fbuf,vga=off,tcp=0.0.0.0:5905 \
> -s 30:1,xhci,tablet \
> -S -s 8:0,passthru,/dev/ppt0,rom=/home2/vm/Navi24.rom \
> -s 8:1,passthru,/dev/ppt1 \
> win10
>
> The GPU can be seen within the vm (error 43), it seems the card 
> doesn't get initialized..
>
> Any hints are welcome..
>
>
> Regards,
>
> Stephan
>
>
>
> ------------------------------------------
> illumos: illumos-developer
> Permalink: 
> https://illumos.topicbox.com/groups/developer/T0e71dad0bc17329a-M635020990ad99de8499b28ad
> Delivery options: 
> https://illumos.topicbox.com/groups/developer/subscription



------------------------------------------
illumos: illumos-developer
Permalink: https://illumos.topicbox.com/groups/developer/T0e71dad0bc17329a-M54c5f6bf883f52445dbaedfd
Delivery options: https://illumos.topicbox.com/groups/developer/subscription