[9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit Hypervisor

[9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit Hypervisor

[Kenneth L Van Alstyne]

Just curious if anyone has any experience running Plan 9 under Xen with a
64-bit hypervisor?  It's my understanding that the PAE kernel should work,
but upon creation of the DomU, the machine quickly panics.  Example output
is below:

wired:~# xm create -c /home/kvanals/plan9/boot.conf
Using config file "/home/kvanals/plan9/boot.conf".
Started domain plan9.kvanals.org

Plan 9
127 holes free
0018b000 00dba000 12775424
12775424 bytes free
cpu0: 2792MHz AuthenticAMD AMD64 (cpuid: AX 0x20F51 DX 0x78BFBFF)
panic: xenupdatema - pte 80185010 value 269a5027
panic: xenupdatema - pte 80185010 value 269a5027
dumpstack
ktrace /kernel/path 8001444a 80002e2c <<EOF
estackx 80003000
80002dcc=8001428b 80002dd8=80040514 80002ddc=8006b902 80002de4=8006b963
80002df0=80072b4a 80002e00=8004065b 80002e10=8006cef8 80002e14=8001444a
80002e28=8001444a 80002e2c=8001428f 80002e34=80040a81 80002e48=8006a91a
80002e84=8007380a 80002e90=80074682 80002ea4=80073a82 80002ed0=8006cf37
80002ed4=8006a91a 80002edc=80072b4a 80002ef0=80070000 80002ef4=80070044
80002f00=8006cef8 80002f0c=8006cef8 80002f10=80070276 80002f18=8007055a
80002f24=8006a91a 80002f38=80070661 80002f44=8006c3e7 80002f60=8006ca30
80002f8c=80011cc0 80002fac=80011be5 80002fb8=80012208 80002fc4=80012254
80002fe4=00000000 80002fe8=801c6ff8 80002fec=80010a37 80002ff0=800b209f
80002ff4=00000000 80002ff8=8001001f 80002ffc=00000000
EOF
cpu0: exiting
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
...

Thanks ahead of time for any assistance anyone is able to provide.

Thanks,
Kenny

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit Hypervisor

[Richard Miller]

> Just curious if anyone has any experience running Plan 9 under Xen with a
> 64-bit hypervisor?  It's my understanding that the PAE kernel should work,
> but upon creation of the DomU, the machine quickly panics.

I've updated /n/sources/xen/xen3 source and sample kernels with a small
mmu correction which was needed for Xen 3.4 (thanks to Peter Bosch for
tracking down the bug).  Maybe this will help for your configuration too.

How much physical RAM do you have?  I'm not aware of the PAE code having
been tested with more than 4GB.  If your machine is bigger, you could
try booting Xen with RAM restricted to, say, 2GB (should be a linux boot
parameter for this) and see if it makes a difference.

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit Hypervisor

[erik quanstrom]

> I've updated /n/sources/xen/xen3 source and sample kernels with a small
> mmu correction which was needed for Xen 3.4 (thanks to Peter Bosch for
> tracking down the bug).  Maybe this will help for your configuration too.
>
> How much physical RAM do you have?  I'm not aware of the PAE code having
> been tested with more than 4GB.  If your machine is bigger, you could
> try booting Xen with RAM restricted to, say, 2GB (should be a linux boot
> parameter for this) and see if it makes a difference.

could you explain this a little further.  pae would normally
not be useful with 2gb of ram and is often of dubious utility
on a 4gb machine.  what's different about xen?

- erik

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit

[Richard Miller]

> could you explain this a little further.  pae would normally
> not be useful with 2gb of ram and is often of dubious utility
> on a 4gb machine.  what's different about xen?

PAE is of no utility at all as far as I can see, but sometimes
we have to run xen guests on somebody else's hypervisor (think
virtual hosting companies).  If the hypervisor is configured
with PAE (or 64-bit, it seems), the guest has to run in PAE mode.

I hacked the Plan 9 xen kernel to support PAE for this reason,
but since my xen test machine has only a couple of gigs I have
never actually seen it work in the non-trivial case where the
upper half of a 64-bit page address is non-zero.  If our friend
has more than 4GB and the panic disappears when he uses less
than 2GB, that might indicate why he's having problems that
nobody else has seen.

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit

[Jano]

On Feb 13, 11:20 am, 9f...@hamnavoe.com (Richard Miller) wrote:
> > I'll also announce to the list when (if) we get it working.
>
> It's working now.  Turned out to be a known error in xen 3.2 (hypervisor
> was initialising the mmu page directory table incorrectly).  9xen kernels
> on sources now have a simple workaround for this.

wow awesome!
I'll try it soon..
Thanks,
John

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit

[Richard Miller]

> I'll also announce to the list when (if) we get it working.

It's working now.  Turned out to be a known error in xen 3.2 (hypervisor
was initialising the mmu page directory table incorrectly).  9xen kernels
on sources now have a simple workaround for this.

Re: [9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit

[Richard Miller]

Jano <sorosj@gmail.com>	Thu 11 Feb 09:55:
> ... if you have continued off-list,
> please cc me

Will do.  I'll also announce to the list when (if) we get it working.