[9fans] Netbooting from Qemu

[9fans] Netbooting from Qemu

[Jerome Ibanes]

Hi,

I am (net)booting a Qemu instance from a Plan 9 fileserver named
colossus(192.168.1.40) running cwfs, an authserver, named
cerberus(192.168.1.50) is also present in the same domain.

The client is named cpu-003(192.168.1.33), it retrieves, via dhcp its
plan9.ini, which is as such:

  bootfile=ether0!/386/9pccpu.gz
  bootargs=tcp!192.168.1.40!564 -D
  fs=192.168.1.40
  auth=192.168.1.50
  sysname=cpu-003
  *nomp=1
  *debugload=1
  *nodumpstack=1

The dhcpd configuration uses /lib/ndb/local to properly serve dhcp
queries, relevants bits as follow:

  ipnet=drawterm.com ip=192.168.1.0 ipmask=255.255.255.0
    dnsdomain=drawterm.com
    authdom=drawterm.com
    ipgw=192.168.1.1
    dns=192.168.1.1
    ntp=192.168.1.1
    auth=cerberus
    fs=colossus
    cpu=cpu-001
    smtp=192.168.1.1

  ip=192.168.1.33 sys=cpu-003 ether=525400123456
    dom=drawterm.com
    bootf=/386/9pxeload
    proto=tcp

When booting a 9pc kernel (which means that bootfile=ether0!/386/9pc.gz)
the system boots, then prompt for a user login, secstore password and
behaves as a terminal, as one can expect.

When booting a 9pccpu kernel (with the configuration above, and not
another change from the 9pc kernel) the system hangs as such (please
notice that I have ipconfig debug mode turned on in the bootargs):

  [...]
  ipconfig: parsebootp: new packet
  ipconfig: parseoptions: type(53) len 1, bytes left 71
  ipconfig: parseoptions: lease(51) len 4, bytes left 68
  ipconfig: parseoptions: serverid(54) len 4, bytes left 62
  ipconfig: parseoptions: ipmask(1) len 4, bytes left 56
  ipconfig: parseoptions: ipgw(3) len 4, bytes left 50
  ipconfig: parseoptions: sys(12) len 12, bytes left 44
  ipconfig: parseoptions: dns(6) len 4, bytes left 30
  ipconfig: parseoptions: dom(15) len 3, bytes left 24
  ipconfig: parseoptions: ntp(42) len 4, bytes left 19
  ipconfig: parseoptions: <nil>(43) len 12, bytes left 13
  ipconfig: got ack from 192.168.1.40
  ipconfig: lease=1800
  ipconfig: ipaddr=192.168.1.33 ipmask=255.255.255.0
  ipconfig: ipgw=192.168.1.1
  ipconfig: dns=192.168.1.1
  ipconfig: ntp=192.168.1.1
  ipconfig: parseoptions: <nil>(128) len 4, bytes left 10
  ipconfig: parseoptions: <nil>(129) len 4, bytes left 4
  ipconfig: fs=192.168.1.40
  ipconfig: auth=192.168.1.50
  ipconfig: new ipaddr=192.168.1.33 new ipmask=255.255.255.0 new ipgw=192.168.1.1
  ipconfig: server=192.168.1.40 sname=colossus

Then the system hangs, the cwfs console reports that no connection was
established from cpu-003(192.168.1.33), while, when booting a 9pc kernel,
the connection is established and the boot sequence follows.

Finally, replacing 'tcp!192.168.1.40!564' by 'tcp' or 'tcp!colossus!564'
leads to the same results, I do not believe it to be a misconfiguration
however as 9pc is booting properly.

Despite that 9pccpu is hanging, I am able to ping this host
cpu-003(192.168.1.33) therefore the network card has been found and
initialized properly. A 'snoopy' reports that after the last dhcp queries,
no packets are ever sent from cpu-003(192.168.1.33).

What would cause this configuration to boot using a stock 9pc kernel but
not a 9pccpu one? I've even increased the memory allocation of the Qemu
instance as I would expect a 9pccpu kernel to be slightly bigger than a
9pc one.


Any suggestion?
  Jerome

Re: [9fans] Netbooting from Qemu

[erik quanstrom]

>   bootfile=ether0!/386/9pccpu.gz
>   bootargs=tcp!192.168.1.40!564 -D
>   fs=192.168.1.40
>   auth=192.168.1.50
>   sysname=cpu-003
>   *nomp=1
>   *debugload=1
>   *nodumpstack=1

i think that should be should be (indent for clarity)

	bootfile=ether0!$mydhcpserver!/386/9pccpu.gz
	nobootprompt=tcp

otherwise i believe you will be prompted on the
console for a root fs.

also, you would be much better off creating a ipnet
entry in ndb.  i have a dim recollection of having trouble
when this was set up incorrectly, causing cs confusion.
setting the ipnet stuff allows the fs ip addr to be
inferred.

in your case, you want entries something like this:

	ipnet=mynet ip=192.168.1.0 ipmask=/120
		fs=myfs.example.net
		ipgw=192.168.1.?
		gw=mygw.example.net
		dns=192.168.1.40
		dnsdomain=example.net
		authdom=myauthdom
		auth=myfs
		cpu=myfs

	sys=mygw ip=192.168.1.? ether=000102030405
		dom=mygw.example.net

	sys=myfs ip=192.168.1.40 ether=000102030405
		dom=myfs.example.net

	sys=cpu-003 ip=192.168.1.33 ether=000102030405
		dom=cpu-003.example.net
		bootf=/386/9pxeload

- erik

Re: [9fans] Netbooting from Qemu

[Jerome Ibanes]

As it appears, the (net)boot configuration, meaning the 'plan9.ini'
located in /cfg/pxe/xxxxxxxxxxxx, where 'xxxxxxxxxxxx' is the netbooted
host ethernet address, was missing nvram parameters, preventing the cpu
kernel (9pccpu) to proceed with the boot sequence.

If this parameter (nvram or nvr) is present, but incorrectly set, the
netbooted host will prompt for the machine's hostowner's key. If this
parameter is missing, the boot sequence halts.

Therefore I would like to ask the 9fans community what are the best
practices to host the nvram key in a diskless environment, in either a
qemu virtualized machine or physical hardware.

I am aware of serial Eeproms connecting to parallel ports to store the
nvram data ( http://rs-rlab.narod.ru/9nvram.html ), I find it a good
solution, but unfortunately, most "modern" hardware doesn't necessarly
include a parallel port anymore, or even a floppy disk.

I assume (perhaps incorrectly) that the netbooted host can not use a nvram
store located on kfs. Please share with the list if you are aware, or use
a different method to store your nvram data, either in virtualized
machines or physical hardware. Preferably without the use of disk/floppy
storage.


Sincerely,
  Jerome

Re: [9fans] Netbooting from Qemu

[erik quanstrom]

> I assume (perhaps incorrectly) that the netbooted host can not use a nvram
> store located on kfs. Please share with the list if you are aware, or use
> a different method to store your nvram data, either in virtualized
> machines or physical hardware. Preferably without the use of disk/floppy
> storage.

all my personal machines and coraid's cpu servers, save the
auth servers are pxe booted yet have a local nvram partition
or a file in 9fat called "plan9.nvr".  the partition must be
a partition made with prep and called "nvram".  if you use
a file, it must be in 9fat, not kfs.

- erik

Re: [9fans] Netbooting from Qemu

[Steve Simon]

I would have thought a small USB stick would be the way to go,
My latest motherboard even had one USB port inside the case
for just this kind of thing.

-Steve

Re: [9fans] Netbooting from Qemu

[Tim Newsham]

> store located on kfs. Please share with the list if you are aware, or use
> a different method to store your nvram data, either in virtualized
> machines or physical hardware. Preferably without the use of disk/floppy
> storage.

If its an emulated environment, just put it on an emulated disk.
Its really not much different than an emulated nvram.

>  Jerome

Tim Newsham
http://www.thenewsh.com/~newsham/