[9fans] new lguest port available

[9fans] new lguest port available

[ron minnich]

this new lguest port pulls us forward to 2.6.25 with the new virtio
interface (ericvh git branch still I guess -- eric?)

works fine with thx9 images. I am booting them now.

Feels peppier but kernel build still slow -- 90 seconds.

if interested, let me know.

ron

Re: [9fans] new lguest port available

[Eric Van Hensbergen]

On Wed, Apr 23, 2008 at 1:01 PM, ron minnich <rminnich@gmail.com> wrote:
> this new lguest port pulls us forward to 2.6.25 with the new virtio
>  interface (ericvh git branch still I guess -- eric?)
>

That tree is slightly stale with respect to mainline -- you may want
to merge up to make sure.  Shouldn't break anything, but there may be
some subtle changes between when I branched and 2.6.25 final.

      -eric

Re: [9fans] new lguest port available

[Juan M. Mendez]

On 23/04/2008, ron minnich <rminnich@gmail.com> wrote:
> this new lguest port pulls us forward to 2.6.25 with the new virtio
>  interface (ericvh git branch still I guess -- eric?)
>
>  works fine with thx9 images. I am booting them now.
>
>  Feels peppier but kernel build still slow -- 90 seconds.
>
>  if interested, let me know.

I'm interested Ron, Any change to thnx to get it working with new lguest?

--
Fidonet: 2:345/432.2

Re: [9fans] new lguest port available

[Stefan Hajnoczi]

I am interested :).  Is there a URL or contrib path to get this stuff?

Thanks,
Stefan

Re: [9fans] new lguest port available

[ron minnich]

On Wed, Apr 23, 2008 at 12:21 PM, Juan M. Mendez <vejeta@gmail.com> wrote:

>  I'm interested Ron, Any change to thnx to get it working with new lguest?

have  not tested with full up thnx yet. I am going to pull down a
release 2.6.25 and make sure it still works with that.

contrib/rminnich/lguest/lg25.tgz
sample linux config file there too.
Also my current 9lguestcpu kernel, 9lguestcpu.2.6.25.elf


to make: tar file includes bin/rc/lguest, to set up paths.
I boot a qemu image and bulid from there, but
cd
tar xvf lg25.tgz
mkdir -p src/ip src/boot src/port src/pc
lguest
cd src/lguest25
mk

and away you go.

My current command line is this:
./Documentation/lguest/lguest    784 /tmp/9lguestcpu.elf
--tunnet=192.168.19.1  \
--block=/u3/thx9.img  \
'venti=#S/sd00/arenas;bootdisk=local!#S/sd00/fossil;bootargs=local!#S/sd00/fossil'


I welcome comments on improving the code. To some extent I'm still
getting a handle on how to set this all up.

Disk IO is not great, net IO seems pretty good. I don't have a stable
timebase on lguest, evidently, or I would run netpipe to test.

ron

Re: [9fans] new lguest port available

[ron minnich]

oh yes, to install lguest, you MUST:

 modprobe lg syscall_vector=64

This sets the right syscall vector.

ron

Re: [9fans] new lguest port available

[erik quanstrom]

> Disk IO is not great, net IO seems pretty good. I don't have a stable
> timebase on lguest, evidently, or I would run netpipe to test.

just put it up on a tee: why not use aoe?

- erik

Re: [9fans] new lguest port available

[ron minnich]

On Wed, Apr 23, 2008 at 4:22 PM, erik quanstrom <quanstro@quanstro.net> wrote:

>  just put it up on a tee: why not use aoe?

I had not even thought of that. How do you recommend setting it up?

ron

Re: [9fans] new lguest port available

[erik quanstrom]

>>  just put it up on a tee: why not use aoe?
>
> I had not even thought of that. How do you recommend setting it up?
>
> ron

i probablly don't know enough about your setup to answer that well.
but here;s an idea nonetheless.

for a single-machine linux-hosted setup, you could run linux vblade
(freshmeat.net/projects/vblade) on the host.  you can use the
instructions in sdaoe(3) to setup, say /dev/sde0 and dd your virtual
hard drive to the aoe storage.  sdaoe provides you with a standard
devsd interface to aoe storage so fdisk, prep and friends will not
know the difference.  there is also a quick example of the plan9.ini
variables
needed to boot directly to aoe:
	aoeif=ether0 ether1
	aoedev=e!#æ/aoe/42.0	# shelf.slot
i suppose that instead of changing your fossil/venti drive configuration
drive letters, you could disable the disk emulation in the guest and tell
sdaoe to take over that letter.  but i've never tried that, so it might not
work for reasons i'm refusing to remember.

(if the disk image is just a plan file on the linux side, linux vblade will
be happy to serve it as-is with no modifications required.)

while on the topic, i should mention that there is a plan 9 vblade
/n/sources/contrib/quanstro/src/vblade.  but really, go buy coraid stuff.
forget this vblade nonsense! ☺

- erik

Re: [9fans] new lguest port available

[Eric Van Hensbergen]

On Wed, Apr 23, 2008 at 6:22 PM, erik quanstrom <quanstro@quanstro.net> wrote:
> > Disk IO is not great, net IO seems pretty good. I don't have a stable
>  > timebase on lguest, evidently, or I would run netpipe to test.
>
>  just put it up on a tee: why not use aoe?
>

The problems of disk I/O are largely a focus issue -- all this stuff
is pretty new and they focused on the network mechanisms first because
those were the ones where the competition has published the most
compelling benchmarks.  The disk stuff will get tuned out and will
likely outperform network for I/O.  As an example, 9P directly over
virtio beats NFS/TCP/virtio-net by 70% without cacheing or
optimization in 9P (which is usually the opposite case on
unvirtualized hardware due to cacheing and what not).

       -eric

Re: [9fans] new lguest port available

[erik quanstrom]

>>  just put it up on a tee: why not use aoe?
>>
>
> The problems of disk I/O are largely a focus issue -- all this stuff
> is pretty new and they focused on the network mechanisms first because
> those were the ones where the competition has published the most
> compelling benchmarks.  The disk stuff will get tuned out and will
> likely outperform network for I/O.  As an example, 9P directly over
> virtio beats NFS/TCP/virtio-net by 70% without cacheing or
> optimization in 9P (which is usually the opposite case on
> unvirtualized hardware due to cacheing and what not).

i wouldn't think that you could tune out rotational latency.  8.4ms is
pretty much forever when you're counting nanoseconds.

since aoe can do wirespeed (120ms/s) on typical physical gige
chipsets i would think it would have no trouble keeping up with
spinning media.  especially when not handicapped by having to
actually stuff bits through a phy.

- erik

Re: [9fans] new lguest port available

[Eric Van Hensbergen]

On Wed, Apr 23, 2008 at 7:29 PM, erik quanstrom <quanstro@quanstro.net> wrote:
> >>  just put it up on a tee: why not use aoe?
>  >>
>  >
>  > The problems of disk I/O are largely a focus issue -- all this stuff
>  > is pretty new and they focused on the network mechanisms first because
>  > those were the ones where the competition has published the most
>  > compelling benchmarks.  The disk stuff will get tuned out and will
>  > likely outperform network for I/O.  As an example, 9P directly over
>  > virtio beats NFS/TCP/virtio-net by 70% without cacheing or
>  > optimization in 9P (which is usually the opposite case on
>  > unvirtualized hardware due to cacheing and what not).
>
>  i wouldn't think that you could tune out rotational latency.  8.4ms is
>  pretty much forever when you're counting nanoseconds.
>
>  since aoe can do wirespeed (120ms/s) on typical physical gige
>  chipsets i would think it would have no trouble keeping up with
>  spinning media.  especially when not handicapped by having to
>  actually stuff bits through a phy.
>

You on the wrong portion of the problem -- the disk solution they have
is effectively AOV (ATA over Virtio), you aren't going to do better by
putting a virtual network driver in between.  They just have to tune
their userspace gateway for disk access -- they put a lot of work into
making the virtio<->tun/tap gateway really efficient and I think they
are just using the crappy Qemu block device at the moment.  Once they
short-out the gateway between the guest-virtio channel and the
in-kernel block driver it'll be much faster than tunneling AOE over
the network device to the host.

Now - if you are talking about supporting an off-server CORAID storage
array -- then you should absolutely go AOE, but I think he was talking
about talking between guest and host partitions on his laptop in which
case you are adding extra layers for nothing.

         -eric

Re: [9fans] new lguest port available

[erik quanstrom]

> You on the wrong portion of the problem -- the disk solution they have
> is effectively AOV (ATA over Virtio), you aren't going to do better by
> putting a virtual network driver in between.  They just have to tune
> their userspace gateway for disk access -- they put a lot of work into
> making the virtio<->tun/tap gateway really efficient and I think they
> are just using the crappy Qemu block device at the moment.  Once they
> short-out the gateway between the guest-virtio channel and the
> in-kernel block driver it'll be much faster than tunneling AOE over
> the network device to the host.

what i'm saying is boils down to 10ms + 100ns  is essentially 10ms.
so it's slower, but at a level a couple (or three) orders of magnitude too
low to be very significant.

> [...] adding extra layers for nothing.

avoiding maintaing a second interface doesn't count?  and according
to ron, the network is fast right now.  this virtual ata interface isn't.

now i'm really dreaming but ... why don't you convince the virtualizer guys
to implement aoe instead of a straight ata interface for high performance.
it would be less work for them too, and would eliminate the extra layer --
vblade.

- erik

Re: [9fans] new lguest port available

[Eric Van Hensbergen]

On Wed, Apr 23, 2008 at 7:53 PM, erik quanstrom <quanstro@quanstro.net> wrote:
>
>  what i'm saying is boils down to 10ms + 100ns  is essentially 10ms.
>  so it's slower, but at a level a couple (or three) orders of magnitude too
>  low to be very significant.
>

ah, but its not always 10ms because of the page cache...but I'm
digressing to a new low level.  Actually,  the argument that Ron and I
both made is that they should ditch both the network and the block
(and the console) drivers/protocols and just use 9p for all three.  It
adds your slight 100ns of overhead, but unifies all God's children.
However, given that the virtio stuff is infinitely better than the Xen
approach and the other craptastic virtualization I/O schemes, we
decided to leave well enough alone and do the 9p stuff ourselves
(which begs the question why Ron is using the disk and net drivers and
not the superior 9p driver, but I digress yet again).

>  > [...] adding extra layers for nothing.
>
>  avoiding maintaing a second interface doesn't count?  and according
>  to ron, the network is fast right now.  this virtual ata interface isn't.
>

come back to his jebus and remember that the only real goal of all
virtualization people is to support windows -- they'll always have a
block device.  It will get faster, you can hear the millions of
monkey's typing in the distance.

>
>  now i'm really dreaming but ... why don't you convince the virtualizer guys
>  to implement aoe instead of a straight ata interface for high performance.
>  it would be less work for them too, and would eliminate the extra layer --
>  vblade.
>

I misled you a bit when I said it was AOV, its actually much closer to
just the straight block device.  No real naming issues and it
back-ends into a bunch of stuff they want like copy-on-write and what
not.  WAIT - I already hear you complaining about my lack of
consistency because I was talking about kernel-cut-thru's -- however,
the nature of the game is there are several back-ends for several
different solutions.  I like AOE, I think its a great general
solution, but I don't see it solving their problems and it complicates
some of their security, failover and migration schemes.  Its close to
the equivalent of saying why doesn't Linux use AOE as its block driver
interface.

          -eric

Re: [9fans] new lguest port available

[ron minnich]

On Wed, Apr 23, 2008 at 11:10 AM, Eric Van Hensbergen <ericvh@gmail.com> wrote:

>  That tree is slightly stale with respect to mainline -- you may want
>  to merge up to make sure.  Shouldn't break anything, but there may be
>  some subtle changes between when I branched and 2.6.25 final.
>

I just verified that the plan 9 guest works on 2.6.25 final, so no
issues there.

ron