[9fans] extending xen to allow driver development in Plan 9

[9fans] extending xen to allow driver development in Plan 9

[ron minnich]

This is mostly for Richard Miller but I don't have his email. But if
you are interested in Xen, read along.

We have an ok xen environment going. Why are we doing this? Per a
certain person at xyz.com, we are looking at giving people a usable
xen-based plan 9 environment, and at the same time letting them do
driver work from Plan 9 by "poking holes" in Xen to let Plan 9 at the
real hardware. Xen supports this, we think, although we have not got
it going yet ...

I already like the situation thus far, as Plan 9 under Xen is a ton
faster than Plan 9 under qemu. You have to see it to believe it; if
anything, the Xen advantage is better than it used to be. I was
surprised.

to get to the point of poking holes in Xen, it turned out I need
pcifront. For pcifront I need xenbus. for xenbus I need xenstore.

There is xenstore support in Plan 9 already, but ...

The xenstore sez: "incomplete". What would it take to complete it?
conservative use of locks in the short term as a hack for really doing
it right in the long term? The comment is this:

 * XXX This is incomplete - needs multiplexing of request/response protocol
 * and locking between driver and kernel-only xenstore_read/write interface.

Should we set up queues for request/response? The locking seems simple
enough, is there something I'm missing?

ron

Re: [9fans] extending xen to allow driver development in Plan 9

[Richard Miller]

> Should we set up queues for request/response?

Not necessarily a queue because I don't think there's a guarantee that
responses come in the same order as requests.  Maybe a hash table of
requests awaiting responses?

> The locking seems simple
> enough, is there something I'm missing?

No, it should be simple.  I just hadn't got around to it yet.

Again: (self)hosted Plan9? Was: [9fans] extending xen to allow driver development in Plan 9

[Georg Lehner]

"ron minnich" <rminnich@gmail.com> writes:
...
> We have an ok xen environment going. Why are we doing this? Per a
> certain person at xyz.com, we are looking at giving people a usable
> xen-based plan 9 environment, and at the same time letting them do
> driver work from Plan 9 by "poking holes" in Xen to let Plan 9 at the
> real hardware. Xen supports this, we think, although we have not got
> it going yet ...
>
> I already like the situation thus far, as Plan 9 under Xen is a ton
> faster than Plan 9 under qemu. You have to see it to believe it; if
> anything, the Xen advantage is better than it used to be. I was
> surprised.
...

I have a similar situation:

- Xen helps me run several Plan9's one the same hardware

- I can give my users a Plan9 environment without taking away the OS
  they are used to work with

- Xen is much faster then Qemu, ok for production use

- as Richard Miller said: ".. the whole point of xen is that physical
  devices become Somebody Else's Problem."

However I think that the same goals could be achieved more natural,
even faster, more stable and more generally aplicable if Plan9 could
be run (self)hosted.

The Hurd can be run as a user space process inside The Hurd.  Made
feasable because of its multi-server nature: the Kernel almost does
not do I/O.  Thus The Hurd allegedly can be debugged and developed
more easily.

I guess the Plan9 Kernel could be separated in two layers, the upper
one just doing "high-level" and 9P-protocol stuff, and a lower one,
providing the #-channel interfaces to the upper layer and doing I/O.

The lower layer could either be comprised of hardware drivers for the
real hardware, or a hosting layer which intermediates between the
block devices and memory managment operations of a certain hosting
operating system and the #-channel interface to the upper layer.

Maybe this approach could also clean up the duplication of code
between 9loader and kernel I have read about in some Plan9 document.

Hardware driver development could also be eased by this approach,
since it is probably easier to pass certain hardware through to a
Linux process (the hosted Plan9 instance), than to go through the
complexities of Xen-Hypervisor - dom0 Linux - domU Plan9 interaction.

And: I know that this approach probably would increase complexity and
reduce performance with respect to the current Plan9 kernel.

Initially I have started to browse the Plan9 kernel source code, Linux
kernel docs, x86 assembler manuals etc., but I realized very fast,
that my spare time will never be sufficient to spot out all required
points to get anywhere with such a project.  However maybe there are
some folks out there who like the idea and have the knowledge to do
it.

Best Regards,

    Jorge-León

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow driver

[Lucio De Re]

> I guess the Plan9 Kernel could be separated in two layers, the upper
> one just doing "high-level" and 9P-protocol stuff, and a lower one,
> providing the #-channel interfaces to the upper layer and doing I/O.

My thoughts are along similar lines.  My approach would be to expose
the device drivers in a hardware dependent "BIOS" as Plan 9 devices,
adding only as much OS glue at this level as makes this possible.
This would be as close as damn what we know as 9load today.

The Plan 9 kernel is the only one that would be able to interface to
this BIOS currently, but over time it ought to be possible, mutatis
mutandis, to boot other OSes on this layer.  My dream is that the BIOS
could then be extended by the hordes of device driver writers and
every compliant OS would be able to use new devices immediately.

But I may be just dreaming.

++L

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow driver

[ron minnich]

On 12/6/06, Lucio De Re <lucio@proxima.alt.za> wrote:
> > I guess the Plan9 Kernel could be separated in two layers, the upper
> > one just doing "high-level" and 9P-protocol stuff, and a lower one,
> > providing the #-channel interfaces to the upper layer and doing I/O.
>
> My thoughts are along similar lines.  My approach would be to expose
> the device drivers in a hardware dependent "BIOS" as Plan 9 devices,
> adding only as much OS glue at this level as makes this possible.
> This would be as close as damn what we know as 9load today.
>
> The Plan 9 kernel is the only one that would be able to interface to
> this BIOS currently, but over time it ought to be possible, mutatis
> mutandis, to boot other OSes on this layer.  My dream is that the BIOS
> could then be extended by the hordes of device driver writers and
> every compliant OS would be able to use new devices immediately.

well, we're trying to sort of do that now. We're using linux as the
driver layer. It's not what you guys want implementation-wise, but it
is something  like the idea.

It's not that drivers are fundamentally hard. It's that the hardware
we work with is undocumented crap. Linux drivers know all the secrets;
we're riding on that knowledge.

ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Lucio De Re]

> It's not that drivers are fundamentally hard. It's that the hardware
> we work with is undocumented crap. Linux drivers know all the secrets;
> we're riding on that knowledge.

And so do millions more, perfectly understandably.  The problem is
that "all the world is Linux" is not a good mantra.  Porting drivers
from Linux to, say, NetBSD is a nightmare, multiplied by the number of
useful target OSes.  Whereas XEN largely suffers (only) from
inefficiencies accessing the lower layer, whenever it (and you, if I
understand your recent discussions) tries to punch through the
barrier, the mysteries strike again.

(In passing, I was looking at ISDN adapter drivers with a view to
implementing the functionality under NetBSD.  The Linux driver, in my
opinion, was orders of magnitude better coded than the FreeBSD
version.  Take that any way you like, it has changed my opinions on
Open Source device driver developers.)

To return to the main issue, I think effort applied towards
documenting "undocumented crap" would have a wider scope than adopting
or reverse engineering the knowledge in Linux drivers code.  The
latter is certainly a more immediate objective.  Of course, one then
also needs to deal with binary-only drivers and other such stumbling
blocks, but my hope would be that eventually hardware manufacturers
will get the message or will get deselected :-)

Given the choice between using Linux kernel source as the
documentation, versus Plan 9 kernel source, there are too many good
reasons to pick the Plan 9 option to list them here, where they are in
any case taken for granted.  Hence my preference for a 9load-type BIOS
on which others besides Plan 9 can build.

(The philosophy, probably flawed, is that Open Source principles are
"right" in some transcendent way and that a "good" manufacturer cannot
continue to overlook the benefits of being on the "right" side of the
line.  Communism had the same underlying principle and landed up on
the scrap-heap of history.)

++L

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

> To return to the main issue, I think effort applied towards
> documenting "undocumented crap" would have a wider scope than adopting
> or reverse engineering the knowledge in Linux drivers code.

I agree. We've been doing this kind of documentation for seven years
now. I would say it has gotten harder in the last four years. There is
ever more paranoia and unwillingness to open programming details up.
Just compare the Intel L440 manuals you could get in 1999 vs. what is
at developer.intel.com now. The L440 chipset manual was wonderful,
with full docs and a tutorial on SDRAM programming. The stuff there
now is basically useless for writing a BIOS. We have been told that is
intentional.

>  Of course, one then
> also needs to deal with binary-only drivers and other such stumbling
> blocks, but my hope would be that eventually hardware manufacturers
> will get the message or will get deselected :-)

It's not working so far. Most people want their games fast; they don't
care about whether the chipsets are documented.


>
> (The philosophy, probably flawed, is that Open Source principles are
> "right" in some transcendent way and that a "good" manufacturer cannot
> continue to overlook the benefits of being on the "right" side of the
> line.

It's an ongoing battle. Toss in DRM and things really get ugly. The
preferred vendor approach is to lock things up when we re not
watching. It's our job to keep watching.

ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Chad Dougherty]

yep.  right on the money.

Theo, who's been a champion for more open hardware documentation, has a 
really nice set of slides on this topic:

<http://www.openbsd.org/papers/opencon06-docs/index.html>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Steve Simon]

I am keen enough to try and get some hardware documentation, and maybe
even have a go at codeing a/some drivers for plan9.

I only use hardware which plan9 is compatible with so I don't know
which way to look. At IWP9 modern, inexpensive SATA cards where
mentioned as somwhere we had a gap in our coverage.

Is this all we need? Can anyone suggest a card that fits the bill?

how about modern laptops, we have a ⅞ finished centrino driver which
needs to be finished off, but what graphics chipset is common enough
to make it worthwhile chasing the manufacturer, is nvidia still king?
Do they have weird interrupt controllers, southbridges etc which will
cause problems?

As usual I promise nothing but I will do what I can.

-Steve

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Lucio De Re]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

I seem to have long range objectives: I believe that Plan 9 could
make a better virtualisation host than the current choice for the job
and that we ought to sell to the world that it is worth (a) adopting
the Plan 9 paradigm for hosted device drivers and (b) developing all
future physical device drivers to this paradigm.

Putting it in a different way, consider replacing GRUB with 9LOAD as
first step.  Then extend 9LOAD so that it somehow contains the correct
device drivers for the particular platform and use it to boot the next
stage which accesses device drivers Plan 9-style.  Now, if we can get
the hordes of device driver implementors to focus on this approach,
Plan 9's greatest shortcoming (and all other Open Source OSes's)
suddenly disappears.

Sure, there are efficiency considerations to take into account, as
well as the unlikelihood of persuading Lunix developers to switch
allegiances, but a concerted effort to head that way is certainly not
doomed before even starting, at least not unless I'm subconsciously
ignoring some obvious obstacle.

This idea has been bugging me ever since rminnich started making
noises about Xen and I was tempted to look at it.  It bothers me that
Xen is designed very much with Linux as its model, not just its
primary target.  My claim is that a design founded on a Plan 9
foundation would be almost guaranteed to be more successful.  That's
what I call "faith", I suppose.

++L

[-- Attachment #2: Type: message/rfc822, Size: 3621 bytes --]

From: "Steve Simon" <steve@quintile.net>
To: 9fans@cse.psu.edu
Subject: Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow
Date: Sat, 9 Dec 2006 11:21:55 +0000
Message-ID: <e1e1d3e3266ab5e656eabdae89aa3030@quintile.net>

I am keen enough to try and get some hardware documentation, and maybe
even have a go at codeing a/some drivers for plan9.

I only use hardware which plan9 is compatible with so I don't know
which way to look. At IWP9 modern, inexpensive SATA cards where
mentioned as somwhere we had a gap in our coverage.

Is this all we need? Can anyone suggest a card that fits the bill?

how about modern laptops, we have a ⅞ finished centrino driver which
needs to be finished off, but what graphics chipset is common enough
to make it worthwhile chasing the manufacturer, is nvidia still king?
Do they have weird interrupt controllers, southbridges etc which will
cause problems?

As usual I promise nothing but I will do what I can.

-Steve

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[erik quanstrom]

"Steve Simon" <steve@quintile.net> writes

| 
| I am keen enough to try and get some hardware documentation, and maybe
| even have a go at codeing a/some drivers for plan9.
| 
| I only use hardware which plan9 is compatible with so I don't know
| which way to look. At IWP9 modern, inexpensive SATA cards where
| mentioned as somwhere we had a gap in our coverage.

plan 9 does support the marvell chipset.  for example:
	http://www.supermicro.com/products/accessories/addon/AoC-SAT2-MV8.cfm
this is an inexpensive card.

as for on-board sata, plan 9 does support the sata on nforce430 
(and maybe most nforce) boards, but doesn't seem to support 
many modes of ich[5-] sata+pata hackery.  i think fixing that
would yield the most bang-for-the-buck.

personally, these things are also at the top of my list
- usb ohic and ehic.
- gbit ethernet: forcedeth, tg3.

- erik

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[geoff]

The SATA controllers I've encountered so far either emulate ordinary
(P)ATA controllers or have a BIOS setting to optionally do so.  I'm
not sure that there's much to be gained by implementing the AHCI or
other oddball interfaces.

As I mentioned at IWP9, I'm integrating Charles Forsyth's OHCI driver
into the devusb framework, and will probably end up having to write
EHCI (USB 2) support.  I've been working on getting Richard Miller's
usbsfs working with a dozen or so USB disk-like devices (MP3 players,
DVD drives, flash disks) and they are now mostly working, after adding
code to probe and use LUNs (logical unit numbers).  Some of the dumber
devices seem to be very sensitive to exactly how you talk to them and
go off into the weeds if they don't approve, so getting a version of
usbsfs that they can all talk to is taking longer than expected.

Gigabit Ethernet seems to be pretty well handled; we've got drivers
for the Intel controllers (though Intel keeps introducing new
not-quite compatible variations) and the Realtek 8169, though it
pauses and thus has low throughput on my machine.  Do the other
gigabit controllers appear on lots of motherboards?

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[andrey mirtchovski]

> Do the other gigabit controllers appear on lots of motherboards?

broadcom (tg3) used to be _the_ ethernet for opterons for quite some
time (we saw one board by iWill which had intels only this year for
the first time)...

all our opterons have an 8169 stuck in a pci slot...

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Charles Forsyth]

>>how about modern laptops, we have a ⅞ finished centrino driver which
>>needs to be finished off, but what graphics chipset is common enough

since i've got a modern laptop, i'm hoping to do an i3945 802.11abg (ok, bg) driver
over christmas, in front of the fire, in between mulled wine and hot toddies.
hmm... wireless and legless.  it's not tuneless, because i've got a good set of CDs.

the incomplete linux driver with binary blobs is only just over 16,000 lines, so how
hard can it be?  hmm.  the free/open bsd driver is only 2800 lines.
shome mishtake shurely.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

On 12/9/06, Steve Simon <steve@quintile.net> wrote:
> I am keen enough to try and get some hardware documentation, and maybe
> even have a go at codeing a/some drivers for plan9.

This is great. All these hardware driver ideas are fine.

But lacking a few other bits, i still believe that all the drivers in
the world are not going to be sufficient. I don't even thinks they are
necessary to get people going.

Put it this way. I have a nice web page served out of a Plan 9 system
that shows google maps data etc. I can't view it on Plan 9. What we
have here, is failure to communicate. Drivers won't help this problem.

So what we're trying to do is give people a path from the linux world
to a better place. The idea is that you'll get a linux kernel as a
device driver. You can even do driver development on Plan 9 in this
world if all works out. The linux kernel will simply ignore the
existence of a piece of hardware; Plan 9 will own it. Over time, more
and more bits can be moved to the Plan 9 domain, until at some point,
we realize that on that particular system, we don't need Linux at all
-- and it vanishes in a puff of smoke and mirrors.

This will be way less extensive than xen-knoppix, which is a full
distro more or less. We don't need X11 -- Aki has got drawterm working
on /dev/fb. Systems get pretty peppy once x11 is out of the picture.
We've realized that we need python to run xen, so one thing we might
do is remove all of gnubin and just have a few .py utililties, so as
to minimize the amount of junk we need to have on the linux dom0
partition. If you have python (NOT that I want it; but you can't avoid
Python and XML RPC in Xen, sigh), and you can import os, it's hard to
see why you want all the gnubin junk. How many implementations of cat
do you need, really?

For apps, you can run a linux app in a window under VNC. Aki has
plumbing working between the two worlds. So our browser for now is the
"light" firefox with its slim-waisted 200 MBYTE footprint.

Think of this stuff as analogous to the V5 JCL command, or the GECOS
field in /etc/passwd; a way to get by until you can cut the cords to
the old-style, primitive environment. And you only use that
environment in a batch mode ...

So, for those of you who don't wish to write drivers, there's still a
large number of us who would appreciate a web browser ... and abiword
or similar or ....

thanks

ron
p.s. Aki leaves in a week, and we're all very sad here. He was on a
training assigment, and unfortunately did not train us as much as we
had hoped. But, hope springs ever anew  ... any students wishing to
spend some time here training us are most welcome to! Note that it's
easier for me to set up if you are a .us, but hey, Aki is from Finland
and we still managed it, even though he doesn't speak Linux.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Francisco J Ballesteros]

16k lines ????
Really?

On 12/10/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> the incomplete linux driver with binary blobs is only just over 16,000 lines, so how
> hard can it be?  hmm.  the free/open bsd driver is only 2800 lines.
> shome mishtake shurely.
>
>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Charles Forsyth]

[-- Attachment #1: Type: text/plain, Size: 92 bytes --]

h% wc -l *.c
  16731 ipw3945.c

and don't forget the binary user-mode supervisory program

[-- Attachment #2: Type: message/rfc822, Size: 3397 bytes --]

From: "Francisco J Ballesteros" <nemo@lsub.org>
To: "Fans of the OS Plan 9 from Bell Labs" <9fans@cse.psu.edu>
Subject: Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow
Date: Sun, 10 Dec 2006 21:56:45 +0100
Message-ID: <8ccc8ba40612101256o718f4ff1ld607351d115b772e@mail.gmail.com>

16k lines ????
Really?

On 12/10/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> the incomplete linux driver with binary blobs is only just over 16,000 lines, so how
> hard can it be?  hmm.  the free/open bsd driver is only 2800 lines.
> shome mishtake shurely.
>
>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Latchesar Ionkov]

What does Java ME mean these days? Is it Java KVM, or Java CVM?  
Porting the first one to Plan9 is easy, I have the CVM code (sans  
graphics and hotspot) ported too. If it is released as open source  
and someone wants to finish the port, I can try to find what I have.

Thanks,
	Lucho

On Dec 13, 2006, at 12:02 PM, Matt wrote:

> There's a bit of a long shot option.
>
> Java ME is now open source.
> http://community.java.net/mobileandembedded/
>
> So, in theory, it could be ported to plan9.
>
> Why would anyone do that ?
>
> Because the Opera Mini Web browser runs in Java ME environments.
>
> http://www.operamini.com/
>
>
> There's quite a bit of software written for J2ME
>
>
> There's also Opera for devices, which might be worth a better look at.
>
> http://www.opera.com/products/devices/
>
>
> I know it would be a horrible thing to swallow, but there may be  
> something there.
>
> matt

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Matt]

There's a bit of a long shot option.

Java ME is now open source.
http://community.java.net/mobileandembedded/

So, in theory, it could be ported to plan9.

Why would anyone do that ?

Because the Opera Mini Web browser runs in Java ME environments.

http://www.operamini.com/


There's quite a bit of software written for J2ME


There's also Opera for devices, which might be worth a better look at.

http://www.opera.com/products/devices/


I know it would be a horrible thing to swallow, but there may be 
something there.

matt

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

On 12/12/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> ...  buying on the internet).
> having to reboot to get these things has the advantage
> that, like undertaking a long journey, i ask myself ``is my journey really necessary?''.
> no, more often than not, so i get back to writing software.

unless it's that present for your wife. Then, it might be life-saving.

ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Bakul Shah]

> Hence the approach we are taking with xen. You get a linux, you get a
> Plan 9, you get holes torn in the I/O and memory spaces from Plan 9 to
> hardware to let you doink hardware and write drivers under Plan 9,
> and, with any luck, crash the machine at will.

Have your considered inverting this setup?  Rather than a
native Linux and a parasitic plan9, have a native plan9 hand
over io and memory space it doesn't understand to the
parasite.  I'd rather have a very lean, clean and thin native
os (AKA hypervisor). Of course I have no idea if this can be
made to work....

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Charles Forsyth]

>>This may solve the no-firefox-and-mplayer-for-plan9 problem, but I  
>>don't see how it solves the no-plan9-drivers-for-my-laptop one.

well no: they must be written, but they are much easier than writing a browser,
even one as annoying as firefox.

when googling and fetching papers or software (my usual
use of the web), i typically get by with charon and hget.  for fancy stuff
i more often than not now manage with firefox, but i still find a few things that
demand windows.  in either case, it's either displacement activity
(ie, time wasting) or costly (buying on the internet).
having to reboot to get these things has the advantage
that, like undertaking a long journey, i ask myself ``is my journey really necessary?''.
no, more often than not, so i get back to writing software.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Scott Schwartz]

On Tue, Dec 12, 2006 at 09:30:35PM +1100, Bruce Ellis wrote:
> i thought that i could get anything when working at the labs.
> oh no.

Wasn't there one time when there was problem getting drivers for *lucent*
wavelan pcmcia cards?  (Maybe I misremember, but the story is better
that way.)

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

On 12/12/06, Latchesar Ionkov <lionkov@lanl.gov> wrote:
> This may solve the no-firefox-and-mplayer-for-plan9 problem, but I
> don't see how it solves the no-plan9-drivers-for-my-laptop one.

Hence the approach we are taking with xen. You get a linux, you get a
Plan 9, you get holes torn in the I/O and memory spaces from Plan 9 to
hardware to let you doink hardware and write drivers under Plan 9,
and, with any luck, crash the machine at will.

I think it's going to work out. At least the crashing part.

Aki is already regularly crashing linux from user mode, so how hard can it be?

ron

p.s. with the new web 2.0 in my firefox browser, with 64M resident out
of 136M, I am seeing that the mouse "sticks" and as I move the mouse
random shit gets highlighted and erased. Fun.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Latchesar Ionkov]

This may solve the no-firefox-and-mplayer-for-plan9 problem, but I  
don't see how it solves the no-plan9-drivers-for-my-laptop one.

	Lucho

On Dec 12, 2006, at 3:01 PM, ron minnich wrote:

> On 12/12/06, Eric Van Hensbergen <ericvh@gmail.com> wrote:
>
>> It'd be sweet to have something I could power off of USB 2.0 or
>> battery, with a hard drive and wireless (and maybe a serial port for
>> jmk).
>
> yeah, this was one of the ideas that came up before we started on Xen
> again, but Aki and Andrey and Lucho beat me up on this idea. It came
> down to the EC on one side, and me on the other, and that ended it.
>
> I suggested running linux on a little 1-5W board, and using it to run
> the linux apps, using a root mount from Plan 9. So Linux is this dumb
> little headless box you only turn on when you want, and otherwise you
> tell it to go away by yanking its power cord, verily.
>
> They thought the idea, uh, lacked merit. (I think they said it sucked,
> but am not sure).
>
> I think one reason the idea may really suck is that Firefox (the "thin
> client") requires a 200 MB footprint, which translates to gobs of
> Watts. Figures. Web 2.0!
>
> [[BTW, anybody but me enjoying the idea of taking an opteron out of
> socket and replacing with ... an ... XML ... accelerator?]]
>
> But I still like the 'stupid little linux CPU' idea. I want a backpack
> full of little computers that spin up on demand. And don't weigh much.
> and take no power. And have no moving parts. And generate no heat. And
> use a fusion reactor for power. And, to reduce weight, have
> antigravity pods. I guess I'll go visit Area 52 this weekend (Area 51
> is always behind schedule and over budget).
>
> ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

On 12/12/06, Eric Van Hensbergen <ericvh@gmail.com> wrote:

> It'd be sweet to have something I could power off of USB 2.0 or
> battery, with a hard drive and wireless (and maybe a serial port for
> jmk).

yeah, this was one of the ideas that came up before we started on Xen
again, but Aki and Andrey and Lucho beat me up on this idea. It came
down to the EC on one side, and me on the other, and that ended it.

I suggested running linux on a little 1-5W board, and using it to run
the linux apps, using a root mount from Plan 9. So Linux is this dumb
little headless box you only turn on when you want, and otherwise you
tell it to go away by yanking its power cord, verily.

They thought the idea, uh, lacked merit. (I think they said it sucked,
but am not sure).

I think one reason the idea may really suck is that Firefox (the "thin
client") requires a 200 MB footprint, which translates to gobs of
Watts. Figures. Web 2.0!

[[BTW, anybody but me enjoying the idea of taking an opteron out of
socket and replacing with ... an ... XML ... accelerator?]]

But I still like the 'stupid little linux CPU' idea. I want a backpack
full of little computers that spin up on demand. And don't weigh much.
and take no power. And have no moving parts. And generate no heat. And
use a fusion reactor for power. And, to reduce weight, have
antigravity pods. I guess I'll go visit Area 52 this weekend (Area 51
is always behind schedule and over budget).

ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[erik quanstrom]

does the gumstix come with enough hardware documentation?  perhaps
i missed it.  but i didn't see the info on how one boots these things.

- erik

On Tue Dec 12 10:20:29 EST 2006, ericvh@gmail.com wrote:
> It'd be sweet to have something I could power off of USB 2.0 or
> battery, with a hard drive and wireless (and maybe a serial port for
> jmk).  Gumstick seems like it comes close - but no real solution for
> portable or piggy-back power.  I suppose an iPaq might be able to be
> tasked to such a solution as well -- but has less than ideal disk
> storage.  Neither has particularly glorious CPU power or memory --
> maybe we can build something with the OLPC mother boards sans
> screen/keyboard.
> 
>      -eric

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Eric Van Hensbergen]

On 12/12/06, ron minnich <rminnich@gmail.com> wrote:
> On 12/12/06, Bruce Ellis <bruce.ellis@gmail.com> wrote:
> > Browser, buy a cheap PC and run whatever you like on it.
> > Wow that solves everything.  That was a good session.
> >
> > brucee
>
> sucks for laptops though. I hate carrying all them thar laptops --
> they get in the way of my shootin' iron.
>

Just need to put your experience of building small systems towards
building a "headless" laptop-server -- then you can use drawterm from
your "browser" laptop - or home desktop, or whatever.

It'd be sweet to have something I could power off of USB 2.0 or
battery, with a hard drive and wireless (and maybe a serial port for
jmk).  Gumstick seems like it comes close - but no real solution for
portable or piggy-back power.  I suppose an iPaq might be able to be
tasked to such a solution as well -- but has less than ideal disk
storage.  Neither has particularly glorious CPU power or memory --
maybe we can build something with the OLPC mother boards sans
screen/keyboard.

     -eric

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[ron minnich]

On 12/12/06, Bruce Ellis <bruce.ellis@gmail.com> wrote:
> Browser, buy a cheap PC and run whatever you like on it.
> Wow that solves everything.  That was a good session.
>
> brucee

sucks for laptops though. I hate carrying all them thar laptops --
they get in the way of my shootin' iron.

ron

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Gabriel Diaz]

hello


> Some companies, LIKE BROADCOM!, won't sell part to Coraid because
> <mocking high pitched voice> we're too small </mocking high pitched voice>.
> No docs, no support, hope they fall into a large hole.

amen :)

seems that the conclusion is what i 'suspected', reading the source
and actually writting drivers is the only way to start

thanks

gabi

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Brantley Coile]

>> there is essentially no documention on many, many boards
>> and chips "just in case we want to rip off their IP'.
> 
> You're probably right.  The only explanation is that these suckers
> actually know that their product can easily be improved upon.
> 
> ++L

Another thing I've ran into was the small companies that
thought it was easier to write drivers themselves than to
write good documentation on their parts.  The driver author
just has to shout over the cube wall.  They have to write
the drivers anyway.

Intel seems to be schizophrenic about it.  Some docs we can get
with a NDA and some are on the web.  Not clear why.

A recent driver was done with no docs but good support from the
company.

Some companies, LIKE BROADCOM!, won't sell part to Coraid because
<mocking high pitched voice> we're too small </mocking high pitched voice>.
No docs, no support, hope they fall into a large hole.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Brantley Coile]

> Browser, buy a cheap PC and run whatever you like on it.

It was few years ago when I realized that machines were cheap, take two.
My secondary is a Mac.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Brantley Coile]

>>  - one significant place where Plan 9 wins is using it as a versatile
>>    base for building pieces that people use without knowing it's Plan 9
>>    (e.g., Sape's wireless base stations, Rangboom, xcpu, and
>>    many Inferno apps that Charles can't talk about).
> 
> I think we all want a bit more glory than that, but if we all convince
> ourselves, we can tuck Plan 9 entirely out of sight.  Problem is, we
> then don't get a community any more and Plan 9 does not have a
> profit-making organisation that can support it without the community.
> Or am I missing something?

It doesn't always have to be completely out of sight.  Good embedded
applications are free from the usual, `but everyone else is using windows'
arguments.  Just a minor point.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Brantley Coile]

>  - one significant place where Plan 9 wins is using it as a versatile
>    base for building pieces that people use without knowing it's Plan 9
>    (e.g., Sape's wireless base stations, Rangboom, xcpu, and
>    many Inferno apps that Charles can't talk about).

Coraid SR1520 and SR420 use Plan 9.  Depending on how many
systems are running Plan 9 out there, we might have more
kernels running than anyone else.  And we keep shipping.

Coraid will be using embedded Plan 9 for years to come.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Charles Forsyth]

> I mean, having no experience with hardware programming, a desire i
> have is to read something to learn from other's experience on writing
> software for manage hardware. (something like the practice of
> programming but focused on hardware issues).

possibly a good way is to read existing Plan 9 drivers

it isn't really a deep mystery, except for some of the peculiar
interfaces on the x86.  i usually blunder my way past them, myself.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Bruce Ellis]

i thought that i could get anything when working at the labs.
oh no.  one contact slipped the reason, your company
makes silicion. we got some good stuff about a chip, but
the conditions included keeping the magic book in a locked
drawer.  no problem - didn't say anything about keeping the key
in the lock.

brucee

On 12/12/06, Lucio De Re <lucio@proxima.alt.za> wrote:
> > there is essentially no documention on many, many boards
> > and chips "just in case we want to rip off their IP'.
>
> You're probably right.  The only explanation is that these suckers
> actually know that their product can easily be improved upon.
>
> ++L

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Lucio De Re]

> there is essentially no documention on many, many boards
> and chips "just in case we want to rip off their IP'.

You're probably right.  The only explanation is that these suckers
actually know that their product can easily be improved upon.

++L

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Bruce Ellis]

well while i'm commenting randomly ...

none.

there is essentially no documention on many, many boards
and chips "just in case we want to rip off their IP'.

correct me if i'm wrong - i have stood corrected.

brucee

On 12/12/06, Gabriel Diaz <gabidiaz@gmail.com> wrote:
> Hello
>
> what books you guys recommend to start with hardware programming?
> (nemo's kernel book of course)
>
> I mean, having no experience with hardware programming, a desire i
> have is to read something to learn from other's experience on writing
> software for manage hardware. (something like the practice of
> programming but focused on hardware issues).
>
> of course i can always re-read my school notes, and start to fight
> with the real life. . . but this looks discouraging, (and becomes much
> more discouraging taking in account the comments of more talented
> programmers on the iwp9 :)
>
> thanks
>
> gabi
>
>
> On 12/12/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> > >> - writing drivers sucks.
> >
> > it's not a big problem in itself.  i quite enjoy it for
> > the reasonably well-documented chipsets one finds in (say)
> > embedded ARM and PowerPC platforms.  for those, i hardly ever
> > bother to look at another driver.  it's just so straightforward.
> > i look at the book and do what it says.  it doesn't work, so i
> > find there's an errrata or fuss about discovering that a bit
> > has the opposite sense from what's documented.  no matter.
> >
> > on the PC, it's rather more troublesome: when i could get
> > reasonable documentation it was much the same as anything else.
> > without it, it's tedious, and perhaps too time-consuming
> > if i'm doing it in my spare time.  theo de raadt's slides
> > were quite a good summary.
> >
> > still, there's not much choice, really.
> >
>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Gabriel Diaz]

Hello

what books you guys recommend to start with hardware programming?
(nemo's kernel book of course)

I mean, having no experience with hardware programming, a desire i
have is to read something to learn from other's experience on writing
software for manage hardware. (something like the practice of
programming but focused on hardware issues).

of course i can always re-read my school notes, and start to fight
with the real life. . . but this looks discouraging, (and becomes much
more discouraging taking in account the comments of more talented
programmers on the iwp9 :)

thanks

gabi


On 12/12/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> >> - writing drivers sucks.
>
> it's not a big problem in itself.  i quite enjoy it for
> the reasonably well-documented chipsets one finds in (say)
> embedded ARM and PowerPC platforms.  for those, i hardly ever
> bother to look at another driver.  it's just so straightforward.
> i look at the book and do what it says.  it doesn't work, so i
> find there's an errrata or fuss about discovering that a bit
> has the opposite sense from what's documented.  no matter.
>
> on the PC, it's rather more troublesome: when i could get
> reasonable documentation it was much the same as anything else.
> without it, it's tedious, and perhaps too time-consuming
> if i'm doing it in my spare time.  theo de raadt's slides
> were quite a good summary.
>
> still, there's not much choice, really.
>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Bruce Ellis]

Browser, buy a cheap PC and run whatever you like on it.
Wow that solves everything.  That was a good session.

brucee

On 12/12/06, Charles Forsyth <forsyth@terzarima.net> wrote:
> >> - writing drivers sucks.
>
> it's not a big problem in itself.  i quite enjoy it for
> the reasonably well-documented chipsets one finds in (say)
> embedded ARM and PowerPC platforms.  for those, i hardly ever
> bother to look at another driver.  it's just so straightforward.
> i look at the book and do what it says.  it doesn't work, so i
> find there's an errrata or fuss about discovering that a bit
> has the opposite sense from what's documented.  no matter.
>
> on the PC, it's rather more troublesome: when i could get
> reasonable documentation it was much the same as anything else.
> without it, it's tedious, and perhaps too time-consuming
> if i'm doing it in my spare time.  theo de raadt's slides
> were quite a good summary.
>
> still, there's not much choice, really.
>

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Charles Forsyth]

>> - writing drivers sucks.

it's not a big problem in itself.  i quite enjoy it for
the reasonably well-documented chipsets one finds in (say)
embedded ARM and PowerPC platforms.  for those, i hardly ever
bother to look at another driver.  it's just so straightforward.
i look at the book and do what it says.  it doesn't work, so i
find there's an errrata or fuss about discovering that a bit
has the opposite sense from what's documented.  no matter.

on the PC, it's rather more troublesome: when i could get
reasonable documentation it was much the same as anything else.
without it, it's tedious, and perhaps too time-consuming
if i'm doing it in my spare time.  theo de raadt's slides
were quite a good summary.

still, there's not much choice, really.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Lucio De Re]

>  - writing drivers sucks.

There are two components: the hardware and the primitives in the
operating system or, preferably, in the programming language.
PC-hardware sucks, but in practice Xen, VMware and their ilk are all
trying to provide an abstraction that doesn't.  Is it really true that
Plan 9 doesn't have anything additional to offer?

>  - copying Linux and Windows will accomplish very little.

To agree violently, it makes Plan 9 like Linux and Windows and that is
precisely not what I, at least, like in Plan 9.  I have Linux and
Windows, why would I want Plan 9 to resemble them?

>  - one significant place where Plan 9 wins is using it as a versatile
>    base for building pieces that people use without knowing it's Plan 9
>    (e.g., Sape's wireless base stations, Rangboom, xcpu, and
>    many Inferno apps that Charles can't talk about).

I think we all want a bit more glory than that, but if we all convince
ourselves, we can tuck Plan 9 entirely out of sight.  Problem is, we
then don't get a community any more and Plan 9 does not have a
profit-making organisation that can support it without the community.
Or am I missing something?

>  - there may be real value in finding a way to use Xen or other
>    virtualization technologies to run Plan 9 on machines (for example,
>    terminals) where you care more about the convenience of having
>    Plan 9 than about the performance (or reliability!) of having it in
>    control of the hardware.
> 
That's hard but feasible.  It's just that it becomes a means to an end
instead of being an objective in itself.  It seems to me that that
leads to stagnation at the core of Plan 9 where its strength ought to
lie.  Applications may make or break Plan 9, but principles are its
meat, in my opinion.  Of course I'm an ignorant but keen hardware
hack, so I guess I'm not an authoritative voice here.

> And perhaps most important of all:
> 
>  - remember to keep it fun!

I can see why that should be, but I'm afraid it's hard to buy.  Either
that, or your idea of fun and mine are poles apart.  Now, if Plan 9
could make writing drivers easily, that would be fun :-)

++L

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Russ Cox]

We talked at length about this issue at IWP9 without a lot of consensus.
However, I think that many of us agreed on these points:

 - writing drivers sucks.
 - copying Linux and Windows will accomplish very little.
 - one significant place where Plan 9 wins is using it as a versatile
   base for building pieces that people use without knowing it's Plan 9
   (e.g., Sape's wireless base stations, Rangboom, xcpu, and
   many Inferno apps that Charles can't talk about).
 - there may be real value in finding a way to use Xen or other
   virtualization technologies to run Plan 9 on machines (for example,
   terminals) where you care more about the convenience of having
   Plan 9 than about the performance (or reliability!) of having it in
   control of the hardware.

And perhaps most important of all:

 - remember to keep it fun!

I can't deny the utility of having Firefox (I'm writing this in a
Firefox window), but even if Plan 9 could run Firefox, the next
thing would be oh but it needs to be able to run these ten
plugins, and so on and so on.  Personally, I think you are going
to be much happier running Plan 9 in some VM environment on
Linux or Windows than putting in the effort for the other way around.

Russ

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[Dave Eckhardt]

> so suppose we have javascript and all that jazz working on plan 9,
> would all that goo have vitiated the reason we were drawn to plan 9
> in the first place?

Because I need to share bits with people who use MS Office, I
need to run OpenOffice roughly daily.  But not all day, so
a combination of VNC to a FreeBSD machine and an emulated Linux
running under Plan 9 for travel would cut it.

I need to run a web browser pretty much all the time, though,
and it's hard to say when I'll need something that renders
actual web pages or some horrible JavaScript thing dreamt up
by HR.  So for me I think the barrier to booting Plan 9 on
my laptop every day (assuming for the moment no ACPI) would
be the lack of a Firefox-class browser.  But maybe the best
way to get one would be a stripped-down BSD release running
in a Plan 9 dom0.  At least, the FreeBSD guys have done pretty
well running random Linux binaries...

Dave Eckhardt

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[erik quanstrom]

honestly, i think that linux is the better place, if what you want
is something to view ajax google mashup stuff.  i don't find it strange
at all that the server can't display it's own data.  hard drives can't
initialiate ata commands, after all.

it's not too suprising that no 9 fan has written a javascript-compatable
browser.  i'd hope they be smart enough to always find something better
to do. ;-)

you know in 1992 or so when i first started using linux (couldn't get my
hands on a plan 9 license), it wasn't very functional.  but it was
tractable.  you could get stuff done.

now linux is pretty functional, i guess, but it is very difficult environment to
program in, and what works is very likely broken in the next release
because somebody thought there were too many bytes in struct work_struct
on 64-bit machines.

so suppose we have javascript and all that jazz working on plan 9,
would all that goo have vitiated the reason we were drawn to plan 9 in the 
first place?

- erik

On Sun Dec 10 15:58:35 EST 2006, rminnich@gmail.com wrote:
> Put it this way. I have a nice web page served out of a Plan 9 system
> that shows google maps data etc. I can't view it on Plan 9. What we
> have here, is failure to communicate. Drivers won't help this problem.
> 
> So what we're trying to do is give people a path from the linux world
> to a better place. The idea is that you'll get a linux kernel as a
> device driver. [...]

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[erik quanstrom]

no, it's not a mistake.  it's talent.

- erik

On Sun Dec 10 15:17:31 EST 2006, forsyth@terzarima.net wrote:
> >>how about modern laptops, we have a ⅞ finished centrino driver which
> >>needs to be finished off, but what graphics chipset is common enough
> 
> since i've got a modern laptop, i'm hoping to do an i3945 802.11abg (ok, bg) driver
> over christmas, in front of the fire, in between mulled wine and hot toddies.
> hmm... wireless and legless.  it's not tuneless, because i've got a good set of CDs.
> 
> the incomplete linux driver with binary blobs is only just over 16,000 lines, so how
> hard can it be?  hmm.  the free/open bsd driver is only 2800 lines.
> shome mishtake shurely.

Re: Again: (self)hosted Plan9? Was: [9fans] extending xen to allow

[erik quanstrom]

the vast majority of motherboards with the nforce
chipset have forcedeth gbe.

- erik

On Sun Dec 10 00:05:38 EST 2006, mirtchovski@gmail.com wrote:
> > Do the other gigabit controllers appear on lots of motherboards?
> 
> broadcom (tg3) used to be _the_ ethernet for opterons for quite some
> time (we saw one board by iWill which had intels only this year for
> the first time)...
> 
> all our opterons have an 8169 stuck in a pci slot...