[9fans] realemu update

[9fans] realemu update

[cinap_lenrek]

for everyone running realemu, please try with the new version to
see if i broke something.

the changes include:

switch back to textmode works now, as we use #P/realmodemem to
wite back changes to physical memory in the range B0000-AFFFF.

with realemu and the patch (see below), i'm able to switch to every resolution
in vmware that is suggested by the vmware vesa bios. so for everyone
who had problems with vmware graphics might give it a try.

added some i/o port emulation for pit, and some other well known ports. it
will trap now if port i/o fails.

theres a loadcom.c program that can be used to load a dos com file
into the vm and execute it (primary used for testing).  the filesystem
handles Tflush now so one can interrupt calls to /dev/realmode.

there was a bug in aux/vga causing it to produce a wrong color channel
descriptor for 32 bit rgb vesa modes, as the vesa bios doesnt report the
unused color component in the mask/pos field in modeinfo call.

see /n/sources/patch/vesa-depth32

--
cinap

Re: [9fans] realemu update

[Lucio De Re]

> for everyone running realemu, please try with the new version to
> see if i broke something.

Why TARG=qi in the mkfile (I would have guessed 8i, but I could be
wrong)? and I think the error:

	term% mk install
	cp 8.out $BIN/qi
	rc: null list in concatenation
	mk: cp 8.out $BIN/qi  : exit status=rc 9367: error

comes from not defining BIN?

Also, there's

	usgae:	./8.out [-Dpt] [-s srvname] [-m mountpoint]

;-) but I'll gladly submit a man page if you throw a couple of
guidelines in my direction.

That said, I haven't tried it yet.  I'm a little concerned about
aux/vga being altered to expect the emulation in /dev/realmode* and
what it may do if it's not there, which probably merely shows how
limited my understanding is.

I would like to see this type of change find its way into the
distribution permanently, I'll certainly help if I can.

++L

Re: [9fans] realemu update

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]

yes, this started as 8i from russ.  it was renamed to realemu after i
made it into a fileserver providing /dev/realmode and
/dev/realmodemem.  this way i dont have to reimplement aux/vga's vesa
code and it can be used as drop in replacement. (just start it before
running aux/vga)

mkfile is fixed now. will install itself into /$objtype/bin/aux/realemu
and install realemu (8) manpage.

impovement on the manpage is welcome as my english is not so good :)

aux/vga doesnt need to be changed for realemu.  the thing that needs
change is the kernel vgavesa driver to use the file /dev/realmode
instead file instead of calling realmode() directly.  this works with both, realemu
and the original #P/realmode file.  so it does not break anything.

the patch:

/n/sources/patch/vesa-softscreen-resize

also fixes some some softscreen related mode switching bugs in devvga.c and
screen.c that are unrelated to realemu. (it would crash with #P/realmode too)

i forgot to submit the patch to allow writing the vga framebuffer
(0xA0000-0xBFFFF) with #P/realmodemem:

/n/sources/patch/realmodemem-vgamem

this makes switch switch back to textmode work properly in realemu.

/n/sources/patch/vesa-depth32

still, aux/vga does the channel string generation wrong for 32 bit depth
modes on vesa, but this is a separate patch and has no connection to realemu.

/n/sources/patch/devdraw-initscreenimage-from-isr

fixes kernel panic.

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 3361 bytes --]

From: Lucio De Re <lucio@proxima.alt.za>
To: 9fans@9fans.net
Subject: Re: [9fans] realemu update
Date: Sun, 6 Mar 2011 09:35:00 +0200
Message-ID: <a3c9b4d5c5119c1f3a4a8d9debb28b08@proxima.alt.za>

> for everyone running realemu, please try with the new version to
> see if i broke something.

Why TARG=qi in the mkfile (I would have guessed 8i, but I could be
wrong)? and I think the error:

	term% mk install
	cp 8.out $BIN/qi
	rc: null list in concatenation
	mk: cp 8.out $BIN/qi  : exit status=rc 9367: error

comes from not defining BIN?

Also, there's

	usgae:	./8.out [-Dpt] [-s srvname] [-m mountpoint]

;-) but I'll gladly submit a man page if you throw a couple of
guidelines in my direction.

That said, I haven't tried it yet.  I'm a little concerned about
aux/vga being altered to expect the emulation in /dev/realmode* and
what it may do if it's not there, which probably merely shows how
limited my understanding is.

I would like to see this type of change find its way into the
distribution permanently, I'll certainly help if I can.

++L

Re: [9fans] realemu update

[Lucio De Re]

> mkfile is fixed now. will install itself into /$objtype/bin/aux/realemu
> and install realemu (8) manpage.
>
> impovement on the manpage is welcome as my english is not so good :)

Again, I'm delaying testing because I need to install a different
video card in the available hardware, but I'm looking forward to doing
that.

I have installed the pertinent bits in preparation, we'll see what
happens.

Thanks for tidying up.  On my side what I'm still missing is a
description of #P/realmode and #P/realmodemem; realemu's manpage
suggests that these can be found in arch(3) but my copy of that man
page has nothing to say about it (not your problem, of course).

The realemu(8) man page is pretty adequate now, thank you for that,
too.

Would it make sense, in your opinion, to backport these changes to 8i
and have that added to the distribution as well?

++L

Re: [9fans] realemu update

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 1947 bytes --]

the /dev/realmode intraface was not documented, but it is very simple.

/dev/realmodemem is just an image of the first megabyte of
physical memory that is addressable from 16 bit realmode.

plan9 reserves a 4k page at 0x9000 (defined as RMBUF) that can be
refered to in the bios call as data buffer.  previously, this was the
only offset range that could be written with /dev/realmodemem.

in /dev/realmode, you write a struct Ureg (from /386/include/ureg.h)
(in x86 machine byte order?) containing the register contents and the
interrupt number of the bios call you want to make.

the write returns when the BIOS call returns and the machine
state can be read back from /dev/realmode.

realemu did a little extension to the interface: it allows reading and
writing the whole address space and in case the trap is zero in the
Ureg, it will copy ss, sp, cs, and pc in the virtual cpu state too and not
make a BIOS interrupt.  this is used by loadcom.c to run dos .com
files in the emulator.

8i was never in a working or finished state...  it was able to execute
the modeinfo vbe calls on my t23 (after changing the KZERO offset) but
that was about it.  there are not many similarities with 8i anymore.
i worked on 8i for a while...  debugging 8i gave me a good overview of
whats needed for 80x86 interpretation so after some time i started
over from scratch as i had to look in the intel docs for every
instruction anyway to figure out whats wrong, so why not just look at
opcode details of opcodes being actually in use...  i borrowed optab
decode tables and the general structure from 8i but the rest is
different.  realemu implements the opcodes in a generic way without
all these switch(opsize){} cases in order to minimize code size.  and
it doesnt dereference pointers in the decode phase, but passes the
opcode handler a Iarg * that will be read or written with ar() or aw()
in the handler.

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 3361 bytes --]

From: Lucio De Re <lucio@proxima.alt.za>
To: 9fans@9fans.net
Subject: Re: [9fans] realemu update
Date: Mon, 7 Mar 2011 06:21:32 +0200
Message-ID: <cc646ef14d7954c699224c5004412ab6@proxima.alt.za>

> mkfile is fixed now. will install itself into /$objtype/bin/aux/realemu
> and install realemu (8) manpage.
>
> impovement on the manpage is welcome as my english is not so good :)

Again, I'm delaying testing because I need to install a different
video card in the available hardware, but I'm looking forward to doing
that.

I have installed the pertinent bits in preparation, we'll see what
happens.

Thanks for tidying up.  On my side what I'm still missing is a
description of #P/realmode and #P/realmodemem; realemu's manpage
suggests that these can be found in arch(3) but my copy of that man
page has nothing to say about it (not your problem, of course).

The realemu(8) man page is pretty adequate now, thank you for that,
too.

Would it make sense, in your opinion, to backport these changes to 8i
and have that added to the distribution as well?

++L

Re: [9fans] realemu update

[Lucio De Re]

On Mon, Mar 07, 2011 at 11:39:10AM +0100, cinap_lenrek@gmx.de wrote:
>
> the /dev/realmode intraface was not documented, but it is very simple.
>
Thank you for explaining this.

> /dev/realmodemem is just an image of the first megabyte of
> physical memory that is addressable from 16 bit realmode.
>
That being where the machine's BIOS resides, if memory serves.
Plus whatever can fit in there if one chooses to use the space.
I'll understand things better with the fesh background, thank you.

> plan9 reserves a 4k page at 0x9000 (defined as RMBUF) that can be
> refered to in the bios call as data buffer.  previously, this was the
> only offset range that could be written with /dev/realmodemem.
>
I think I get it, but I'll experiment with it to make sure.

> in /dev/realmode, you write a struct Ureg (from /386/include/ureg.h)
> (in x86 machine byte order?) containing the register contents and the
> interrupt number of the bios call you want to make.
>
> the write returns when the BIOS call returns and the machine
> state can be read back from /dev/realmode.
>
That is a neat idea.

> realemu did a little extension to the interface: it allows reading and
> writing the whole address space and in case the trap is zero in the
> Ureg, it will copy ss, sp, cs, and pc in the virtual cpu state too and not
> make a BIOS interrupt.  this is used by loadcom.c to run dos .com
> files in the emulator.
>
This bit is harder to get my mind around, but I'll explore the source
code to make better sense of it.  It all sounds very sensible.

> 8i was never in a working or finished state...

That suggests that you see no value in a working 8i implementation,
it's tempting to take your word for it :-)

All in all, there are only so many brain cycles available.

++L

Re: [9fans] realemu update

[cinap_lenrek]

> On Mon, Mar 07, 2011 at 11:39:10AM +0100, cinap_lenrek@gmx.de wrote:
> > /dev/realmodemem is just an image of the first megabyte of
> > physical memory that is addressable from 16 bit realmode.
> >
> That being where the machine's BIOS resides, if memory serves.
> Plus whatever can fit in there if one chooses to use the space.
> I'll understand things better with the fesh background, thank you.

yes, exactly. it also contains the data setup by the bios.

this is the memory map from realemu main.c, one entry for each
64k block:

static Bus memio[] = {
	/* 0 */ memory, rmem, wmem,	/* RAM: IVT, BIOS data area */
	/* 1 */ memory,	rmem, wmem,	/* custom */
	/* 2 */ nil,	rbad, wbad,
	/* 3 */ nil,	rbad, wbad,
	/* 4 */ nil,	rbad, wbad,
	/* 5 */ nil,	rbad, wbad,
	/* 6 */ nil,	rbad, wbad,
	/* 7 */ nil,	rbad, wbad,
	/* 8 */ nil,	rbad, wbad,
	/* 9 */ memory,	rmem, wmem,	/* RAM: extended BIOS data area */
	/* A */ memory,	rmem, wrealmem,	/* RAM: VGA framebuffer */
	/* B */ memory,	rmem, wrealmem,	/* RAM: VGA framebuffer */
	/* C */ memory,	rmem, wbad,	/* ROM: VGA BIOS */
	/* D */ nil,	rbad, wbad,
	/* E */ memory,	rmem, wbad,	/* ROM: BIOS */
	/* F */ memory,	rmem, wbad,	/* ROM: BIOS */
};

on startup, we read in all the readable memory from the host machine.
this means, that if the host machine is not a x86 with a bios that
mapped the roms for us it will not work.  to make this work have to do
the same thing the bios on a pc makes like setting up a realmode
interrupt table and discover and map in the vga roms.

> > the write returns when the BIOS call returns and the machine
> > state can be read back from /dev/realmode.
> >
> That is a neat idea.

yeah, its the power of filesystem abstraction all over again :)

also, with realemu, you can interrupt the write to /dev/realmode in
case the code hangs in a endless loop.  otherwise it will return if
the code comes back from the interrupt call to the bios or when the
code produces some kind of exception.  errstr will be set to a string
describing the exception and the exception number will be in u->trap
when reading /dev/realmode back.

--
cinap

Re: [9fans] realemu update

[erik quanstrom]

> in /dev/realmode, you write a struct Ureg (from /386/include/ureg.h)
> (in x86 machine byte order?) containing the register contents and the
> interrupt number of the bios call you want to make.

yes.  you should use libmach to do this dirty work.

- erik

Re: [9fans] realemu update

[Lucio De Re]

On Mon, Mar 07, 2011 at 07:23:36AM -0500, erik quanstrom wrote:
>
> > in /dev/realmode, you write a struct Ureg (from /386/include/ureg.h)
> > (in x86 machine byte order?) containing the register contents and the
> > interrupt number of the bios call you want to make.
>
> yes.  you should use libmach to do this dirty work.
>
Erik, what have you got in mind?  Philosophically, you are perfectly
correct, but the implementation would seem like overkill to me.  Maybe a
libmach function to build the registers from user-mode values, but
surely not additional complexities in /dev/realmode to set registers,
shall we say, from user-mode text values?

++L

Re: [9fans] realemu update

[Russ Cox]

On Mon, Mar 7, 2011 at 7:23 AM, erik quanstrom <quanstro@quanstro.net> wrote:
>> in /dev/realmode, you write a struct Ureg (from /386/include/ureg.h)
>> (in x86 machine byte order?) containing the register contents and the
>> interrupt number of the bios call you want to make.
>
> yes.  you should use libmach to do this dirty work.

huh?  what does libmach (which takes apart executables)
have to do with any of this?

the interface is well defined and portable: it is an x86 Ureg
struct in x86 byte order, because you are executing x86 code.

i think cinap's structuring of realemu as a file server is very
elegant, and i say that as the author of both 8i and /dev/realmode.
it's one of those things that is completely unexpected and yet
falls out quite naturally from the general plan 9 approach.
very very cool.

russ

Re: [9fans] realemu update

[Lucio De Re]

On Mon, Mar 07, 2011 at 09:19:58AM -0500, Russ Cox wrote:
>
> huh?  what does libmach (which takes apart executables)
> have to do with any of this?
>
Did I get the wrong impression when I perceived libmach, as released
with GoLang - cause that's where I looked - seemingly quite capable of
synthesising as well as analysing binary images?

++L

Re: [9fans] realemu update

[Russ Cox]

>> huh?  what does libmach (which takes apart executables)
>> have to do with any of this?
>>
> Did I get the wrong impression when I perceived libmach, as released
> with GoLang - cause that's where I looked - seemingly quite capable of
> synthesising as well as analysing binary images?

i think you did.
libmach has routines to read executables and object files (8.out and *.8)
but not to write them.  the programs that write them (8a, 8l, 8c, 8g) all
use custom code, mainly because they have their own internal
representations of the instruction stream, tailored to the kinds of
annotations they need.  8l even has its own reader for *.8 files,
for much the same reason.

russ

Re: [9fans] realemu update

[Stanley Lieber]

[-- Attachment #1: Type: text/plain, Size: 5605 bytes --]

With the latest realemu, graphics and rio are finally working in my
VMware Workstation 6.5.1 guest:

% aux/vga -m vesa -p
vesa flag            Ulinear|Hlinear
vesa sig            VESA 2.0
vesa oem            V M ware, Inc. VBE support 2.0 2.0
vesa vendor         VMware, Inc
vesa product        VMware virtual machine
vesa rev            2.0
vesa cap             8-bit-dac not-vga
vesa mem            134217728
vesa mode           0x100 640x400x8 m8 packed
vesa mode           0x101 640x480x8 m8 packed
vesa mode           0x103 800x600x8 m8 packed
vesa mode           0x105 1024x768x8 m8 packed
vesa mode           0x107 1280x1024x8 m8 packed
vesa mode           0x10e 320x200x16 r5g6b5 direct
vesa mode           0x111 640x480x16 r5g6b5 direct
vesa mode           0x114 800x600x16 r5g6b5 direct
vesa mode           0x117 1024x768x16 r5g6b5 direct
vesa mode           0x11a 1280x1024x16 r5g6b5 direct
vesa mode           0x120 320x200x8 m8 packed
vesa mode           0x121 320x400x8 m8 packed
vesa mode           0x122 640x400x8 m8 packed
vesa mode           0x123 640x480x8 m8 packed
vesa mode           0x124 800x600x8 m8 packed
vesa mode           0x125 1024x768x8 m8 packed
vesa mode           0x126 1152x864x8 m8 packed
vesa mode           0x127 1280x960x8 m8 packed
vesa mode           0x128 1280x1024x8 m8 packed
vesa mode           0x129 1400x1050x8 m8 packed
vesa mode           0x12a 1600x1200x8 m8 packed
vesa mode           0x12b 1792x1344x8 m8 packed
vesa mode           0x12c 1856x1392x8 m8 packed
vesa mode           0x12d 1920x1440x8 m8 packed
vesa mode           0x12e 320x200x16 r5g6b5 direct
vesa mode           0x12f 320x400x16 r5g6b5 direct
vesa mode           0x130 640x400x16 r5g6b5 direct
vesa mode           0x131 640x480x16 r5g6b5 direct
vesa mode           0x132 800x600x16 r5g6b5 direct
vesa mode           0x133 1024x768x16 r5g6b5 direct
vesa mode           0x134 1152x864x16 r5g6b5 direct
vesa mode           0x135 1280x960x16 r5g6b5 direct
vesa mode           0x136 1280x1024x16 r5g6b5 direct
vesa mode           0x137 1400x1050x16 r5g6b5 direct
vesa mode           0x138 1600x1200x16 r5g6b5 direct
vesa mode           0x139 1792x1344x16 r5g6b5 direct
vesa mode           0x13a 1856x1392x16 r5g6b5 direct
vesa mode           0x13b 1920x1440x16 r5g6b5 direct
vesa mode           0x13c 320x200x32 x8r8g8b8 direct
vesa mode           0x13d 320x400x32 x8r8g8b8 direct
vesa mode           0x13e 640x400x32 x8r8g8b8 direct
vesa mode           0x13f 640x480x32 x8r8g8b8 direct
vesa mode           0x140 800x600x32 x8r8g8b8 direct
vesa mode           0x141 1024x768x32 x8r8g8b8 direct
vesa mode           0x142 1152x864x32 x8r8g8b8 direct
vesa mode           0x143 1280x960x32 x8r8g8b8 direct
vesa mode           0x144 1280x1024x32 x8r8g8b8 direct
vesa mode           0x145 1400x1050x32 x8r8g8b8 direct
vesa mode           0x146 1600x1200x32 x8r8g8b8 direct
vesa mode           0x147 1792x1344x32 x8r8g8b8 direct
vesa mode           0x148 1856x1392x32 x8r8g8b8 direct
vesa mode           0x149 1920x1440x32 x8r8g8b8 direct
vesa mode           0x14a 1366x768x8 m8 packed
vesa mode           0x14b 1366x768x16 r5g6b5 direct
vesa mode           0x14c 1366x768x32 x8r8g8b8 direct
vesa mode           0x14d 1680x1050x8 m8 packed
vesa mode           0x14e 1680x1050x16 r5g6b5 direct
vesa mode           0x14f 1680x1050x32 x8r8g8b8 direct
vesa mode           0x150 1920x1200x8 m8 packed
vesa mode           0x151 1920x1200x16 r5g6b5 direct
vesa mode           0x152 1920x1200x32 x8r8g8b8 direct
vesa mode           0x153 2048x1536x8 m8 packed
vesa mode           0x154 2048x1536x16 r5g6b5 direct
vesa mode           0x155 2048x1536x32 x8r8g8b8 direct
vesa mode           0x156 320x240x8 m8 packed
vesa mode           0x157 320x240x16 r5g6b5 direct
vesa mode           0x158 320x240x32 x8r8g8b8 direct
vesa mode           0x159 400x300x8 m8 packed
vesa mode           0x15a 400x300x16 r5g6b5 direct
vesa mode           0x15b 400x300x32 x8r8g8b8 direct
vesa mode           0x15c 512x384x8 m8 packed
vesa mode           0x15d 512x384x16 r5g6b5 direct
vesa mode           0x15e 512x384x32 x8r8g8b8 direct
vesa mode           0x15f 854x480x8 m8 packed
vesa mode           0x160 854x480x16 r5g6b5 direct
vesa mode           0x161 854x480x32 x8r8g8b8 direct
vesa mode           0x162 1280x720x8 m8 packed
vesa mode           0x163 1280x720x16 r5g6b5 direct
vesa mode           0x164 1280x720x32 x8r8g8b8 direct
vesa mode           0x165 1920x1080x8 m8 packed
vesa mode           0x166 1920x1080x16 r5g6b5 direct
vesa mode           0x167 1920x1080x32 x8r8g8b8 direct
vesa mode           0x168 1280x800x8 m8 packed
vesa mode           0x169 1280x800x16 r5g6b5 direct
vesa mode           0x16a 1280x800x32 x8r8g8b8 direct
vesa mode           0x16b 1440x900x8 m8 packed
vesa mode           0x16c 1440x900x16 r5g6b5 direct
vesa mode           0x16d 1440x900x32 x8r8g8b8 direct
vesa mode           0x16e 720x480x8 m8 packed
vesa mode           0x16f 720x480x16 r5g6b5 direct
vesa mode           0x170 720x480x32 x8r8g8b8 direct
vesa mode           0x171 720x576x8 m8 packed
vesa mode           0x172 720x576x16 r5g6b5 direct
vesa mode           0x173 720x576x32 x8r8g8b8 direct
vesa mode           0x174 800x480x8 m8 packed
vesa mode           0x175 800x480x16 r5g6b5 direct
vesa mode           0x176 800x480x32 x8r8g8b8 direct
vesa mode           0x177 1280x768x8 m8 packed
vesa mode           0x178 1280x768x16 r5g6b5 direct
vesa mode           0x179 1280x768x32 x8r8g8b8 direct

-sl

[-- Attachment #2: Type: message/rfc822, Size: 3576 bytes --]

From: Russ Cox <rsc@swtch.com>
To: lucio@proxima.alt.za, Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net>
Subject: Re: [9fans] realemu update
Date: Mon, 7 Mar 2011 10:03:41 -0500
Message-ID: <AANLkTikfV7eHfjNCdCrzPqu0933Or0UdLF1VXPfEo5=p@mail.gmail.com>

>> huh?  what does libmach (which takes apart executables)
>> have to do with any of this?
>>
> Did I get the wrong impression when I perceived libmach, as released
> with GoLang - cause that's where I looked - seemingly quite capable of
> synthesising as well as analysing binary images?

i think you did.
libmach has routines to read executables and object files (8.out and *.8)
but not to write them.  the programs that write them (8a, 8l, 8c, 8g) all
use custom code, mainly because they have their own internal
representations of the instruction stream, tailored to the kinds of
annotations they need.  8l even has its own reader for *.8 files,
for much the same reason.

russ

Re: [9fans] realemu update

[Stanley Lieber]

Cinap suggested invoking realemu in a subshell so that the
process exits after aux/vga completes. The following is a patch
for the man page.

-sl


% diff -n -c /sys/man/8/realemu.orig /sys/man/8/realemu
/sys/man/8/realemu.orig:84,89 - /sys/man/8/realemu:84,102
  the
  .I srvname
  argument then it is ignored, otherwise it will used.
+ .SH EXAMPLES
+ The
+ .I realemu
+ process is only needed when accessing
+ .B /dev/realmode.
+ To invoke a subshell so that
+ .I realemu
+ exits normally after
+ .B aux/vga
+ completes:
+ .IP
+ .EX
+ % @{rfork n; aux/realemu; aux/vga -m vesa -l $vgasize}
  .SH SOURCE
  .B /sys/src/cmd/aux/realemu
  .SH "SEE ALSO"

[9fans] realemu update

[cinap_lenrek at gmx.de]

applied, thank you. also fixed some bugs in code:

properly convert the register values from the 386 ureg.

read back #P/realmodemem on every access as some graphics cards
use the a0000 segment as memory mapped i/o. (Matrox MILLENNIUM)

we dont use #v/vgabios anymore and rely on #P/realmodemem to
have the latest patches applied.

fix instruction formating bug.

always truncate physical memory addresses to 20bit.

--
cinap
-------------- next part --------------
An embedded message was scrubbed...
From: "Stanley Lieber" <stanley.lieber at gmail.com>
Subject: Re: [9fans] realemu update
Date: Tue, 8 Mar 2011 05:10:13 -0600
Size: 3150
URL: <http://mail.9fans.net/private/9fans/attachments/20110308/19764092/attachment.mht>