Re: [9fans] plan9 go output faults on 9vx with rfork

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

OK, Pavel sent me a nice piece of code that implements cmpswap using a
gcc trick. I did not want to use the trick for a few reasons, and
thought to use futex instead, as it seemed appropriate. Weirdly
enough, I can not find a simple implementation of cmpswap that uses
futex, though I can find several papers describing how tricky futex
is, and one Linux person who told me that futex had evolved in ways
not necessarily to our advantage.

Every time I'm ready to throw in the towel on Plan 9 and just go with
Linux I hit something like futex and it's back into the fray ... :-)

So, I have committed changes to my vx32 repo and Pavel's sieve now
works on 9vx on a 32-bit linux vmware image. It should also work if
you build on freebsd. It should also work if you build on 64-bit linux
but there are no guarantees, since all this relies on some gcc
builtins.

You can see what I've done if you look at my repo.

BTW, the sieve ran to 17xxx on linux and then I got some kind of
malloc failure, I'll assume it's another 32-bit gc issue. I'm up to
57917 on osx.

You're going to need Pavel's mods to go, I suppose. but that's another story ...

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 2893 bytes --]

On Wed, Jan 19, 2011 at 10:59 PM, ron minnich <rminnich@gmail.com> wrote:

> OK, Pavel sent me a nice piece of code that implements cmpswap using a
> gcc trick. I did not want to use the trick for a few reasons, and
> thought to use futex instead, as it seemed appropriate. Weirdly
> enough, I can not find a simple implementation of cmpswap that uses
> futex, though I can find several papers describing how tricky futex
> is, and one Linux person who told me that futex had evolved in ways
> not necessarily to our advantage.
>
> Every time I'm ready to throw in the towel on Plan 9 and just go with
> Linux I hit something like futex and it's back into the fray ... :-)
>

Actually the mutex implementations I've seen with futex use cmpswap.  Also
my understanding of futex is that it's just an API for a portion of what
needs to be done to implement a fast user space mutex. Essentially you have
to examine the value in user space, see that it's what you expect and move
on unblocked.  If the value differs from what is expected, you must block,
and wait.  This is why they're "tricky" :-).

I do not see anything immediately wrong with using the compiler intrinsic
functions that are supported pretty well across GNU, clang, Intel C/C++ and
I think even Microsoft's compilers.  I do think that gcc does not implement
them for i386, or if it does, it does so with a library routine instead of
the compiler just generating the assembly for that architecture's version.

I've run smack into this problem myself just about 8 months ago, and it was
due to Boost being built with a mismatched architecture from some other code
that was built with another.  The result was that since these builtins get
resolved sometimes at link time, and sometimes at compile time, they don't
behave like normal functions, and you get a mess.

You've got to pick one mess, or implement your own multi-architecture
library of cmpswap and like routines :-).


>
> So, I have committed changes to my vx32 repo and Pavel's sieve now
> works on 9vx on a 32-bit linux vmware image. It should also work if
> you build on freebsd. It should also work if you build on 64-bit linux
> but there are no guarantees, since all this relies on some gcc
> builtins.
>
> You can see what I've done if you look at my repo.
>
> BTW, the sieve ran to 17xxx on linux and then I got some kind of
> malloc failure, I'll assume it's another 32-bit gc issue. I'm up to
> 57917 on osx.
>

If your compiler is generating code for -march=i386, you could see some
funny stuff on 32bit platforms.  Better to use -march=i686.  Might not come
up, but it's something to look for.

I really need to try this version of 9vx sometime instead of just thinking
about it :-)

Dave


>
> You're going to need Pavel's mods to go, I suppose. but that's another
> story ...
>
> ron
>
>

[-- Attachment #2: Type: text/html, Size: 3690 bytes --]

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

This started to look like a 32/64 problem and I think it is. If you
trace the cmpswap, at some point we see 100000000 as a value, just
like somebody added 1 to ffffffff, and it went to, not 0, as expected,
but a bigger value.

See this code:

long
syssemacquire(uint32 *arg)
{
        int block;
        long *addr;
        Segment *s;

        addr = uvalidaddr(arg[0], sizeof(long), 1);
        evenaddr(arg[0]);
        block = arg[1];

        if((s = seg(up, arg[0], 0)) == nil)
                error(Ebadarg);
        if(*addr < 0)
                error(Ebadarg);
        return semacquire(s, addr, block);
}

And note how the u32int is actually converted to a pointer to a long,
which is 64 bits. I put a panic in there on sizeof(*addr) !=
sizeof(*arg) and bang! away we go.

So it's a 32/64 bit cleanliness problem. And it's not just for go,
it's for everything.

This is a huge problem as pentium is little-endian and you're going to
end up trashing data. Bad.

I will try to put a fix in now. We just need to be a little cleaner
about realizing that the syscall interface is always 32 bits, even
when 9vx is built as 64 bits. This is pretty much the TOS thing all
over again.

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[erik quanstrom]

> And note how the u32int is actually converted to a pointer to a long,
> which is 64 bits. I put a panic in there on sizeof(*addr) !=
> sizeof(*arg) and bang! away we go.
>
> So it's a 32/64 bit cleanliness problem. And it's not just for go,
> it's for everything.
>
> This is a huge problem as pentium is little-endian and you're going to
> end up trashing data. Bad.
>
> I will try to put a fix in now. We just need to be a little cleaner
> about realizing that the syscall interface is always 32 bits, even
> when 9vx is built as 64 bits. This is pretty much the TOS thing all
> over again.

the plan 9 core needs to religiously use uintptr and not long.  long
is always wrong for these things.

- erik

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

On Sat, Jan 22, 2011 at 6:48 PM, erik quanstrom <quanstro@quanstro.net> wrote:

> the plan 9 core needs to religiously use uintptr and not long.  long
> is always wrong for these things.

It's a bit messier than that. When 9vx is built as 64-bit, it's a
64-bit kernel supporting 32-bit binaries. uintptr is still a 64-bit
entity. There's no way to take a 32-bit reference parameter as used in
semacquire and treat it as a pointer to a 64-bit item and have it end
well ...

What has to happen is the 9vx syscall interfaces need to be scrubbed
just a bit more for these types of cases and corrected. Or,simpler,
just keep building 32-bit 9vx; that may be the best bet.

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[erik quanstrom]

> It's a bit messier than that. When 9vx is built as 64-bit, it's a
> 64-bit kernel supporting 32-bit binaries. uintptr is still a 64-bit
> entity. There's no way to take a 32-bit reference parameter as used in
> semacquire and treat it as a pointer to a 64-bit item and have it end
> well ...

right.  i think the situation confused me.  but i don't feel too bad.
the linux guys, with their 10s of thousands can't get this right.

> What has to happen is the 9vx syscall interfaces need to be scrubbed
> just a bit more for these types of cases and corrected. Or,simpler,
> just keep building 32-bit 9vx; that may be the best bet.

that makes more sense to me.

- erik

Re: [9fans] plan9 go output faults on 9vx with rfork

[Devon H. O'Dell]

2011/1/23 Charles Forsyth <forsyth@terzarima.net>:
> in the plan 9 world, a 64-bit kernel runs 64-bit applications,
> and 32-bit applications run on a 32-bit kernel.
> a 64-bit 9vx would run programs produced by 6[acl]
> (well, in principle: it would need to be derived from the 64-bit kernel)

It would, but vx32 still just "emulates" an i386. So even if 9vx is
built to run on amd64, the underlying Plan 9 environment still uses
8*.

--dho

Re: [9fans] plan9 go output faults on 9vx with rfork

[Charles Forsyth]

in the plan 9 world, a 64-bit kernel runs 64-bit applications,
and 32-bit applications run on a 32-bit kernel.
a 64-bit 9vx would run programs produced by 6[acl]
(well, in principle: it would need to be derived from the 64-bit kernel)

Re: [9fans] plan9 go output faults on 9vx with rfork

[EBo]

>> in the plan 9 world, a 64-bit kernel runs 64-bit applications,
>> and 32-bit applications run on a 32-bit kernel.
>
> It would, but vx32 still just "emulates" an i386. So even if 9vx is
> built to run on amd64, the underlying Plan 9 environment still uses
> 8*.

 So, what changes are needed to set it up for full 64bit?

   EBo --

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

On Sun, Jan 23, 2011 at 12:40 PM, EBo <ebo@sandien.com> wrote:
>>> in the plan 9 world, a 64-bit kernel runs 64-bit applications,
>>> and 32-bit applications run on a 32-bit kernel.
>>
>> It would, but vx32 still just "emulates" an i386. So even if 9vx is
>> built to run on amd64, the underlying Plan 9 environment still uses
>> 8*.
>
> So, what changes are needed to set it up for full 64bit?

not a huge number.

Implement oscmpswap32 for gccmagic/cmpswap.c

create a function pointer like cmpswap called cmpswap32
assigned oscmpswap32 to cmpswap32 in main.c

change all sem* bits in a/sysproc.c to use uint32 not long
change ed script so it won't do the wrong thing in future.

ugly, but I don't see a non-ugly way to do it.

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[Charles Forsyth]

> So, what changes are needed to set it up for full 64bit?

i'm confused. why would you bother if you can only run 32-bit x86 code in vx32?

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

On Sun, Jan 23, 2011 at 3:40 PM, Charles Forsyth <forsyth@terzarima.net> wrote:
>> So, what changes are needed to set it up for full 64bit?
>
> i'm confused. why would you bother if you can only run 32-bit x86 code in vx32?

I'm not sure either, unless there is some feeling that it runs better
as a 64-bit binary.

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[EBo]

>> So, what changes are needed to set it up for full 64bit?
>
> not a huge number.
>
> Implement oscmpswap32 for gccmagic/cmpswap.c
>
> create a function pointer like cmpswap called cmpswap32
> assigned oscmpswap32 to cmpswap32 in main.c
>
> change all sem* bits in a/sysproc.c to use uint32 not long
> change ed script so it won't do the wrong thing in future.
>
> ugly, but I don't see a non-ugly way to do it.

 Thanks for the general idea.  I assume that there is also a bunch of
 stuff in vx32, but that might be a different question.

   EBo --

Re: [9fans] plan9 go output faults on 9vx with rfork

[EBo]

 On Sun, 23 Jan 2011 17:18:30 -0800, ron minnich wrote:
> On Sun, Jan 23, 2011 at 3:40 PM, Charles Forsyth
> <forsyth@terzarima.net> wrote:
>>> So, what changes are needed to set it up for full 64bit?
>>
>> i'm confused. why would you bother if you can only run 32-bit x86
>> code in vx32?
>
> I'm not sure either, unless there is some feeling that it runs better
> as a 64-bit binary.

 Every once and awhile I get bit by a 32/64 bit compatibility issue, and
 was curious.  I like to keep general track of things so that I have some
 idea of how much work things are going to be if I get a wild hair and
 decide to tackle something.  Dealing with vx32 (to make it vx64 I guess)
 would probably be the hair puller.

   EBo --

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

apropos this discussion, I just "fixed" (maybe in the veterinary
sense?) my vx32 repo to only build 32-bit, regardless of what machine
one is on.

Tested on linux/32 and linux/64 and darwin

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[yy]

2011/1/23 ron minnich <rminnich@gmail.com>:
> change all sem* bits in a/sysproc.c to use uint32 not long
> change ed script so it won't do the wrong thing in future.

Just for the records, the ed scripts are not working with current
kernel sources, they need to be updated. I gave it a try, but
sometimes I started from the ed script and sometimes from the .c file.
I was just trying to make it work, but would be surprised if it worked
well (as a matter of fact, last time I tried to update the code using
the ed scripts I broke 9vx and it was a hassle for everyone, just see
this thread: http://9fans.net/archive/2010/09/190). In particular, the
use of uint32 or long is not uniform. The situation is even worse for
the ip code, there are no scripts at all. This is in my TODO list, but
I'm not working on it now and I don't know when (or if) I will.

--
- yiyus || JGL . 4l77.com

Re: [9fans] plan9 go output faults on 9vx with rfork

[erik quanstrom]

On Mon Jan 24 12:19:47 EST 2011, yiyu.jgl@gmail.com wrote:
> 2011/1/23 ron minnich <rminnich@gmail.com>:
> > change all sem* bits in a/sysproc.c to use uint32 not long
> > change ed script so it won't do the wrong thing in future.
>
> Just for the records, the ed scripts are not working with current
> kernel sources, they need to be updated. I gave it a try, but
> sometimes I started from the ed script and sometimes from the .c file.
> I was just trying to make it work, but would be surprised if it worked
> well (as a matter of fact, last time I tried to update the code using
> the ed scripts I broke 9vx and it was a hassle for everyone, just see
> this thread: http://9fans.net/archive/2010/09/190). In particular, the
> use of uint32 or long is not uniform. The situation is even worse for
> the ip code, there are no scripts at all. This is in my TODO list, but
> I'm not working on it now and I don't know when (or if) I will.

the type use should be uniform.  the only type that should change size
depending on the kernel is uintptr.  everything else is assumed fixed.
unfortunately, ulong is still used where uintptr should be.  it's fairly
rampant.  we should have

short	= 2 bytes
int
long	= 4 bytes
vlong	= 8 bytes

uintptr	= sizeof(void*)

note that long is almost completely useless in our type system, except
that we get to add a spurious ell to formats.

(i'm also not convinced of the utility of uint<bits>, but that's just me.)

- erik

Re: [9fans] plan9 go output faults on 9vx with rfork

[ron minnich]

On Mon, Jan 24, 2011 at 9:18 AM, yy <yiyu.jgl@gmail.com> wrote:
> 2011/1/23 ron minnich <rminnich@gmail.com>:
>> change all sem* bits in a/sysproc.c to use uint32 not long
>> change ed script so it won't do the wrong thing in future.
>
> Just for the records, the ed scripts are not working with current
> kernel sources, they need to be updated. I gave it a try, but
> sometimes I started from the ed script and sometimes from the .c file.


well, that is a bit depressing. Given the choice between bringing in
some code and having the ed scripts bring in the code, I guess the
choice is clear, and I would be inclined not to bring code in until
the script is fixed :-(

Can you please send me a list of ed scripts that are 'not working'.

ron

Re: [9fans] plan9 go output faults on 9vx with rfork

[Steve Simon]

> Just for the records, the ed scripts are not working with current
> kernel sources, they need to be updated.

Have you tried gcc 4.6? This includes Ian Lance-Taylor's changes which makes
gcc willing to accept kencc lovelyness. This could/should/may mean that
some/many/all of the ed scripts become irrelevant.

This should help p9p and drawterm as 4.6 becomes more widely used.

I have not had time to try this myself yet but hope to soon (ymmv)

-Steve

Re: [9fans] plan9 go output faults on 9vx with rfork

[Richard Miller]

> unfortunately, ulong is still used where uintptr should be.  it's fairly
> rampant.

"Because it descended from typeless languages, C has traditionally
been rather permissive in allowing dubious mixtures of various types;
the most flagrant violations of good practice involved the confusion
of pointers and integers."
-- Johnson & Ritchie: 'Portability of C Programs and the UNIX System'
   (BSTJ Jul-Aug 1978)

Re: [9fans] plan9 go output faults on 9vx with rfork

[yy]

2011/1/24 ron minnich <rminnich@gmail.com>:
> On Mon, Jan 24, 2011 at 9:18 AM, yy <yiyu.jgl@gmail.com> wrote:
>> 2011/1/23 ron minnich <rminnich@gmail.com>:
>>> change all sem* bits in a/sysproc.c to use uint32 not long
>>> change ed script so it won't do the wrong thing in future.
>>
>> Just for the records, the ed scripts are not working with current
>> kernel sources, they need to be updated. I gave it a try, but
>> sometimes I started from the ed script and sometimes from the .c file.
>
>
> well, that is a bit depressing. Given the choice between bringing in
> some code and having the ed scripts bring in the code, I guess the
> choice is clear, and I would be inclined not to bring code in until
> the script is fixed :-(
>
> Can you please send me a list of ed scripts that are 'not working'.
>

Hi ron, sorry for the late response, things have been busy around here.

The ed scripts in the a/ directory should work, and the resulting code
should compile, but that's it. Probably many things should be properly
fixed, I only tweaked it until it compiled. There are also ed scripts
in libdraw/, libmemdraw/, libmemlayer/ and libsec/ which I have not
looked at, and there should be scripts for the ip code too, but nobody
has written them yet.

I hope to have some time to look into this this week, although the
task is very big and surely I'm not going to be able to do everything
myself. Anyway, just wanted to be sure that you are not already
working on it.

--
- yiyus || JGL . 4l77.com